Internet Measurement System
draft-ooki-lmap-internet-measurement-system-00
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft whose latest revision state is "Expired".
|
|
|---|---|---|---|
| Authors | m.ooki@ntt.com , Satoshi Kamei | ||
| Last updated | 2014-07-04 | ||
| RFC stream | (None) | ||
| Formats | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-ooki-lmap-internet-measurement-system-00
INTERNET-DRAFT Motoyuki Ooki
Satoshi Kamei
Intended Status: Informational NTT Communications
Expires: January 5, 2014 July 4, 2014
Internet Measurement System
draft-ooki-lmap-internet-measurement-system-00
Abstract
This document describes an experience of Japanese Internet
measurement system to measure Internet performance. We have developed
the system toward the enhancement of the network performance in an
ISP since October 2013. The considerations about the Internet
measurement are introduced along with our current status. This
document is expected to be useful for the standardization of Internet
measurements.
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/1id-abstracts.html
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
Copyright and License Notice
Copyright (c) 2014 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Ooki, et al. Expires January 5, 2014 [Page 1]
INTERNET DRAFT Internet Measurement System July 4, 2014
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Motivation of Internet Measurement . . . . . . . . . . . . . . 3
3. The framework of Internet Measurement System . . . . . . . . . 4
3.1 Measurement Agent . . . . . . . . . . . . . . . . . . . . . 4
3.2 Master Server . . . . . . . . . . . . . . . . . . . . . . . 4
3.3 Architecture . . . . . . . . . . . . . . . . . . . . . . . 5
3.4 Performance Metrics . . . . . . . . . . . . . . . . . . . . 6
4. The Operation of Internet Measurement System . . . . . . . . . 7
4.1 Measurement Target Contents . . . . . . . . . . . . . . . . 7
4.2 Measurement Schedule . . . . . . . . . . . . . . . . . . . 7
4.3 Application of Measurement . . . . . . . . . . . . . . . . 8
5. The problems of Internet Measurement System . . . . . . . . . 8
6. Security Considerations . . . . . . . . . . . . . . . . . . . 9
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 10
Ooki, et al. Expires January 5, 2014 [Page 2]
INTERNET DRAFT Internet Measurement System July 4, 2014
1 Introduction
In Japan, it is common to use a high speed Internet such as 100Mbps
and 1Gbps as an ISP's customer connection. Users only know the
maximum bandwidth of the last one mile for the ISP connection. The
maximum bandwidth value is ranging from 100Mbps to 2Gbps in ISP's
price plan as a FTTH connection.
Of course the performance of actual Internet connection is below the
bandwidth value. Internet users can obtain actual performance depends
on various ISP conditions such as congestions. Internet users don't
know the performance of the actual network.
On the other hand, ISPs also don't know the quality that Internet
users experience. For the ISP's point of view, it is important to
understand the service quality for its customers in order to design
its network properly. For this reason, it is necessary to measure the
actual performance of typical Internet users.
The Large-Scale Measurement of Broadband Performance (LMAP) working
group is formed to standardize a large scale measurement system to
measure broadband network performance. The LMAP WG does not focus on
the measurement of global Internet at the moment. However, we believe
that either way someday it will be necessary to establish a method
for Internet measurement and the standardization of the end to end
performance measurement, that is not closed to a certain ISP.
This document describes the system and our current status for
Internet measurement. We have measured the Internet performance by
using Internet measurement system we have been operating since
October 2013. We expect the experience of our case can contribute to
the standardizations in LMAP.
2. Motivation of Internet Measurement
There are two reasons that ISPs want to measure the actual
performance of its access services.
First, ISPs want to keep the customer satisfaction. Typically ISPs
provide the list of maximum bandwidth and the service prices, such as
the estimated total fee and the discount rate after the result of a
cash back campaign. Japanese users select a ISP based on only those
information without knowing the actual Internet performance results.
The poor performance causes the lower customer satisfaction.
Second, contents providers are beyond the control of ISPs. The
traffic volume of Contents Delivery Network (CDN) providers such as
Ooki, et al. Expires January 5, 2014 [Page 3]
INTERNET DRAFT Internet Measurement System July 4, 2014
AKAMAI and LEVEL3 is increasing in the Internet in recent years. How
much users are connecting to which contents providers impact the
actual performance. ISPs need to understand their behavior to decide
ISP's strategies and operation.
Hence, ISPs should measure Internet performances from Internet users
to multiple content providers. Then, ISPs can show a performance of
the actual network to build brand value compared with other
providers.
Sandvine and Google corporation presented the reports about Internet
traffic and ISP performance based on each criterion recently
[Sandvine Report][Google Report]. The Google report presents the
results of multiple ISPs measured in for each locations in USA. These
reports will have huge impacts on user's choice of the ISP selection.
It is better for an ISP to investigate and comprehend the status of
Internet performance between ISPs.
3. The framework of Internet Measurement System
We introduce the framework of Internet Measurement System in this
section. The words, such as Measurement Agent, Controller, and
Collector conforms to the glossary of the LMAP document (cite draft-
ietf-lmap-framework).
3.1 Measurement Agent
The MA has the functions that receive instructions from the Master
Server (described below), perform measurement tasks, and send the
measurement results to the Master Server. We used a Japanese product,
called OpenBlocks [OpenBlocks], which is the Linux box with Dual Core
Marvell ARMADA XP 1.33GHz, 1GB SDRAM memory. We selected the box as
the MA because of the affordable price, software stability, small
form factor, flexible functionally, and extendability. The MA needs
some CPU power in order to connect PPPoE access line. Using the box
we can perform regular measurements, and can configure tasks and
tools easily.
We have distributed MAs on many places all over Japan. The number of
locations is approximately 130 in June 2014. MAs are located in
houses where the residents can respond our requests (e.g., not
turning off the power to constantly perform the measurement) to
manipulate the device.
3.2 Master Server
The Master Server is a Linux server. The Master Server has the
Ooki, et al. Expires January 5, 2014 [Page 4]
INTERNET DRAFT Internet Measurement System July 4, 2014
functions to instruct the MAs. The Master Server connects to a MA via
a VPN connection. If we want to add a particular measurement task to
a MA, we can access through VPN and configure the measurement task.
Upon completion of the measurement task, the MA returns the results
to the Master Server. The Master Server in turn sends the results to
the database server at regular intervals.
3.3 Architecture
The architecture of the measurement system is composed of Linux box
MAs and the Linux Master Server.
+-------------+ +------------+ +------------+
| Master | Sync | | | |
| | -----> | Database | -----> | Analyzer |
| Server | | | | |
+-------------+ +------------+ +------------+
^ |
| | Instruction
|Send Data
|-------++-----------------++--------------------++
|| || ||
|v |v |v
+-------------+ +-------------+ +------------+
| Measurement | | Measurement | | Measurement|
| Agents | | Agents | | Agents |
+-------------+ +-------------+ +------------+
| ^ | ^ | ^
| | Active | | Obtain | |
| | Measurement | | Results | |
v | v | v |
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+
| |
| FTTH Access Line |
| |
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+
| ^
| |
| |
v |
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+
| |
| ISP network |
| |
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+
Ooki, et al. Expires January 5, 2014 [Page 5]
INTERNET DRAFT Internet Measurement System July 4, 2014
| ^
| |
| |
v |
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+
| |
| Internet |
| |
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+
| ^ | ^
| | | |
| | | |
v | v |
+--------------------------+ +--------------------------+
| Target Contents | | Target Contents |
+--------------------------+ +--------------------------+
Figure 1: Architecture of the Internet Measurement System
When a MA is powered on, it tries to establish the FTTH access PPPoE
connection with the ISP. After obtaining an IP address, it
automatically sets up a VPN Tunnel to the Master Server.
The MA prepare for the measurement tasks that is configured
beforehand, performs the tasks for Target Contents actively, and
collects the measurement results.
After the completion of the measurement tasks, the MA sends the
measurement results using SSH protocol through the VPN connection.
The measurement tasks are performed for each of multiple ISPs. The
list of ISPs is pre-configured in a file (called an ISP file). The MA
repeatedly connects to an ISP and performs the measurement, for each
ISP.
The measurement tasks and the ISP file are script files of Shell and
Perl programming language. The script files are set up on crontab.
The MAs can autonomously perform active measurement tasks without any
control sequence from the Master Server.
When the specification of the LMAP WG's protocol and framework is
finished, we will deploy the protocol in our measurement system.
3.4 Performance Metrics
The MAs perform active measurements for Target Contents at a regular
interval. Examples of the Target Contents include Video Streaming
Ooki, et al. Expires January 5, 2014 [Page 6]
INTERNET DRAFT Internet Measurement System July 4, 2014
files and OS update files. The regular interval is currently thirty
minutes.
In our measurements, performance metrics are below.
Round Trip Time (RTT) : This is the time between the submission of
the ICMP echo request packet and the reception of the ICMP echo reply
packet. This is measured by the ping command. We took the min/avg/max
time and the loss rate. The metric can also be regarded as the
network latency.
Hop Count : This metric refers to the number of intermediate devices
(like routers) through which the data must pass between the MA and
the Target Contents. This metric is regarded as the network distance
between the MA and the Target Contents. This is measured by the
traceroute command. Afterwards, by checking the hop counts, we can
find the change of the network routing on the Internet.
Throughput : This metric refers to how much data can be transferred
from the MA to the Target Contents in a given amount of time. This is
measured by the wget command.
4. The Operation of Internet Measurement System
We introduce the operation of Internet Measurement System we have
been operating since October 2013. in this section.
4.1 Measurement Target Contents
The selection of the Target Contents is important for the Internet
measurement; the type, the length, and the number of the contents. We
need to measure the representative contents on the Internet. In order
to find such contents, we select contents based on the volume of
transferred data of network traffic in an ISP.
We obtained partial traffic data on multiple prefectures in Japan. We
selected the Target Contents which were higher in the transferred
traffic volume ranking. Our Target Contents are Youtube Video
Streaming files and Mac OS update file on AKAMAI currently.
4.2 Measurement Schedule
MAs start measurement tasks every thirty minutes. The measurements
take some time (about five minutes) because the MA connects to the
access line through PPPoE authentication, connects a VPN connection
to the Master Server, and performs measurement based on our
performance metrics on as many Target Contents as pre-configured.
When there are many ISPs pre-configured, it will take longer time. We
Ooki, et al. Expires January 5, 2014 [Page 7]
INTERNET DRAFT Internet Measurement System July 4, 2014
used 300 seconds for wget time out, and 60 seconds for time out of
transferring the results (by scp). When the measurement do not
complete before the next scheduled time, the MA kills the measurement
process and moves to the next measurement.
4.3 Application of Measurement
Using the system we have studied the difference of network
performance between Japanese ISPs, based on the combinations of
Target Contents, measurement time, and areas. We are going to take
the measurement results in consideration to ISP network design and
ISP operation as a reference information.
We developed the portal site that publishes the analysis of the
measurement results. The site is used to improve the quality of daily
ISP operation.
5. The problems of Internet Measurement System
We introduce the problems of Internet Measurement System we have been
operating in this section.
- IPv6 Support
IPv6 network is constructed totally independently from IPv4 network.
Hence, the performance of the IPv6 network is highly likely different
from that of the IPv4 network.
Although the IPv6 network is not the majority yet, it is growing. NTT
EAST and WEST provided only 2.7% in NGN (Next Generation Network) on
December 2013. The rate of IPv6 enabled network in Japan is 27% in
June 2014[IPv6-Promotion Council]. NTT EAST and WEST presented the
IPv6 support in PPPoE connection on March 2014. All CPE devices for
NTT access line already support IPv6 tunneling, allowing users to
adapt IPv6 easily.
In order to achieve the broad applicability of our measurement
results, we will need to investigate the IPv6 performance also.
- Selection of Measurement Target Contents
It is difficult to decide what contents should be measured to present
the representative performance. There are many kinds of contents on
the Internet.
This time we have selected the Target Contents based on the volume of
transferred data at some points in an ISP. However, there are more
metrics to consider, such as the number of accesses to that contents,
Ooki, et al. Expires January 5, 2014 [Page 8]
INTERNET DRAFT Internet Measurement System July 4, 2014
rather than the transferred volume. Other metrics are not studied in
this document.
- Scalability
The Master Server is connected from multiple MAs via VPN connections.
This means that the Master Server needs to hold as many VPN
connections as the number of MAs. The number of MAs can easily grow
beyond the number of VPN connections that a Master server can hold.
If we place hundreds of MAs all over Japan, we will need to improve
the scalability of our system.
- Stable Operation
We had experiences where the measurement results were not sent
immediately, and the measurements for some Target Contents were
failed. Although the actual causes of these difficulties vary (e.g.,
accidentally disconnected LAN cable or power cable), we could easily
respond to those issues using informations (e.g., time and place)
contained in the centralized logs in the Master Server. Another
difficulty is the change in the settings of the contents provider.
For example, wget command for a video content has not worked due to a
change in a setting in the contents provider. This problem is
difficult to tackle and is left for future work.
6. Security Considerations
This section describes security consideration for Internet
measurement. We placed approximately 130 MAs all over Japan. These
MAs may become DDoS attackers by wrong commands from the Master
Server. From this reason, the list of commands MAs can perform should
be restricted. And also, the MAs must deny illegal accesses and
logins. MAs should permit only access through VPN connection from the
Master Server.
7. IANA Considerations
No need to describe any request regarding number assignment.
8. References
[Sandvine Report] https://www.sandvine.com/pr/2014/5/14/sandvine-
report-netflix%E2%80%99s-british-invasion.html
[Google Report] http://www.google.com/get/videoqualityreport/
[OpenBlocks] http://openblocks.plathome.com/products/ax3/
Ooki, et al. Expires January 5, 2014 [Page 9]
INTERNET DRAFT Internet Measurement System July 4, 2014
[IPv6-Promotion Council] http://v6pc.jp/jp/spread/ipv6spread_02.phtml
Authors' Addresses
Motoyuki Ooki
NTT Communications
3-4-1 Shibaura, Minato-ku, Tokyo
108-8118, Japan
EMail: m.ooki@ntt.com
Satoshi Kamei
NTT Communications
3-4-1 Shibaura, Minato-ku, Tokyo
108-8118, Japan
EMail: skame@nttv6.jp
Ooki, et al. Expires January 5, 2014 [Page 10]