ALTO C. Griffiths
Internet-Draft J. Livingood, Ed.
Intended status: Informational Comcast
Expires: September 10, 2009 L. Popkin
Pando
R. Woundy, Ed.
Comcast
Y. Yang
Yale
March 9, 2009
Comcast's ISP Experiences In a P4P Technical Trial
draft-livingood-woundy-p4p-experiences-03
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 10, 2009.
Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents in effect on the date of
publication of this document (http://trustee.ietf.org/license-info).
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document.
Griffiths, et al. Expires September 10, 2009 [Page 1]
Internet-Draft Comcast P4P Experiences March 2009
Abstract
This document describes the experiences of Comcast, a large cable
broadband Internet Service Provider (ISP) in the U.S., in a Proactive
Network Provider Participation for P2P (P4P) technical trial in July
2008. This trial used iTracker technology being considered by the
IETF, as part of the Application Layer Transport Optimization (ALTO)
working group.
Table of Contents
1. Requirements Language . . . . . . . . . . . . . . . . . . . . . 3
2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. High-Level Details . . . . . . . . . . . . . . . . . . . . . . 3
4. High-Level Trial Results . . . . . . . . . . . . . . . . . . . 4
4.1. Impact on Downloads, or Downstream Traffic . . . . . . . . 5
4.2. Other Impacts and Interesting Data . . . . . . . . . . . . 5
5. Differences Between the P4P iTrackers Used . . . . . . . . . . 6
5.1. P4P Fine Grain . . . . . . . . . . . . . . . . . . . . . . 6
5.2. P4P Coarse Grain . . . . . . . . . . . . . . . . . . . . . 7
5.3. P4P Generic Weighted . . . . . . . . . . . . . . . . . . . 7
6. Next Steps . . . . . . . . . . . . . . . . . . . . . . . . . . 7
7. Security Considerations . . . . . . . . . . . . . . . . . . . . 8
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 8
10. Normative References . . . . . . . . . . . . . . . . . . . . . 9
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 9
Griffiths, et al. Expires September 10, 2009 [Page 2]
Internet-Draft Comcast P4P Experiences March 2009
1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
2. Introduction
Comcast is a large broadband ISP, based in the U.S., serving the
majority of its customers via cable modem technology. A trial was
conducted in July 2008 with Pando Networks, Yale, and several ISP
members of the P4P Working Group, which is part of the Distributed
Computing Industry Association (DCIA). Comcast is a member of the
P4P Working Group, whose mission is to work with Internet service
providers (ISPs), peer to peer (P2P) companies, and technology
researchers to develop "P4P" mechanisms that accelerate distribution
of content and optimize utilization of ISP network resources. P4P
theoretically allows P2P networks to optimize traffic within each
ISP, reducing the volume of data traversing the ISP's infrastructure
and creating a more manageable flow of data. P4P can also accelerate
P2P downloads for end users.
P4P's so-called "iTracker" technology was conceptually discussed with
the IETF at the Peer to Peer Infrastructure (P2Pi) Workshop held on
May 22, 2008, at the Massachusetts Institute of Technology (MIT).
This work was discussed in greater detail at the 72nd meeting of the
IETF, in Dublin, Ireland, in the ALTO BoF on July 29, 2008. Due to
interest from the community, Comcast shared P4P trial data at the
73rd meeting of the IETF, in Minneapolis, Minnesota, in the ALTO BoF
on November 18, 2008. Since that time, discussion of iTrackers and
alternative technologies has continued among participants of the ALTO
working group.
The P4P trial was conducted, in cooperation with Pando, Yale, and
three other P4P member ISPs, from July 2 to July 17, 2008. This was
the first P4P trial over a cable broadband network. The trial used a
Pando P2P client, and Pando distributed a special 21 MB licensed
video file as in order to measure the effectiveness of P4P iTrackers.
A primary objective of the trial was to measure the effects that
increasing the localization of P2P swarms would have on P2P uploads,
P2P downloads, and ISP networks, in comparison to normal P2P
activity.
3. High-Level Details
There were five different swarms for the content used in the trial.
Griffiths, et al. Expires September 10, 2009 [Page 3]
Internet-Draft Comcast P4P Experiences March 2009
The first was a random P2P swarm, as a control group. The second,
third, and fourth used different P4P iTrackers: Generic, Coarse
Grained, and Fine Grained. The fifth was a proprietary Pando
mechanism. (The results of the fifth swarm, while very good, are not
included here since our focus is on open standards and a mechanism
which may be leveraged for the benefit of the entire community of P2P
clients.) During the trial, there were 15,518 downloads to Comcast-
based P2P clients. Comcast deployed an iTracker server in our
production network to support this trial, and configured multiple
iTracker files to provide varying levels of localization to clients.
In the trial itself, a P2P client begins a P2P session by querying a
pTracker, which runs and manages the P2P network. The pTracker
occasionally queries the iTracker, which in this case was maintained
by Comcast, the ISP. Other ISPs either managed their own iTracker or
used Pando or Yale to host their iTracker files. The iTracker
returns network topology information to the pTracker, which then
communicates with P2P clients, in order to enable P2P clients to make
network-aware decisions regarding peers.
The Pando client was enabled to capture extended logging. The
logging included the source and destination IP address of all p2p
transfers, the number of bytes transferred, and the start and end
timestamps. This information gives a precise measurement of the data
flow in the network, allowing computation of data transfer volumes as
well as data flow rates at each point in time. Pando also captured
the start and completion times of every download, as well as the
average transfer rate observed by the client for the download.
Pando served the data from an origin server external to Comcast's
network. This server served about 10 copies of the file, after which
all transfers (about 1 million downloads) were performed purely via
P2P.
The P2P clients in the trial start with tracker-provided peers, then
use peer exchange to discover additional peers. Thus, the initial
peers were provided according to P4P guidance (90% guidance based on
P4P topology, and 10% random guidance), then later peers discover the
entire swarm via either additional announces or peer exchange.
4. High-Level Trial Results
Trial data was collected by Pando Networks and Yale University, and
raw trial results were shared with Comcast and all of the other ISPs
involved in the trial. Analysis of the raw results was performed by
Pando and Yale, and these organizations delivered an analysis of the
P4P trial. Using the raw data, Comcast also analyzed the trial
Griffiths, et al. Expires September 10, 2009 [Page 4]
Internet-Draft Comcast P4P Experiences March 2009
results. Furthermore, the raw trial results for Comcast were shared
with Net Forecast, Inc., which performed an independent analysis of
the trial for Comcast.
4.1. Impact on Downloads, or Downstream Traffic
The results of the trial indicated that P4P can improve the speed of
downloads to P2P clients. In addition, P4P was effective in
localizing P2P traffic within the Comcast network.
Impact of P4P on Downloads:
+--------------+------------+------------+-------------+------------+
| Swarm | Global Avg | Change | Comcast Avg | Change |
| | bps | | bps | |
+--------------+------------+------------+-------------+------------+
| Random | 144,045 | n/a | 254,671 bps | n/a |
| (Control) | bps | | | |
| ---------- | ---------- | ---------- | ---------- | ---------- |
| P4P Fine | 162,344 | +13% | 402,043 bps | +57% |
| Grained | bps | | | |
| ---------- | ---------- | ---------- | ---------- | ---------- |
| P4P Generic | 163,205 | +13% | 463,782 bps | +82% |
| Weight | bps | | | |
| ---------- | ---------- | ---------- | ---------- | ---------- |
| P4P Coarse | 166,273 | +15% | 471,218 bps | +85% |
| Grained | bps | | | |
+--------------+------------+------------+-------------+------------+
Table 1: Data Collected with Pando Networks and Yale University
4.2. Other Impacts and Interesting Data
An analysis of the effects of P4P on upstream utilization and
Internet transit was also interesting. It did not appear that P4P
significantly increased upstream utilization in our access network;
in essence uploading was already occurring no matter what and P4P in
and of itself did not appear to materially increase uploading for
this specific, licensed content. (P4P is not intended as a solution
for the potential of network congestion to occur.) Random was
143,236 MB and P4P Generic Weight was 143,143 MB, while P4P Coarse
Grained was 139,669 MB. We also observed that P4P reduced outgoing
Internet traffic by an average of 34% at peering points. Random was
134,219 MB and P4P Generic Weight was 91,979 MB, while P4P Coarse
Grained was 86,652 MB.
In terms of downstream utilization, we observed that P4P reduced
incoming Internet traffic by an average of 80% at peering points.
Griffiths, et al. Expires September 10, 2009 [Page 5]
Internet-Draft Comcast P4P Experiences March 2009
Random was 47,013 MB and P4P Generic Weight was 8,610 MB, while P4P
Coarse Grained was 7,764 MB. However, we did notice that download
activity in our access network increased somewhat, from 56,030 MB for
Random, to 59,765 MB for P4P Generic Weight, and 60,781 MB for P4P
Coarse Grained. Upon further investigation, we noticed that 1) there
were 2% to 7% more downloads started of the optimized swarms than the
'random' swarm, and 2) the optimized swarms had a lower cancel rate
(1.77% to 2.59%) than the random swarm (3.17%).
During the trial, downloads peaked at 24,728 per day, per swarm, or
nearly 124,000 per day for all five swarms. The swarm size peaked at
11,703 peers per swarm, or nearly 57,000 peers for all five swarms.
We observed a comparable number of downloads in each of the five
swarms.
5. Differences Between the P4P iTrackers Used
Given the size of the Comcast network, it was felt that in order to
truly evaluate the iTracker application we would need to test various
network topologies that reflected our network and would help gauge
the level of effort and design requirements necessary to get correct
statistical data out of the trial. In all cases, iTrackers were
configured with automation in mind, so that any successful iTracker
configuration would be automatically updating, rather than manually
configured on an on-going basis. All iTrackers were hosted on the
same small server, and it appeared to be relatively easy and
inexpensive to scale up an iTracker infrastructure should P4P-like
mechanisms become standardized and widely adopted.
5.1. P4P Fine Grain
The Fine Grain topology was the first and most complex iTracker that
we built for this trial. It was a detailed mapping of Comcast
backbone-connected network Autonomous System Numbers (ASN) to IP
Aggregates which were weighted based on priority and distance from
each other. Included in this design was a prioritization of all Peer
and Internet transit connected ASNs to our backbone to ensure that
P4P traffic would prefer settlement free and lower cost networks
first, and then more expensive transit links. This attempted to
optimize and lower transit costs associated with this traffic. We
then took the additional step of detailing each ASN and IP aggregate
into IP subnets down to our Optical Transport Nodes (OTN) where all
Cable Modem Termination Systems (CMTS) reside. This design gave a
highly localized and detailed description of our network for the
iTracker to disseminate. This design defined 1,182 iTracker node
identifiers, and resulted in a 210,727 line configuration file.
Griffiths, et al. Expires September 10, 2009 [Page 6]
Internet-Draft Comcast P4P Experiences March 2009
This iTracker was obviously the most time-consuming to create and the
most complex to maintain. Trial results indicated that this level of
localization was too high, and was less effective compared to lower
levels of localization.
5.2. P4P Coarse Grain
Given the level of detail in the Fine Grain design, it was important
that we also enable a high-level design which still used priority and
weighting mechanisms for our backbone and transit links. The Coarse
Grain design was a limited or summarized version of the Fine Grain
design, which used the ASN to IP Aggregate and weighted data for
transit links, but removed all additional localization data. This
insured we would get similar data sets from the Fine Grain design,
but without the more detailed localization of each of our networks
off of our backbone. This design defined 22 iTracker node
identifiers, and resulted in a 1,461 line configuration file.
From an overall cost, complexity, risk, and effectiveness standpoint,
this was judged to be the optimal iTracker for Comcast. Importantly,
this did not require revealing the complex, internal network topology
that the Fine Grain did. Updates to this iTracker were also far
simpler to automate, which will better ensure that it is accurate
over time, and keeps administrative overhead relatively low.
However, the differences, costs, and benefits of Coarse Grain and
Generic Weighted (see below) likely merit further study.
5.3. P4P Generic Weighted
The Generic Weighted design was a copy of the Coarse Grained design
but instead of using our ISP-designated priority and weights, all
weights were defaulted to pre-determined parameters that the Yale
team had designed. All other data was replicated from the Coarse
Grain design. Providing the information necessary to support the
Generic Weighted iTracker was roughly the same as for Coarse Grain.
6. Next Steps
One objective of this document is to share with the IETF community
the results of one P4P trial in a large broadband network, given
skepticism regarding the benefits to P2P users as well as to ISPs.
From the perspective of P2P users, P4P potentially delivers faster
P2P downloads. At the same time, ISPs can increase the localization
of swarms, enabling them to reduce bytes flowing over transit points,
while also delivering an optimized P2P experience to customers.
However, an internal analysis of varying levels of iTracker adoption
by ISPs leads us to believe that, while P4P-type mechanisms are
Griffiths, et al. Expires September 10, 2009 [Page 7]
Internet-Draft Comcast P4P Experiences March 2009
valuable on a single ISP basis, the value of P4P increases
dramatically as many ISPs choose to deploy it.
We believe these results can inform the technical discussion in the
IETF over how to use iTracker mechanisms. Should such a mechanism be
standardized, the use of ISP-provided iTrackers should probably be an
opt-in feature for P2P users, or at least a feature of which they are
explicitly aware of and which has been enabled by default in a
particular P2P client. In this way, P2P users could choose to opt-in
either explicitly or by their choice of P2P client in order to choose
to use the iTracker to improve performance, which benefits both the
user and the ISP at the same time. Importantly in terms of privacy,
the iTracker makes available only network topology information, and
would not in its current form enable an ISP, via the iTracker, to
determine what P2P clients were downloading what content.
It is also possible that an iTracker type of mechanism, in
combination with a P2P cache, could further improve P2P download
performance, which merits further study. In addition, this was a
limited trial that, while very promising, indicates a need for
additional technical investigation and trial work. Such follow-up
study should explore the effects of P4P when more P2P client software
variants are involved, with larger swarms, and with additional and
more technically diverse content (file size, file type, duration of
content, etc.).
7. Security Considerations
There are no security considerations to include at this time.
8. IANA Considerations
There are no IANA considerations in this document.
9. Acknowledgements
The authors wish to acknowledge the hard work of all of the P4P
working group members, and specifically the focused efforts of the
teams at both Pando and Yale for the trial itself. Finally, the
authors recognize and appreciate Peter Sevcik and John Bartlett, of
NetForecast, Inc., for their valued independent analysis of the trial
results.
Griffiths, et al. Expires September 10, 2009 [Page 8]
Internet-Draft Comcast P4P Experiences March 2009
10. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
Authors' Addresses
Chris Griffiths
Comcast Cable Communications
One Comcast Center
1701 John F. Kennedy Boulevard
Philadelphia, PA 19103
US
Email: chris_griffiths@cable.comcast.com
URI: http://www.comcast.com
Jason Livingood (editor)
Comcast Cable Communications
One Comcast Center
1701 John F. Kennedy Boulevard
Philadelphia, PA 19103
US
Email: jason_livingood@cable.comcast.com
URI: http://www.comcast.com
Laird Popkin
Pando Networks
520 Broadway Street
10th Floor
New York, NY 10012
US
Email: laird@pando.com
URI: http://www.pando.com
Griffiths, et al. Expires September 10, 2009 [Page 9]
Internet-Draft Comcast P4P Experiences March 2009
Richard Woundy (editor)
Comcast Cable Communications
27 Industrial Avenue
Chelmsford, MA 01824
US
Email: richard_woundy@cable.comcast.com
URI: http://www.comcast.com
Richard Yang
Yale University
51 Prospect Street
New Haven, CT 06520
US
Email: yry@cs.yale.edu
URI: http://www.cs.yale.edu
Griffiths, et al. Expires September 10, 2009 [Page 10]
