DiffServ Applied to Real-time Transports D. York, Ed.
Internet-Draft Internet Society
Intended status: Standards Track D. Black, Ed.
Expires: December 8, 2014 EMC
C. Jennings
P. Jones
Cisco
June 6, 2014
Differentiated Services (DiffServ) and Real-time Communication
draft-york-dart-dscp-rtp-00
Abstract
This document describes the interaction between Differentiated
Services (DiffServ) network quality of service (QoS) functionality
and real-time network communication, including communication based on
the Real-time Transport Protocol (RTP). DiffServ is based on network
nodes applying different forwarding treatments to packets whose IP
headers are marked with different DiffServ Code Points (DSCPs). As a
result, use of different DSCPs within a single traffic flow may cause
transport protocol interactions (e.g., due to reordering). In
addition, DSCP markings may be changed or removed between the traffic
source and destination. This document covers the implications of
these DiffServ aspects for real-time network communication, including
RTCWEB.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on December 8, 2014.
York, et al. Expires December 8, 2014 [Page 1]
Internet-Draft DiffServ and RT Communication June 2014
Copyright Notice
Copyright (c) 2014 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3
2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1. Differentiated Services (DiffServ) . . . . . . . . . . . 4
2.2. Diffserv PHBs (Per-Hop Behaviors) . . . . . . . . . . . . 6
2.3. DiffServ and Transport Protocols . . . . . . . . . . . . 7
2.4. Traffic Classifiers and DSCP Remarking . . . . . . . . . 8
3. RTP Multiplexing Background . . . . . . . . . . . . . . . . . 9
4. Recommendations . . . . . . . . . . . . . . . . . . . . . . . 10
5. RTCWEB Examples . . . . . . . . . . . . . . . . . . . . . . . 10
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 11
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11
8. Security Considerations . . . . . . . . . . . . . . . . . . . 11
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 11
9.1. Normative References . . . . . . . . . . . . . . . . . . 11
9.2. Informative References . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13
1. Introduction
This document describes the interactions between Differentiated
Services (DiffServ) network quality of service (QoS) functionality
[RFC2475] and real-time network communication, including
communication based on the Real-time Transport Protocol (RTP)
[RFC3550]. DiffServ is based on network nodes applying different
forwarding treatments to packets whose IP headers are marked with
different DiffServ Code Points (DSCPs)[RFC2474]. As a result use of
different DSCPs within a single traffic stream may cause transport
protocol interactions (e.g., due to reordering). In addition, DSCP
markings may be changed or removed between the traffic's source and
destination. This document covers the implications of these DiffServ
York, et al. Expires December 8, 2014 [Page 2]
Internet-Draft DiffServ and RT Communication June 2014
aspects for real-time network communication, including RTCWEB traffic
[I-D.ietf-rtcweb-overview].
1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
2. Background
Real-time communications enables communication in real-time over an
IP network using communication modalities, such as voice, video,
text, content sharing, etc. It is possible to utilize any one or
more modalities in parallel in order to provide for a richer
communication experience.
A simple example of real-time communications is a voice call placed
over the Internet wherein an audio flow is transmitted in each
direction between two users. A more complex example is an immersive
videoconferencing system that has multiple video screens, multiple
cameras, multiple microphones, and some means of sharing content.
For such complex systems, there may be multiple media flows that may
be transmitted via a single IP address and port or via multiple IP
addresses and ports.
The most common protocol used for real time media is the Real-Time
Transport Protocol (RTP)[RFC3550]. RTP defines the mechanism by
which real-time data is transmitted between hosts on the Internet.
With most applications, a single media type (e.g., audio) is
transmitted within a single RTP session. However, it is possible to
transmit multiple, distinct media flows over the same RTP session as
individual RTP packet streams. This is referred to as RTP
multiplexing.
For the purposes of this draft, the term "media flow" refers to a
sequence of packets that is transmitted as a single RTP packet
stream.
Other transport protocols may also be used to transmit real-time data
or near-real-time data. For example, SCTP might be utilized to carry
application sharing or whiteboarding information as part of an
overall interaction that includes real time media flows. These
additional transport protocols can be multiplexed with one or more
RTP sessions via UDP encapsulation, thereby using a single pair of
UDP ports. The RTCWEB protocol suite [I-D.ietf-rtcweb-transports]
employs two layers of multiplexing:
York, et al. Expires December 8, 2014 [Page 3]
Internet-Draft DiffServ and RT Communication June 2014
1. Individual media flows are carried in individual RTP packet
streams that can multiplexed into a single RTP session (for
RTCWEB,an individual media flow is a MediaStreamTrack, and a
MediaStream may contain multiple MediaStreamTracks
[W3C.WD-mediacapture-streams-20130903]); and
2. One or more RTP session(s) multiplexed could be multiplexed with
other transport protocols via UDP encapsulation over a common
pair of UDP ports. The resulting unidirectional UDP flow is
uniquely identified by a 5-tuple, i.e., a combination of two IP
addresses (source and destination), two UDP ports (source and
destination), and the use of the UDP protocol.
For more information on use of RTP in RTCWEB, see .
[I-D.ietf-rtcweb-rtp-usage].
The number of media and other transport flows in an overall real-time
interaction can be surprisingly large. In addition to a voice flow
and a video flow, there could be separate media flows for each of the
cameras or microphones on a videoconferencing system. For each of
those video flows, and especially for layered video codecs, there
might be flows that carry spatial and temporal data separately from
the base layer. There might also be a separate flow that provides
protection to a media flow, using techniques such as Forward Error
Correction or redundancy. Still another example is simulcast
transmission, where a video flow can be transmitted at high
resolution and low resolution at the same time. In this case, a
media processing function might choose to send one or both flows
downstream to a receiver based on bandwidth availability or who the
active speaker is in a multipoint conference. Lastly, a transmitter
might send a the same media content concurrently as two media flows
using different encodings (e.g., VP8 in parallel with H.264) to allow
a media processing function to select a media encoding that best
matches the capabilities of the receiver.
2.1. Differentiated Services (DiffServ)
The DiffServ architecture is intended to enable scalable service
discrimination in the Internet without requiring per-flow state and
signaling at every network node. The services may be end-to-end or
within a network; they include both those that can satisfy
quantitative performance requirements (e.g., peak bandwidth) and
those based on relative performance (e.g., "class" differentiation).
Services can be constructed by a combination of well-defined building
blocks deployed in network nodes that:
o classify traffic and setting bits in an IP header field at network
boundaries or hosts,
York, et al. Expires December 8, 2014 [Page 4]
Internet-Draft DiffServ and RT Communication June 2014
o use those bits to determine how packets are forwarded by the nodes
inside the network, and
o condition the marked packets (e.g., meter, mark, shape, police) at
network boundaries in accordance with the requirements or rules of
each service.
A network node that supports DiffServ includes a classifier that
selects packets based on the value of the DS field in IP headers,
along with buffer management and packet scheduling mechanisms capable
of delivering the specific packet forwarding treatment indicated by
the DS field value. Setting of the DS field and fine-grain
conditioning of marked packets need only be performed at network
boundaries; internal network nodes operate on traffic aggregates that
share a DS field value, or in some cases, a small set of related
values.
The DiffServ architecture[RFC2475] maintains distinctions among:
o the service provided to a traffic aggregate,
o the conditioning functions and per-hop behaviors (PHBs) used to
realize services,
o the DS field value (DS codepoint, or DSCP) used to mark packets to
select a per-hop behavior, and
o the particular implementation mechanisms that realize a per-hop
behavior.
This document focuses on PHBs and the usage of DSCPs to obtain those
behaviors. In a network node's forwarding path, the DSCP in a field
in the IP packet header is mapped to a particular forwarding
treatment, or per-hop behavior (PHB) that specifies the forwarding
treatment.
A per-hop behavior (PHB) is a description of the externally
observable forwarding behavior of a network node for network traffic
marked with a DSCP that selects that PHB. In this context,
"forwarding behavior" is a general - for example, if only one DSCP is
used for all traffic on a link, the observable forwarding behavior
(e.g., loss, delay, jitter) will often depend only on the relative
loading of the link. To obtain useful behavioral
differentiation,multiple traffic subsets are marked with different
DSCPs for different PHBs for which node resources such as buffer
space and bandwidth are allocated. PHBs provide the framework for a
DiffServ network node to allocates resources to traffic subsets, with
York, et al. Expires December 8, 2014 [Page 5]
Internet-Draft DiffServ and RT Communication June 2014
network-scope differentiated services constructed on top of this
basic hop-by-hop (per-node) resource allocation mechanism.
The codepoints (DCSPs) may be chosen from a set of mandatory values
(the class selector codepoints), or may be chosen from a set of
recommended values defined in PHB specifications, or may be values
may have purely local meaning to the network.
The mandatory DSCPs are the class selector code points as specified
in [RFC2474]. The class selector codepoints (CS0-CS7) extend the
deprecated concept of IP Precedence in the IPv4 header; three bits
are added, so that the class selector DSCPs are of the form 'xxx000'.
The all-zero DSCP ('00000') designates a Default PHB that provides
best-effort forwarding behavior and the remaining class selector code
points were originally specified to provide relatively better per-
hop-forwarding behavior in increasing numerical order, but:
o There is no requirement that any two adjacent class selector
codepoints select different PHBs; adjacent class selector
codepoints may use the same pool of resources on each network node
in some networks.
o CS1 ('001000') was subsequently recommended for "less than best
effort" service when such a service is offered by a network
[RFC3662]. Not all networks offer such a service.
Applications and traffic sources in general cannot rely upon
different class selector codepoints providing differentiated services
or upon the presence of a "less than best effort" service that is
selected by the CS1 DSCP.
2.2. Diffserv PHBs (Per-Hop Behaviors)
Although Differentiated Services is a general architecture that may
be used to implement a variety of services, three fundamental
forwarding behaviors (PHBs) have been defined and characterized for
general use. These are:
1. Default Forwarding (DF) for elastic traffic [RFC2474]. The
Default PHB is always selected by the all-zero DSCP.
2. Assured Forwarding (AF) [RFC2597] to provide differentiated
service to elastic traffic. Each instance of the AF behavior
consists of three PHBs that differ only in drop precedence, e.g.,
AF11, AF12 and AF13; such a set of three AF PHBs is referred to
as an AF class, e.g., AF1x. There are four defined AF classes,
AF1x through AF4x.
York, et al. Expires December 8, 2014 [Page 6]
Internet-Draft DiffServ and RT Communication June 2014
3. Expedited Forwarding (EF) [RFC3246]intended for inelastic
traffic. Beyond the basic EF PHB, the VOICE-ADMIT PHB [RFC5865]
is an admission controlled variant of the EF PHB.
2.3. DiffServ and Transport Protocols
[Editor's note: This section and the recommendations in Section 4 are
centered on TCP, UDP, and SCTP. They could use generalization to
include other transport protocols - DCCP is a likely one to include,
although it is not necessary to include every known transport
protocol.]
Transport protocols provide data communication behaviors beyond those
possible at the IP layer. An important example is that TCP provides
reliable in-order delivery of a data stream with congestion control.
SCTP provides additional properties such as preservation of message
boundaries, and the ability to avoid head-of-line blocking that may
occur with TCP. In contrast, UDP is a basic unreliable datagram
protocol whose primary functionality is port-based multiplexing and
demultiplexing on top of IP.
Transport protocols that provide reliable delivery (e.g., TCP, SCTP)
are sensitive to network reordering of traffic. When a protocol that
provides reliable delivery receives a packet other than the next
expected packet for an ordered connection or stream, it usually
assumes that the expected packet has been lost and respond with a
retransmission request for that packet. In addition, congestion
control functionality in transport protocols usually infers
congestion when packets are lost, creating an additional sensitivity
to significant reordering - such reordering may be (mis-)interpreted
as indicating congestion-caused packet loss, causing a reduction in
transmission rate. This remains true even when ECN [RFC3168] is in
use, as receivers continue to treat missing packets as potential
indications of congestion because extreme congestion conditions may
cause ECN-capable network nodes to drop packets and ECN traffic may
transit network nodes that do not support ECN. Congestion control is
an important aspect of the Internet architecture, see [RFC2914] for
further discussion.
In general, marking traffic with different DSCPs results in different
PHBs being applied at network nodes, making reordering possible due
to use of different pools of forwarding resources for each PHB. The
primary exception is that reordering is prohibited within each AF
class (e.g., AF1x), as the three PHBs in an AF class differ solely in
drop precedence. Reordering within a PHB or AF class may occur for
other transient reasons (e.g., route flap).
York, et al. Expires December 8, 2014 [Page 7]
Internet-Draft DiffServ and RT Communication June 2014
UDP is the primary transport protocol that is not sensitive to
reordering in the network, because it does not provide reliable
delivery or congestion control.
2.4. Traffic Classifiers and DSCP Remarking
DSCP markings are not end-to-end in general. Each network is free to
make its own decisions about what PHBs to use and which DSCP
corresponds to each PHB. While every PHB specification includes a
recommended DSCP, and RFC 4594 [RFC4594] recommends their end-to-end
usage, there is no requirement that every network support any PHBs or
use any DSCPs, with the exception of the requirements for the class
selector codepoints in RFC 2474 [RFC2474]. When DiffServ is used,
the edge or boundary nodes of a network are responsible for ensuring
that all traffic entering that network conforms to that network's
policies for DSCP and PHB usage, and such nodes remark traffic
(change the DSCP marking as part of traffic conditioning)
accordingly. As a result, DSCP remarking is possible at any network
boundary, including the first network node that traffic sent by a
host encounters. Remarking is also possible within a network, e.g.,
for traffic shaping.
DSCP remarking is part of traffic conditioning; the traffic
conditioning functionality applied to packets at a network node is
determined by a traffic classifier [RFC2475]. BA (Behavioral
Aggregate) traffic classifiers are usually used by network nodes
within a DiffServ network; they classify traffic based solely on
DSCPs. MF (Multi-Field) classifiers are usually used by network
nodes at the edges of a DiffServ network, but may also be used for
finer grain traffic conditioning within a DiffServ network; they
classify based on selected header fields, but typical implementations
do not look beyond the traffic's 5-tuple in the IP and transport
protocol headers. As a result, when multiple DSCPs are used for
network traffic that shares a 5-tuple, remarking at a network
boundary (or within a network) may result in all of the traffic being
forwarded with a single DSCP, removing any differentiation within the
5-tuple beyond the point at which this remarking occurs.
In addition, remarking may remove application-level distinctions in
forwarding behavior - e.g., if multiple PHBs within an AF class are
used to distinguish different types of frames within a video flow,
token-bucket-based remarkers operating in Color-Blind mode (see
[RFC2697] and [RFC2698] for examples) may remark solely based on flow
rate and burst behavior, removing the drop precedence distinctions
specified by the source.
Backbone and other carrier networks may employ a small number of
DSCPs (e.g., less than half a dozen) in order to manage a small
York, et al. Expires December 8, 2014 [Page 8]
Internet-Draft DiffServ and RT Communication June 2014
number of traffic aggregates; hosts that use a larger number of DSCPs
may find that much of the intended differentiation is removed by such
networks.
3. RTP Multiplexing Background
Section2 explains how media flows can be multiplexed over RTP
sessions which can in turn be multiplexed over UDP with other RTP
sessions as well as flows generated by other transport protocols.
This section provides background on why this level of media flow
multiplexing is desirable. The rationale in this section applies
both to multiplexing of media flows in RTP sessions and multiplexing
of one or more RTP sessions with traffic from other transport
protocols via UDP encapsulation.
Multiplexing reduces the number of ports utilized for real-time and
related communication in an overall interaction. While a single
endpoint might have plenty of ports available for communication,
these media flows are often traverse points in the network that are
constrained on the number of available ports. A good example is a
NAT/FW device sitting at the network edge. As the number of
simultaneous protocol sessions increases, so does the burden placed
on these devices in order to provide port mapping.
Another reason for multiplexing is to help reduce the time required
to establish bi-directional traffic flows. Since any two
communicating users might be situated behind different NAT/FW
devices, it is necessary to employ techniques like STUN/ICE/TURN in
order to get traffic to flow between the two devices
[I-D.ietf-rtcweb-transports]. Performing the tasks required of
STUN/ICE/TURN take time and requiring an endpoint to perform these
tasks for several flows can increase the time required. While tasks
for different flows can be performed in parallel, it is nonetheless
necessary for applications to wait for all flows to be opened before
communication between to users can begin. Reducing the number of
STUN/ICE/TURN steps reduces the probability of losing a packet and
introducing delay in setting up a communication session. Further,
reducing the number of STUN/ICE/TURN tasks means that there is a
lower burden placed on the STUN and TURN servers.
Multiplexing may reduce the complexity and resulting load on an
endpoint. A single instance of STUN/ICE/TURN is simpler to execute
and manage than multiple instances STUN/ICE/TURN operations happening
in parallel, as the latter require synchronization and create more
complex failure situations that have to be cleaned up by additional
code.
York, et al. Expires December 8, 2014 [Page 9]
Internet-Draft DiffServ and RT Communication June 2014
4. Recommendations
The only use of multiple standardized PHBs and DSCPs that does not
allow network reordering among packets marked with different DSCPs is
the use of PHBs from a single AF class. All other uses of multiple
PHBs and/or the class selector DSCPs allow network reordering of
packets that are marked with different DSCPs.
[Editor's note: The following are preliminary and subject to change.
Please keep in mind that this is a -00 draft] Summary - Applications
and other traffic sources:
o SHOULD NOT use different PHBs and DSCPs that may cause reordering
within a single media flow. If this is not done, significant
network reordering may overwhelm implementation assumptions about
limits on reordering (e.g., available buffering) resulting in poor
user experiences and the like.
o SHOULD NOT use different PHBs and DSCPs that may cause reordering
within an ordered session for a reliable transport protocol (e.g.,
TCP, SCTP) , Receivers for such protocols interpret reordering as
indicating loss of out-of-order packets causing undesired
retransmission requests, and will infer congestion from
significant reordering, causing throughput reduction.
o MAY use different PHBs and DSCPs that cause reordering within a
single UDP 5-tuple, subject to the above constraints. The service
differentiation provided by such usage is unreliable, as it may be
removed at network boundaries for the reasons described in
Section 2.4 above.
o SHOULD NOT rely on end-to-end preservation of DSCPs or of drop
precedence distinctions within an AF class (e.g., different DCSPs
applied to different types of video frames), as network node
remarking can change DSCPs and remove drop precedence distinctions
see Section 2.4 above.
o SHOULD use the CS1 codepoint only for traffic that is acceptable
to forward as best effort traffic, as network support for use of
CS1 to select a "less than best effort" PHB is inconsistent.
Further, some networks may treat CS1 as providing "better than
best effort" forwarding behavior.
5. RTCWEB Examples
[Editor's Note: This section will provide examples of DiffServ/DSCP
application to RTCWEB and related limitations.]
York, et al. Expires December 8, 2014 [Page 10]
Internet-Draft DiffServ and RT Communication June 2014
6. Acknowledgements
This document is the result of many conversations that have occurred
within multiple RAI and TRANSPORT area working groups. Thanks for
review and input from James Polk.
7. IANA Considerations
This document includes no request to IANA.
8. Security Considerations
[Editor's Note: There are security considerations, start by pointing
to security considerations in the relevant references. Multiplexing
multiple transport protocols onto a single UDP 5-tuple has firewall
configuration and traffic inspection/monitoring implications.]
9. References
9.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
9.2. Informative References
[I-D.ietf-rtcweb-overview]
Alvestrand, H., "Overview: Real Time Protocols for Brower-
based Applications", draft-ietf-rtcweb-overview-09 (work
in progress), February 2014.
[I-D.ietf-rtcweb-rtp-usage]
Perkins, C., Westerlund, M., and J. Ott, "Web Real-Time
Communication (WebRTC): Media Transport and Use of RTP",
draft-ietf-rtcweb-rtp-usage-15 (work in progress), May
2014.
[I-D.ietf-rtcweb-transports]
Alvestrand, H., "Transports for RTCWEB", draft-ietf-
rtcweb-transports-04 (work in progress), April 2014.
[RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black,
"Definition of the Differentiated Services Field (DS
Field) in the IPv4 and IPv6 Headers", RFC 2474, December
1998.
York, et al. Expires December 8, 2014 [Page 11]
Internet-Draft DiffServ and RT Communication June 2014
[RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
and W. Weiss, "An Architecture for Differentiated
Services", RFC 2475, December 1998.
[RFC2597] Heinanen, J., Baker, F., Weiss, W., and J. Wroclawski,
"Assured Forwarding PHB Group", RFC 2597, June 1999.
[RFC2697] Heinanen, J. and R. Guerin, "A Single Rate Three Color
Marker", RFC 2697, September 1999.
[RFC2698] Heinanen, J. and R. Guerin, "A Two Rate Three Color
Marker", RFC 2698, September 1999.
[RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, RFC
2914, September 2000.
[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
of Explicit Congestion Notification (ECN) to IP", RFC
3168, September 2001.
[RFC3246] Davie, B., Charny, A., Bennet, J., Benson, K., Le Boudec,
J., Courtney, W., Davari, S., Firoiu, V., and D.
Stiliadis, "An Expedited Forwarding PHB (Per-Hop
Behavior)", RFC 3246, March 2002.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC3662] Bless, R., Nichols, K., and K. Wehrle, "A Lower Effort
Per-Domain Behavior (PDB) for Differentiated Services",
RFC 3662, December 2003.
[RFC4594] Babiarz, J., Chan, K., and F. Baker, "Configuration
Guidelines for DiffServ Service Classes", RFC 4594, August
2006.
[RFC5865] Baker, F., Polk, J., and M. Dolly, "A Differentiated
Services Code Point (DSCP) for Capacity-Admitted Traffic",
RFC 5865, May 2010.
[W3C.WD-mediacapture-streams-20130903]
Burnett, D., Bergkvist, A., Jennings, C., and A.
Narayanan, "Media Capture and Streams", World Wide Web
Consortium WD WD-mediacapture-streams-20130903, September
2013, <http://www.w3.org/TR/2013/
WD-mediacapture-streams-20130903>.
York, et al. Expires December 8, 2014 [Page 12]
Internet-Draft DiffServ and RT Communication June 2014
Authors' Addresses
Dan York (editor)
Internet Society
Keene, N.H.
USA
Phone: +1-802-735-1624
Email: dyork@lodestar2.com
David Black (editor)
EMC
176 South Street
Hopkinton, MA 01748
USA
Phone: +1 508 293-7953
Email: david.black@emc.com
Cullen Jennings
Cisco
170 West Tasman Drive
MS: SJC-21/2
San Jose, CA 95134
USA
Phone: +1 408 421-9990
Email: fluffy@cisco.com
Paul Jones
Cisco
York, et al. Expires December 8, 2014 [Page 13]