Internet Engineering Task Force Steven Blake
INTERNET-DRAFT IBM Corporation
Expires: June 1998
December 1997
Some Issues and Applications of Packet Marking
for Differentiated Services
<draft-blake-diffserv-marking-00.txt>
Status of This Memo
This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
To learn the current status of any Internet-Draft, please check the
"1id-abstracts.txt" listing contained in the Internet-Drafts Shadow
Directories on ftp.is.co.za (Africa), ftp.nordu.net (Europe),
munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or
ftp.isi.edu (US West Coast).
Abstract
''Packet marking'' is proposed as an architectural generalization of
the type of service (TOS) and precedence facilities of IPv4 [RFC795,
RFC1349], as well as the traffic class facilities of IPv6 [IPv6]. It
is intended to encompass all mechanisms by which a host or a router
may mark a packet to invoke some differentiated packet handling
behavior by another node along the transit path of the packet. This
memo examines several proposed applications of a packet marking
facility and attempts to categorize each application in terms of the
behavioral requirements it imposes on hosts and routers. In
addition, issues related to the deployment of packet marking,
including provisioning, authorization, and security, are examined.
This memo is proposed as a framework to focus discussion on
implementation issues and mechanisms as new differentiated services
enabled by a packet marking facility are introduced into the
Internet.
Blake Expires: June 1998 [Page 1]
INTERNET-DRAFT Packet Marking December 1997
Table of Contents
1. Introduction .................................................... 3
2. Motivation ...................................................... 4
3. Some Proposed Applications of Packet Marking .................... 6
3.1 Explicit Priority ............................................ 6
3.1.1 Delay Priority ........................................... 6
3.1.2 Drop Priority ............................................ 7
3.1.3 Network Control Priority ................................. 8
3.2 Explicit Service Class Indication ............................ 8
3.2.1 Precedence Service Classes ............................... 9
3.2.2 Transport Isolation ...................................... 10
3.2.3 Aggregated Integrated Services Classes ................... 10
3.2.4 Service-based Route Selection ............................ 11
3.3 Best-Effort Service Allocation ............................... 11
3.4 Integrated Services Conformant Packet Indication ............. 13
3.5 Forward Explicit Congestion Notification ..................... 14
4. Differentiation Mechanism Categorization ........................ 16
4.1 Host Packet Processing Mechanism Categorization .............. 16
4.2 Router Packet Processing Mechanism Categorization ............ 17
4.3 Biased vs. Substitute Best-Effort Router Mechanisms .......... 19
4.3.1 Transmission Mechanism Categorization .................... 19
4.3.2 Path Selection Mechanism Categorization .................. 22
5. Service Categorization .......................................... 22
5.1 Service Granularity .......................................... 22
5.2 Service Invocation ........................................... 23
5.3 Service Behavior ............................................. 24
5.4 Direction of Value ........................................... 25
6. Fairness and Congestion Control Considerations .................. 26
7. Provisioning Considerations ..................................... 27
8. Authorization Considerations .................................... 29
9. Routing Considerations .......................................... 30
10. System Implementation Considerations ............................ 31
11. Standardization Considerations .................................. 33
12. Security Considerations ......................................... 34
13. Acknowledgements ................................................ 35
14. References ...................................................... 35
Author's Address .................................................... 39
Blake Expires: June 1998 [Page 2]
INTERNET-DRAFT Packet Marking December 1997
1. Introduction
Best-effort networks such as the current Internet provide a service
to users which can best be characterized as delivering connectivity
plus a weak guarantee of fair access to network resources. In this
sort of network the performance of individual applications is highly
dependent on the instantaneous demand for network resources.
Although this level of service has proven satisfactory for a wide
variety of uses, there exist both applications and users which would
benefit substantially from a more predictable level of performance,
or from an "inequitable" share of the network resources. As a
consequence, there has been significant effort to define new
mechanisms to enable differentiated services within the Internet.
This document examines a particular class of differentiation
mechanisms that are triggered by a facility we refer to loosely as
"packet marking". Network nodes (routers and hosts) can implement,
in addition to the classical best-effort service, a variety of packet
processing, forwarding, buffer management, and scheduling behaviors
to differentiate packet queueing delay, packet loss, and application
flow throughput. These alternative differentiation mechanisms can
be invoked for a particular packet by marking it; i.e., by setting
some combination of one or more bits in a "packet handling" field in
the packet header. Note that both the IPv4 and IPv6 protocols
possess header fields intended specifically for this purpose (TOS/
Precedence [RFC1349], and Class [IPv6], respectively). We will use
the term "PH field" to signify the packet handling field in the
general case.
A differentiated service is provided for a stream of packets by
marking (possibly a subset of) the constituent packets thereby
invoking some differentiation mechanism for each marked packet on the
nodes along the stream's path. A stream here may represent various
granularities of traffic ranging from an individual application flow
all the way to the aggregate traffic exchanged between a pair of
service providers. The differentiated service when invoked is
visible to the application or user as some change in a quantitative
characteristic of the aggregate packet transport (e.g., average delay,
loss, or throughput).
A distinguishing feature of the differentiated services mechanisms
examined here is that they treat packets with identical markings
equivalently; the mechanisms act on aggregated classes of packets
(where a class represents those packets with particular markings) and
they operate without per-flow state in every node along a packet's
path. This is in contrast to the Integrated Services model
[RFC1633], where per-flow classification, policing, and scheduling
state is installed and maintained on nodes along the path either via
application signaling [RSVP] or via administrative configuration.
While the Integrated Services mechanisms provide more granular QoS
guarantees to individual application flows, the requirement for
application signaling and per-flow state in the network introduces
Blake Expires: June 1998 [Page 3]
INTERNET-DRAFT Packet Marking December 1997
performance, scalability, and application compatibility issues.
Many applications can still benefit while utilizing a simpler and
more scalable set of differentiation mechanisms. Note also that
packet marking may help facilitate Integrated Services state
aggregation in the interior of the Internet (see Sec. 3.2.3).
The concept of packet marking described in this document should be
distinguished from the efforts of the Multi-protocol Label Switching
working group [MPLS]. MPLS is primarily intended to simplify router
forwarding implementations and to enable enhanced routing services.
However, some of the issues discussed in this document may be
relevant to MPLS as well.
Section 2 of this document elaborates on the motivations for
deploying packet marking differentiated services. Section 3 outlines
some of the proposed applications of packet marking. Section 4
introduces a categorization of differentiation mechanisms for hosts
and routers and describes how the proposals examined in Sec. 2 fit
into this categorization. Section 5 provides a categorization of
differentiated services enabled by packet marking. Sections 6-12
address various issues of fairness, congestion control, provisioning,
authorization, routing, implementation, standardization and security
which are introduced with the implementation and deployment of packet
marking differentiated services.
2. Motivation
As discussed in [Shenker], different applications have different
utility functions of the bandwidth provided for the application by
the network infrastructure. These different applications (e.g.,
elastic (best-effort); hard, delay-adaptive, and rate-adaptive real-
time) exhibit varying sensitivity to the transient and steady-
state level of resources available from the network. This
sensitivity is often a function of human factors considerations, but
can also result from the fundamental characteristics of the
application (e.g., distributed database synchronization).
In addition, users and organizations may realize greater utility from
a particular subset of the applications (either elastic or inelastic)
in use, due to personal or business objectives. As an example, many
corporate networks prioritize transaction processing and interactive
traffic from some applications over batch and other applications'
traffic due to their immediate impact to business operations.
Finally, different customers of a network service provider may
realize greater or lesser utility from the network service they
receive. These customer utilities are not necessarily proportional
to the speeds of their access links to the service provider.
Network owners, be they private network managers, or public Internet
service providers, wish to maximize their return on investment in
Blake Expires: June 1998 [Page 4]
INTERNET-DRAFT Packet Marking December 1997
network infrastructure. Their ability to increase revenue is
constrained by the elasticity of demand, the mixture of application
types, and the means of pricing and cost-recovery available to them,
but in general they stand to benefit if they can tailor their service
offerings and pricing structures to satisfy the entire range of
application and customer service requirements, by allowing each
customer to maximize his utility/cost function. Because of the
variation in customer utility functions, differentiated pricing
(e.g., by Service Level Agreements) is a key revenue generating
mechanism, but its success depends on the ability to engineer the
network to satisfy the diversity of application/customer
service requirements.
One means of satisfying these requirements is to engineer
differentiated packet handling mechanisms into the network. These
mechanisms, which can be conceptualized as mechanisms for
prioritization and resource allocation, allow the service provider
to provision the network for each of the offered classes of service
so as to meet the application/customer requirements at the level of
statistical assurance promised.
Another means of satisfying these requirements is to over-provision
the network, so that it tends to run at low utilization with minimal
congestion. One problem with over-provisioning as a strategy for
enabling differentiated services is that any best-effort network
(i.e., without admission control) with concentration points can
experience transient congestion and loss, which will make it
difficult to support the most rigorous of application requirements.
Another more fundamental difficulty is that over-provisioning
provides better service to some customers than they would be willing
to pay for (as judged by their utility functions). Whether over-
provisioning is a cost-effective method of service differentiation
as compared to providing differentiation mechanisms within the
network depends on the level and type of application/customer demand,
the incremental cost of additional network infrastructure, and the
rate of change in demand. However, it appears clear that in an
environment (such as today) where there is explosive growth in the
users of and traffic in the Internet, that low-complexity
differentiation mechanisms offer a more rapid and effective means of
tailoring network service offerings than network over-provisioning.
Classifying packets to determine their service class can be
implemented by a number of means, including source/destination
address, protocol, and TCP/UDP port filtering. A problem with
filtering at this level of granularity is that each router along the
path of a packet will require the necessary filtering rules to
determine the service class. This has scaling problems, both in
terms of the number of filtering rules, as well as in the need for
mechanisms to dynamically add and delete new rules according to
changes in customer and application traffic. A further problem is
that, with the deployment of IP Security, transport payloads
encrypted within an Encapsulated Security Payload (ESP) cannot be
Blake Expires: June 1998 [Page 5]
INTERNET-DRAFT Packet Marking December 1997
identified by a router, because the protocol and port values are
obscured [ESP]. This will prevent any service differentiation of
encrypted traffic at the application level.
A motivation for deploying packet marking is that the network routers
need only filter on the value of the PH field to determine the
appropriate differentiation mechanisms to apply to a packet. When
coupled with aggregate buffer management and packet scheduling
mechanisms, as well as network authorization of the PH values and
adequate provisioning, packet marking provides a scalable mechanism
for offering differentiated transport services to different traffic
streams. These differentiation mechanisms may be useful without
explicit network authorization and provisioning to allow best-effort
applications to trade some fraction of their fair share packet rate
for a lower loss rate or for lower average queueing delay [RFC1046].
Packet marking may also be useful as a means for improving the
scalability of per-flow Integrated Services by simplifying the
implementation of flow aggregation and by improving the efficiency of
the Integrated Services packet classification mechanism [CLASSY,
GBH97].
3. Some Proposed Applications of Packet Marking
In this section we describe some proposed applications of a packet
marking facility. We are using the term "application" here to refer
to one or more services which could be delivered by specifying the
semantics of one or more PH bits and by specifying the
differentiation mechanisms they invoke within the network. Some of
the proposed semantics are explicit regarding the service requested
(e.g., transport isolation) while others could be used to provide a
variety of services. Several proposals overlap in terms of the
differentiation mechanisms utilized, and as such, a common set of
PH bits could be used to enable these proposed services. We will
examine the proposals individually here and will then categorize them
more rigorously in Sec. 4.
3.1 Explicit Priority
One application of packet marking is to provide an explicit request
for priority handling of the packet by routers. Priorities are
usually ranked in a strict hierarchy relative to some metric (e.g.,
delay, loss probability). The priority value is usually intended to
reflect the importance of a packet relative to other packets from the
source as well as to packets from other sources [RFC795, RFC1046,
RFC1812 Sec. 5.3.3]. In this section we assume the router does not
perform priority based path selection (this is discussed in Sec.
3.2.4).
3.1.1 Delay Priority
Delay priority indication within a packet is intended to convey the
Blake Expires: June 1998 [Page 6]
INTERNET-DRAFT Packet Marking December 1997
sensitivity of an application to router queueing delay. A possible
range of values is the following:
o High -- prefer low maximum queueing delay; jitter
sensitive,
o Interactive -- prefer low average queueing delay; jitter
insensitive,
o Regular -- can tolerate the normal delay distribution
delivered by best-effort FIFO queueing,
o Low -- can tolerate extensive queueing delay or jitter.
A greater granularity of delay priority values is possible. However,
without strict per-flow admission control and policing, quantifiable
bounds on the delay distribution at a particular priority level are
difficult to determine. Delay priority is useful for allowing
applications which are delay sensitive to avoid large queues,
possibly at the expense of packet loss rate, while permitting
applications which are not sensitive to queueing delay to utilize
router buffers to avoid packet loss and achieve higher throughput (or
to avoid much worse round-trip time (RTT) delays). One simple
implementation mechanism is to provide a separate queue for each
delay priority level, with strict priority service between the
levels. A problem with this sort of implementation is possible
starvation of service at the lower delay priority levels.
Implementation issues of delay priority are further discussed in Sec.
4.
3.1.2 Drop Priority
Drop or discard priority indication within a packet is intended to
convey the sensitivity of an application (or a sub-layer of its
traffic) to packet losses. A possible range of values is the
following:
o Critical -- extremely loss sensitive; do not discard while other
non-critical packets are queued,
o High -- loss sensitive; drop only under extreme congestion,
o Regular -- can tolerate normal loss rates under active buffer
management [RED, ACTIVE],
o Low -- not sensitive to loss; discard under light
congestion.
A greater granularity of drop priority values is possible, however,
as with delay priority, in the absence of strict per-flow admission
control and policing, quantifiable bounds on the loss probabilities
at a particular priority level are difficult to determine.
Furthermore, it may be difficult to engineer several levels of drop
priority without introducing delay for the higher drop priority
levels under congestion. One possible implementation of drop
priority is to use multiple thresholds of packet occupancy in a
single FIFO queue to trigger the discard action for incoming packets
at a particular drop priority level. These thresholds could be based
Blake Expires: June 1998 [Page 7]
INTERNET-DRAFT Packet Marking December 1997
on the instantaneous queue occupancy with deterministic discard or on
an averaged queue occupancy with stochastic discard [Clark97, Feng97]
(see Sec. 4. for further discussion of implementation issues). Drop
priority is useful both for improving the throughput of more
important application flows as well as in enabling rate-adaptive
multi-layer audio and video applications, which can adjust their
rates after detecting impending congestion due to the drop of lower
priority packets of the encoded signal, while still protecting the
higher quality components of the signal from loss. Such an approach
to layering has superior control-plane scalability to alternatives
such as receiver-driven layered multicast [McCanne] (however, there
are issue of fairness and congestion which may bias an application to
the alternative method (see Sec. 6)).
3.1.3 Network Control Priority
Network Control priority indication within a packet is intended to
indicate that the packet is a component of a network control protocol
exchange whose correct and timely operation is critical to the
stability of the network. It is primarily intended for use with
routing protocols (e.g., RIP, OSPF, IS-IS, BGP), but could also be
used for other network signaling and control protocols (e.g., SNMP,
RSVP, MPLS) [RFC1812 Sec. 7.1.2]. The value of prioritizing routing
traffic over data traffic is to prevent routing collapse under heavy
load (e.g., preventing BGP connection timeouts due to excessive TCP
losses and retransmits). The value of prioritizing SNMP traffic is
to eliminate a denial-of-service attack (where the network manager
cannot monitor or configure a network element). A sensible
implementation will both guarantee an extremely low loss rate for
network control packets (i.e., by never discarding a network control
packet when other types of packets are queued) and will attempt to
bound the queueing delay they experience. This could be accomplished
by implementing a separate network control queue with strict
priority, or by providing priority pushout within a single FIFO queue
(implementation issues are further discussed in Sec. 4). Because
network control traffic is usually a small fraction of the total
traffic within a network, this prioritization should not have a
noticeable impact on data transport performance. However, because of
the high priority provided for this class of traffic, only routers
and network management stations should be allowed to set the network
control priority indication, and the network should take steps to
authenticate the source of a packet with the priority indication set
(see Sec. 12).
3.2 Explicit Service Class Indication
Another application of packet marking is to explicitly indicate a
"service class" for a packet. A service class is a more general
concept than delay or drop priority. It can be associated with a set
of resources provisioned within the interior of the network (e.g.,
bandwidth, buffers, routes) for a particular set of application/
customer traffic flows which are mapped onto that class. The service
Blake Expires: June 1998 [Page 8]
INTERNET-DRAFT Packet Marking December 1997
class concept does not impose any strict hierarchy of delay, loss,
or throughput priority between classes, but instead may permit the
specification of quantitative bounds on delay, loss, or throughput
performance for a class for a particular traffic profile within that
class. Issues regarding the implementation of explicit service
classes are discussed in Sec. 4.
3.2.1 Precedence Service Classes
One instantiation of the service class concept is to provide a set of
"precedence" service classes, in a manner very similar to the delay
and drop priorities discussed in Secs. 3.1.1 and 3.1.2, but with
potentially more flexibility in the provisioning of the classes.
Each individual class would be provisioned to provide "better"
service than the class of immediately lower rank, where the precise
definition of better service could for instance be defined as a
higher probability of timely delivery; i.e., lower probability of
loss and lower average delay. Each class in the hierarchy could be
engineered to provide a statistically quantifiable service for some
expected or regulated load, while also being engineered to prevent
starvation of service to the lowest precedence classes. Regulation
could be implemented in the form of dynamic service class mapping and
policing at the edge of the network. The result of implementing this
range of services would likely be improved throughput for application
flows in the higher precedence classes.
One application of this scheme would be to map business-critical
transaction traffic to a service class of high precedence, while
mapping casual web browsing traffic to a lower precedence class.
These classes could be implemented using a variety of methods,
including some variant of Weighted Fair Queueing (WFQ) or Class-based
Queueing (CBQ) [HPFQA, HFSC, CBQ]. Depending on the precise service
guarantees promised for the classes, they could potentially be
implemented using combinations of the explicit delay and drop
priority PH markings and router mechanisms described in Secs. 3.1.1
and 3.1.2.
[TWOBIT] describes a particular precedence service class
implementation which relies on authorization and policing/shaping at
the network edge and strict delay priority queueing in the interior
routers. Flows or flow aggregates assigned to the "Premium" service
class are policed based on a peak rate limit and any residual bursts
are shaped at the network edge; this smoothes the characteristics of
the Premium traffic which results in minimal accumulated queueing
delay in the interior routers (when the total Premium service load
is moderate). Packets which exceed the negotiated peak-rate limit
are discarded. Per-router service class provisioning is not
required in this scheme since two-level strict priority queueing is
used as the differentiation mechanism. However, Premium service
must be conservatively allocated to prevent starvation of the best-
effort service queue.
Blake Expires: June 1998 [Page 9]
INTERNET-DRAFT Packet Marking December 1997
3.2.2 Transport Isolation
Although TCP is the most widely used transport protocol in the public
Internet, there are alternative transport protocols which may have a
more or less aggressive response to network congestion or packet
loss. When run in parallel with TCP traffic, these transport
protocols, some of which are also used in alternative protocol packet
networks, may be unable to achieve their fair share rate (if they are
less aggressive), or may prevent TCP flows from achieving their fair
share rates (if they are more aggressive). These transport protocols
can be isolated from each other by mapping the flows which utilize
them to different service classes which are appropriately provisioned
for some level of minimal service. A router may queue these
transport-isolated service classes separately. The flows within each
service class queue then only compete with each other for the minimal
guaranteed bandwidth which is provisioned for that class, and can
temporarily consume the bandwidth provisioned for other classes
whenever they are unloaded. An advantage of transport isolation is
that it can protect normal best-effort TCP traffic from some well-
known mis-behaved transport protocols. The minimal bandwidth for
each transport service class must be provisioned at each router.
3.2.3 Aggregated Integrated Services Classes
Integrated Services and RSVP as currently specified depend on per-
flow signaling and per-flow packet classification, using the
destination address, protocol, and destination port of a packet, and
often also the source address and source port [RFC1633, RSVP]. As
has been mentioned previously, the requirement for per-flow control
and classification state may introduce scalability problems in the
interior of the Internet, where the demand for reservations on high-
speed links may exceed several thousand simultaneous flows.
Scalability can be improved by aggregating both the control and
packet classification state generated by a set of (unicast) flows
transiting a particular path through a segment of the Internet. This
approach was introduced in [CLASSY], and is examined in more detail
in [GBH97] and [TWOBIT]. It should be noted that no feasible design
for the aggregation of multicast flows has been published.
The proposals for control-state aggregation are not the topic of this
memo (the reader is encouraged to see [GBH97] and [TWOBIT]).
However, the details of packet classification aggregation are
relevant here. The basic concept is to mark traffic corresponding to
an Integrated Services traffic class with a particular PH service
class marking (e.g., Controlled Load, Guaranteed) at the router along
the flows' path which initiates aggregation. Subsequent routers
within the aggregating region do not classify the aggregated reserved
packets using the normal per-flow/session packet filters but instead
classify them based on their PH service class marking. These flows
are then serviced using a queue which has been either statically or
is dynamically provisioned to provide the required QoS for the total
set of aggregated Integrated Services flows of a particular traffic
Blake Expires: June 1998 [Page 10]
INTERNET-DRAFT Packet Marking December 1997
class which traverses that link. Aggregated RSVP control messages
between routers on the edge of the aggregating region could be used
to specify the aggregated reservation request between those nodes,
and the interior routers could use this information to perform
admission control and to dynamically adjust the resources (e.g.,
bandwidth, buffers) which are allocated to each aggregated Integrated
Services traffic class. More than two service classes could be used
for aggregation, each provisioned to deliver a particular QoS for the
flows utilizing that class.
One additional packet marking requirement introduced by aggregation
is the need to explicitly mark those packets of a reserved flow which
do not conform to the flow's reservation Tspec (see Sec. 3.4). This
marking is necessary since per-flow policing is not possible within
the interior of the aggregating region, and a single non-conformant
flow could reduce the QoS delivered to all other flows aggregated
into its service class. The alternative of marking these non-
conformant flows as best-effort could lead to unnecessary packet re-
ordering. Furthermore, it is critical that flows not policed at an
RSVP aggregation point not be marked with one of the aggregated
Integrated Service class PH markings and serviced using the resources
dedicated to the aggregated flows. These aggregated service classes
require isolation from other (potentially real-time) traffic, since
resources have been specifically dedicated to them based on
advertised and regulated traffic loads. The routers on the edge of
the aggregating region must prevent unauthorized use of these PH
markings by non-reserved flows.
3.2.4 Service-based Route Selection
An alternative means of differentiating the service provided to a
given class of traffic is to implement service-specific routes (i.e,
TOS routes) [RFC1349, RFC1583]. Service-specific routes can be
defined based on those characteristics of packet transport that are
largely affected by the path selected between two end-nodes. The
canonical set of characteristics used are delay, reliability,
throughput, and cost [RFC1349], although routes based on a more
detailed set of characteristics could in principle be defined.
Routing protocols can then be used to compute service-specific routes
by factoring in different link and path metrics (e.g., propagation
delay, bit error rate, link rate, transport cost).
Issues surrounding service-specific route selection are examined in
Sec. 9.
3.3 Best-Effort Service Allocation
There have been several recent proposals to use packet marking
mechanisms to provide best-effort service allocation [Clark97, SIMA,
Crow96, May97, Feng97, Bohn93, Ferg97, TWOBIT]. The term best-effort
service allocation refers to the notion of providing different
expectations of best-effort service to different categories of users
Blake Expires: June 1998 [Page 11]
INTERNET-DRAFT Packet Marking December 1997
or applications based on some negotiated service profile with the
network. These expectations could be characterized in terms of
average throughput, loss, or delay, along with variance estimates
(statistical assurance levels) for these metrics. These schemes are
primarily motivated to provide different tiers of service to elastic
best-effort applications; in their simplest form they rely on network
dimensioning with authorization and enforcement at the network edge
to provide statistical assurances on performance which may not be
suitable for all application types. Explicit provisioning of
resources in the interior of the network is not precluded, but the
proposals are designed to work effectively using only simple
differentiation mechanisms in the interior routers, such as strict
drop or delay priority. We outline three of the proposed schemes
here.
[Clark97] describes two proposals for service allocation; one relying
on marking at the network edge near the sender (sender based scheme),
and the other relying on marking within the network and reaction at
the network edge near the receiver (receiver scheme). We describe
the sender scheme here and the receiver scheme in Sec. 3.5.
In the sender scheme, a profile meter monitors the traffic load
generated by a source and marks traffic which exceeds the negotiated
profile (which might be defined by a token bucket, for example) with
an in-profile indicator. The in-profile marking is interpreted as
the inverse of a drop preference indicator by the interior routers,
which preferentially discard drop preference traffic whenever
impending congestion is detected. The proposed differentiation
mechanism is a weighted variant of the RED algorithm [RED], termed
"RED with In and Out" (RIO), which uses a single FIFO queue and two
RED drop thresholds, the lower being assigned to the drop preference
traffic. This mechanism is designed to shut down out-of-profile
flows as the in-profile traffic utilization on a link approaches
one. No explicit provisioning of resources to the two levels of
service are required in the interior. This basic scheme can be
augmented by defining two or more levels of service assurance (e.g.,
statistical, assured). The service provider must dimension the
network and provision the profiles of assured sources carefully to
reduce the probability of congestion loss. In addition, some form of
differentiation must be implemented in the router (such as separate
provisioned queues) to preferentially deliver in-profile assured
packets under congestion.
A conceptually similar scheme is described in [SIMA]. In this
proposal, the user contracts with the network for a Nominal Bit Rate
(NBR) of service. A monitor at the edge of the network measures the
traffic load relative to the NBR and dynamically computes a level of
drop preference (out of seven) for the packet. A packet flow at rate
NBR would be marked with a mid-range drop-preference; the network
would be dimensioned to provide a small loss ratio at this level.
Pricing of service is governed by the size of the NBR; users can
achieve better service by purchasing a larger NBR and underutilizing
Blake Expires: June 1998 [Page 12]
INTERNET-DRAFT Packet Marking December 1997
it (thereby receiving low drop-preference for their packets). The
network routers implement preferential discard of traffic based on
a series of thresholds for each drop-preference level. The system
also allows the user to select real-time or non-real-time service.
Real-time packets are served by a small queue which receives strict-
priority service over the non-real-time queue. The small size of the
queue and the potentially higher loss rate gives the user incentive
not to utilize the real-time service for elastic applications.
[Feng97] describes a design that is similar to [Clark97]. One
salient difference is that the profile-meter at the network edge
(termed the Packet Marking Gateway (PMG)) statistically marks packets
for priority service based on some computation of the number of
priority marked packets needed to achieve a target throughput. The
network routers perform service differentiation using a weighted RED
implementation. One extension of this work is the incorporation of
the packet marking facility within the TCP congestion control
algorithm.
3.4 Integrated Services Conformant Packet Indication
Packet marking can be used to simplify and enhance the implementation
of Integrated Services, by marking packets within an Integrated
Services flow which conform to the flow's Tspec (or conversely by
marking non-conformant packets). As was mentioned in Sec. 3.2.3,
non-conformant packet marking is essential to permitting RSVP
aggregation, as per-flow policing is not possible when the control-
state is aggregated and non-conformant packets can degrade the QoS of
other aggregated flows. Packet marking of conformant packets may be
useful for non-aggregated Integrated Services flows as well, as it
can provide a hint to routers as to which packets may require
classification (a computationally expensive procedure) as well as
providing an indication as to of which packets of the flow have
failed policing upstream. We describe both a conformant packet
marking scheme and its dual below.
In the first scheme, a single bit is used to indicate that the packet
belongs a flow and that that packet has not failed a policing
function (it may be conformant). Only packets which have this bit
set are flow-classified by the routers, and only these packets are
counted against the flow's Tspec. There are three alternatives for
where and when this bit should be set:
o this bit is set by the source when it has sent a PATH message for
the flow,
o the bit is set by the source only when it has received a RESV
message for the flow,
o the bit is set in the network by the farthest upstream router
which accepted a RESV for the flow (often the source's first hop
router).
Blake Expires: June 1998 [Page 13]
INTERNET-DRAFT Packet Marking December 1997
An issue with alternative one is that the bit may be set for flows
which are never reserved. An issue for alternative two is that the
semantics of the bit do not permit partial reserved paths (where the
reservation succeeds partially upstream from the receiver but fails
before reaching the source) since the bit will never be set (by the
host) and the routers will never classify the packet. This issue can
be addressed in part by alternative three, but in this case, the
farthest upstream router must classify every packet received on the
same interface as traversed by the flow to identify its packets; this
offsets the main advantage of providing the indication. Furthermore,
this introduces a dependency on the behavior of an upstream router
(since the furthest upstream router which accepted the reservation
must wait for RESVERR messages to guess whether an upstream node has
accepted the reservation and will mark the packets).
In the second scheme, the bit is only used to mark packets which have
failed a policing function (non-conformant). Every packet which does
not have the bit set is flow-classified, eliminating a potential
performance advantage of the first scheme since in this case all
best-effort packets are also classified. The bit is set if a packet
fails a flow's Tspec policing function (token bucket). Downstream
routers could choose not to classify these packets or could choose
not to count them against the flow's Tspec.
There are potential re-ordering hazards for both schemes, depending
on how non-conformant packets are serviced at the router. If non-
conformant packets are not classified and are serviced in the best-
effort queue, then re-ordering is likely whenever there is a
disparatity in the queueing delay between the flow's normal service
queue and the best-effort queue. The first scheme only appears to be
able to reduce the amount of packets which are classified while
preserving the defined RSVP network behavior if alternative one is
chosen. The second scheme can only reduce the number of packets
which are classified significantly if a large fraction of the
Integrated Services packets are non-conformant. However, the
semantics of the second scheme much more closely match the
requirements for aggregated flows (where flow-classification is
eliminated). Both schemes are mutually compatible if separate PH
bits are utilized for each.
3.5 Forward Explicit Congestion Notification
The final packet marking application we will discuss is Forward
Explicit Congestion Notification (FECN). FECN is one-half of a
bi-directional scheme where the network marks packets which are
transmitted across a congested link, and some process at the
receiving node sends a Backwards Explicit Congestion Notification
(BECN) back to the source, to influence its rate of transmission so
that the congestion within the network will subside. The BECN need
not be returned in the PH field but may be sent in some higher-layer
message. There are two proposed implementations of FECN which we
will examine.
Blake Expires: June 1998 [Page 14]
INTERNET-DRAFT Packet Marking December 1997
In the approach describe in [ECN94] and [ECN97], a router sets the
FECN bit stochastically based on the RED algorithm, which computes a
probability of packet detection which is an increasing function of
the queue's average packet occupancy [RED]. The router sets this bit
as an alternative action to discarding the packet, which it would
have done if the associated transport protocol had not advertised its
ECN-capability). There are two variants proposed; one a single-bit
scheme and the other a two-bit scheme. In the single bit scheme, the
application transport protocol (which is ECN-capable) sets the FECN
bit, which is reset by any router which randomly detects the packet
due to a build up of queued packets. The packet can then no longer
be distinguished from packets utilizing non-ECN-capable transports,
and if it is detected downstream at another congested router, it will
be discarded. In the two-bit scheme, the transport protocol's ECN-
capability is advertised explicitly in a separate bit, and packets
which are detected at multiple routers are not discarded. Upon
receipt of a packet with the FECN, the receiver sends a BECN back to
the source, either as a IP-layer message (e.g., ICMP Source Quench)
or as a transport-layer acknowledgement (e.g., TCP ACK option). The
source transport protocol is supposed to treat the receipt of a BECN
equivalently to the loss of a packet and back-off its transmission
rate accordingly. Such a mechanism when widely deployed may
significantly reduce the number of lost packets and retransmissions
in the network. This reduction in the number of packet losses is
especially beneficial to interactive applications like Telnet which
are sensitive to RTT delays which result from packet loss and
retransmission intervals.
The approach described in [Clark97] is the receiver-based best-effort
service allocation scheme mentioned in Sec. 3.3. In this (one-bit)
scheme, routers set the FECN bit for every packet which experiences
congestion (deterministic marking vs. stochastic marking). A
receiver profile meter further downstream monitors the number of
packets which are marked by FECN and resets the FECN of all those
packets within the receiver's service profile. Packets with FECN in
excess of the profile are forwarded to the receiver with the FECN
set. The receiver transport protocol is supposed to take the same
action as described for the first scheme (send a BECN), and the
source transport protocol is also supposed to behave as in the first
scheme (back-off). In addition, the receiver profile meter may take
explicit action against flows from mis-behaving sources (those which
do not appear to honor the BECN).
The key difference between these two proposals is the marking action
in the interior routers (stochastic vs. deterministic marking). As
such, it does not appear that the schemes are compatible using the
same PH FECN bit, unless the network is configured such that there is
a receiver profile meter downstream of every interior router, in
which case the interior routers can be configured to mark the FECN
bit as described in [Clark97] and the receiver profile meters can
filter the FECN indications as appropriate prior to forwarding the
packet further downstream.
Blake Expires: June 1998 [Page 15]
INTERNET-DRAFT Packet Marking December 1997
It is not clear how well ECN will scale for multicast traffic, due to
the potential implosion of BECNs at a multicast source [CCBES].
4. Differentiation Mechanism Categorization
Packet marking is intended to invoke one or more differentiation
mechanisms -- either in a host (source/destination) or in the routers
along the packet's transit path -- so as to differentiate the data
transport performance. In the above statement we are using the term
"router" generally to refer to any node in the packet's path which
forwards it towards the destination; e.g., a profile-meter [Clark97]
or firewall. In Sec. 3 we examined several proposed uses of a packet
marking facility and the differentiation mechanisms they might
invoke. In this section we reverse the analysis by examining the
differentiation mechanisms which could be implemented within a host
or router, and then detail which of the mechanisms are invoked by the
proposals described in Sec. 3. It should be noted that, for ease of
deployment, and with the exception of FECN, most of the proposals
attempt to provide differentiated services using only router
mechanisms, without substantial changes to the host (if any).
4.1 Host Packet Processing Mechanism Categorization
Host packet processing mechanisms relevant to differentiated services
can be categorized into the following functions:
o Path selection -- selection of the output interface or
next-hop router,
o Transmission scheduling -- selection of the next packet to
forward from the transmission queue;
selection of the link-layer priority,
o Reception scheduling -- selection of the next packet to
process and deliver to the transport
layer from the reception queue,
o Congestion control -- selection of the transmission rate, or
of the interval over which to suspend
transmission, based on congestion
indications from the network.
Host path selection is rarely invoked since most hosts are single-
homed (single network interface) and most do not run a routing
protocol which would allow them to intelligently select the next-hop
router for service-specific routing (Sec. 3.2.4); however, nothing
precludes a properly configured host from making service-specific
route selections.
Non-FIFO host transmission scheduling may be invoked to promote a
delay prioritized packet, or one within a precedence service class
(or within an aggregated Integrated Services class if the host
supports aggregation). It may also be used to control the
transmission rate of a flow, if that flow's service class is known to
Blake Expires: June 1998 [Page 16]
INTERNET-DRAFT Packet Marking December 1997
be rate regulated by the network. Host transmission scheduling may
also invoke link-layer prioritization features; e.g., by selecting a
particular ATM QoS VC, or by marking the packet with a particular
802.1p priority [IS802].
Non-FIFO reception scheduling may in principle be invoked by a delay
prioritized or precedence service class packet, or by a transport-
isolated class (where one transport protocol has priority over
others). Loss priority may be utilized if the receiving host's
buffer is saturated and packets must be discarded. However, any non-
FIFO or drop-prioritized reception processing may introduce
complexity in the receiving host's networking protocol stack that is
not justified in practice by improved performance.
Receipt of a backwards explicit congestion notification should
directly affect the congestion control function of the source's
transport protocol (causing it to reduce its rate of transmission).
Also, a transport protocol modified as described in [Feng97] will
mark packets for priority as required to achieve a negotiated
throughput.
4.2 Router Packet Processing Mechanism Categorization
Router packet processing mechanisms relevant to differentiated
services can be categorized into the following functions:
o Reception scheduling -- selection of the next packet to process
from the reception queue,
o Packet classification -- identification of the flow by header
filtering; identification of the
differentiation mechanisms to apply by
PH lookup,
o Path selection -- selection of the next-hop node,
o Traffic policing -- setting of PH based on the monitored
rate of traffic within a flow/class,
o Buffer Management -- selection of the queue/discard action;
pushout under congestion,
o Transmission Scheduling -- selection of the next queue to service
for transmission.
Note that buffer management and transmission scheduling can be
strongly coupled.
Reception scheduling is normally FIFO in routers. This is usually
because the forwarding subsystem is integrated with the header
processing subsystem, and the packet must be received from the
reception queue (if any) before the PH field can be examined. In
addition, most wire-speed routers do not encounter significant
reception queues.
Any differentiation mechanism which utilizes packet marking will
require that the routers check the PH field to determine the
Blake Expires: June 1998 [Page 17]
INTERNET-DRAFT Packet Marking December 1997
differentiation mechanisms to apply to the packet. An Integrated
Services conformant packet indication may invoke flow classification.
Also, some explicit service class may be defined which invokes some
packet classification function at some point in the network for
authorization purposes or for finer-granularity service
differentiation.
Path selection may be affected if service-class based routing is
configured. In this case the PH field will determine the set of
routes to search first.
Traffic monitoring and policing is required by several of the best-
effort service allocation proposals, to determine whether the traffic
of a flow is within a negotiated profile (or how it varies relative
to it). Integrated Services policing can utilize a non-conformant
packet indication to signal an out-of-profile packet within a
reserved flow.
Any of the delay priority or explicit service class markings may
direct the buffer management subsystem to queue the packet into a
non-default queue. Services requiring isolation and/or provisioned
resources in the router (e.g., buffer space, bandwidth) generally
require a separate queue. Within a queue, an explicit drop priority
or precedence service class marking, or an out-of-profile indication,
may invoke a buffer management discard action, depending on the
current state and history of buffer occupancy in the queue [RED].
The same buffer discard logic may be utilized to set FECN (if the
transport protocol is ECN-capable).
When multiple queues are supported to provide delay prioritization,
provisioned service class bandwidth, or isolation, a queue scheduling
algorithm must be implemented to determine which queue's head-of-line
packet to transmit next. The scheduling algorithm could vary in
complexity from simple strict priority, to one of the more
sophisticated rate-based scheduling algorithms such as WFQ or CBQ
[HPFQA, HFSC, CBQ]. While the complexity of strict priority queueing
is usually O(1), the complexity of the more sophisticated rate-based
scheduling algorithms is usually O(log N) for N queues. This may
impose an upper bound on the number of economically implementable
delay priorities or service classes.
It is useful to examine the amount of state that each packet marking
application imposes within routers. Some of the stateless
applications include an explicit delay/drop priority (when
implemented as strict priority) or service class or out-of-profile
indicator that indicates drop-preference. These impose no per-flow
or per-class state in the routers, although any network authorization
or policing/monitoring function which sets these indicators may
require per-flow state (although these functions are usually located
near the source or receiver). An Integrated Services aggregating
router requires per-flow classification and policing state prior to
service class aggregation. Any provisioned explicit service class
Blake Expires: June 1998 [Page 18]
INTERNET-DRAFT Packet Marking December 1997
mechanism will impose per-class buffer management and scheduling
state in the routers. In the general case, packet marking
applications only impose per-flow state at aggregation points in the
network where the number of flows is not large.
4.3 Biased vs. Substitute Best-Effort Router Mechanisms
We have examined in a general way the different router subsystems
which may be parameterized to differentiate packet transport
behavior. This analysis gives us the basis to more specifically
examine the scope of differentiation that can be provided. We
characterize a router's differentiation capabilities into two broad
categories: biased and substitute best-effort mechanisms.
Normal best-effort service is generally considered to be fair (under
the assumption of cooperating, well-behaved applications utilizing
transport protocols with TCP-like congestion control). This service
is often implemented using a single FIFO queue, with no per-flow
identification, or path-selection, or special buffer management or
scheduling (we accept the possibility of active buffer management,
perhaps incorporating fairness enforcement mechanisms [ACTIVE, RED,
FRED, Floyd97]). Such a service can be considered fair because there
are no explicit biases in the packet handling behavior. It is in
fact the case that there are biases in best-effort service, even
among well-behaved applications (e.g., flows with large RTTs achieve
lower througput [Floyd91]), but these biases are artifacts of the
congestion control algorithms utilized and are not due to explicit
biasing mechanisms in the network. We define a biased mechanism here
as one which explicitly allocates more resources (e.g., buffers,
queue service rate) to some set of traffic flows, permitting these
flows to achieve superior (loss/delay/throughput) performance over
other flows. We will more carefully define a substitute best-effort
mechanism in Sec. 4.3.1, but for now we define it as mechanism which
does not provide additional resources to flows over any long time
scale, but which may temporally provide such resources over short
time scales.
4.3.1 Transmission Mechanism Categorization
Router transmission mechanisms -- buffer management, packet
scheduling, and link-layer priority selection -- are one basic means
by which a router can differentiate the service of a flow. To
provide a framework for the discussion of the possible axis of
differentiation we will first describe a hypothetical router
transmission subsystem which enforces per-flow link-fairness. This
design is motivated by the discussion in [CCBES]. Such a system
would incorporate per-flow queueing with a fair (equal service weight
per queue) scheduling algorithm; for example a variant of WFQ
[HPFQA]. In a system with finite buffers, per-flow buffer management
would also be implemented. The buffer management system might impose
absolute bounds on the instantaneous buffer occupancy of a flow. To
account for bursty flows, a RED-based discard policy might be
Blake Expires: June 1998 [Page 19]
INTERNET-DRAFT Packet Marking December 1997
implemented based on the long-term traffic history of the flow (and
not the flow's short-term average queue occupancy). The goal of this
buffer management system would be to deliver equivalent packet loss
rates to flows with equal long-term average rates (within some time
horizon), with minimal packet loss for flows under-utilizing their
fair-share rate, and higher loss rates for flows attempting to exceed
their fair-share rate. The goal of the scheduling system is to
provide equal service rates to all flows under backlogged conditions.
In practice, once flows had stabilized to their fair-share rate, only
the bursty flows would queue more than one packet at a time. Note
that maintaining per-flow state (dynamically generated in the case of
best-effort sources) is probably too complex for economical
implementation; it is only proposed as a hypothetical example to
highlight some properties of flow service differentiation. Note also
that link-fairness is not the only, nor necessarily the best
definition of fairness in a network; it is used here only for
illustrative purposes.
Flow service can be differentiated by modifying any of the parameters
of our hypothetical transmission subsystem. For example, to increase
the nominal throughput of a flow, that flow's queue weight could be
increased, thereby increasing its backlog drain rate. To decrease
the probability of packet loss, a flow's maximum allowable buffer
consumption could be increased, and the parameters of the discard
policy could be modified to preferentially allow the flow's packets
to have access to buffers slot under congestion. To increase the
probability of loss under congestion, the reverse actions could be
taken, and threshold mechanisms could be implemented if there are
packets of multiple drop priorities within a flow (as in best-effort
service allocation). To reduce the queueing delay of a flow's
packets (without increasing the long-term service rate of the flow),
the scheduling algorithm could be amended so that a delay prioritized
packet could be transmitted prior to that flow's queue receiving its
turn at the scheduler. This delay prioritization capability would be
bounded by a token bucket with a token pool scaled proportionally to
the typical burst size of the flow, and with a token rate equal to
the flow's fair-share rate (thus turning the flow's queue into a
variable bit-rate shaper). Each of these modifications are an
example of a biased differentiation mechanism. Note that the impact
of this biasing is degradation of the service of other flows under
contention for transmission resources.
We can further categorize biased transmission mechanisms into
provisioned and non-provisioned mechanisms. Provisioned mechanisms
are specifically configured to provide a particular service for a
flow, such as explicit delay or drop priority, or throughput priority
within a precedence service class. A non-provisioned biased
mechanism such as FECN implements a biased discard policy for packets
which are marked as ECN-capable. Such a mechanism is not intended to
enable biased service for well-behaved applications, however, they
introduce the possibility of service bias for badly behaved
applications (e.g., those that do not honor BECN) by allowing them
Blake Expires: June 1998 [Page 20]
INTERNET-DRAFT Packet Marking December 1997
to achieve better-than-fair throughput due to the lower loss rate.
In contrast to fair and biased transmission mechanisms, we may also
hypothesize the possibility of substitute best-effort mechanisms.
The stability of the current Internet depends on the fact that its
existing service model is fair [CCBES]. Introduction of biased
service capabilities will require provisioning and traffic
regulation. However, the normal "best-effort" service available to
applications may not suit all of their needs, and it may be the case
that applications could improve their performance without subscribing
to any particular provisioned differentiated service from the
network. This would only be possible if these alternative mechanisms
did not aggravate network stability, which implies that they must
also be fair. For the purposes of this discussion we define a
substitute best-effort mechanism as fair if, when selected by a flow,
it does not degrade the overall performance of other active flows,
where we define "performance" for normal best-effort flows as average
throughput, loss, and delay. One example of a substitute best-effort
mechanism would be queueing isolation to protect a flow with a long
RTT (note that this does not strictly meet our definition of fairness
since, if selected, other flows are not able to achieve an unfair
share of the link capacity). Another example would be a mechanism
which allowed a flow to trade its fair-share service rate or its
average packet loss rate for low queueing delay [RFC1046]. Trading
rate for low delay could be achieved by giving a flow delay priority
within a token bucket whose token rate was less than the flow's fair-
share service rate. Trading loss for low delay could be achieved by
queueing the flow in a delay-prioritized queue with a small per-flow
buffer slot quota. Such a capability might be useful for low-
throughput interactive applications like IP telephony.
An alternative mechanism would allow a flow to trade queueing delay
or service rate for lower loss. The former could be achieved by
queueing the packet in a queue with a larger per-flow quota but with
low delay priority. The latter could be achieved by queueing the
flow in a larger buffer with a lower fair-share service rate. This
capability might be useful for some short-term transaction traffic
(e.g., RPC, some WWW) which is insensitive to queueing delay but
which is sensitive to RTT delays.
The third alternative, allowing a flow to trade delay or loss for an
improved service rate, does not make sense in the context of a
congestion-controlled best-effort network (See Sec. 6).
It is not known to the author whether the substitute best-effort
mechanisms proposed have been researched, and whether they exacerbate
fairness and stability within a best-effort network. Furthermore,
although we have discussed the transmission service mechanisms in the
context of per-flow queueing and buffer management, in fact one of
the goals of packet marking differentiated services is to eliminate
per-flow state in the core of the network. Aggregate queueing and
buffer management mechanisms which provide differentiated transport
Blake Expires: June 1998 [Page 21]
INTERNET-DRAFT Packet Marking December 1997
services may suffer from fairness problems within a service class
similar to the current best-effort Internet with single FIFO queues
[Floyd97].
4.3.2 Path Selection Mechanism Categorization
We can categorize path selection mechanisms using the same framework
as was used in Sec. 4.3.1. A provisioned biased path selection
mechanism would compute a route based on metrics (e.g., delay, loss,
link rate, and transmission cost) that would suit the requirements of
a particular class of traffic. The flows that had access to these
paths would be authorized prior to entry, and their traffic would be
regulated. The paths would be provisioned to satisfy the regulated
traffic load (perhaps using statistical assumptions). The "out-of-
class" traffic taking these paths would also be regulated to preserve
the service level of the in-class flows.
Non-provisioned biased path selection mechanisms (which also fit our
definition of substitute fair mechanisms) would not utilize per-flow
authorization and traffic regulation. Examples include computing
paths which avoid satellite hops for delay sensitive traffic, or
which avoid wireless hops for loss sensitive traffic. Whether these
alternative paths would actually improve the service of the flows
which took them may depend on the relative load on the paths from
other traffic flows. The assumption that would justify their use by
non-regulated flows is that these paths are in some other way
inferior to the normal shortest-hop path (longer delay, higher loss,
or lower link rate).
5. Service Categorization
Marking a packet invokes a particular router or host differentiation
mechanism on that packet. This facility is used to instantiate a
service for a flow of traffic. Some of the packet marking
applications discussed in Sec. 3 imply a specific differentiation
mechanism (e.g., FECN); others imply a general service from the
network (e.g., precedence) without implying any particular
differentiation mechanism implementation.
In this section we propose a set of criteria for categorizing
differentiated service implementations.
5.1 Service Granularity
We can categorize differentiated services implementations by the
granularity at which they act (at which they differentiate transport
performance). At the lowest level of granularity is per-packet
mechanisms such as the services described in Sec. 3.3 and 3.4, where
packets are marked based on the characteristics of the corresponding
packet flow relative to some traffic specification. These
differentiation mechanisms may be invoked for a subset of the flow's
Blake Expires: June 1998 [Page 22]
INTERNET-DRAFT Packet Marking December 1997
packets, with the aggregate effect (and its interaction with host
congestion control) delivering the desired service. FECN is another
example of a per-packet differentiation service.
Per-flow differentiated services utilize the same packet marking for
each packet of a flow. Examples of this type include explicit delay/
drop/network control priority, explicit service class indication, and
service-based route selection. This granularity can be relaxed to
provide per-source host differentiation, where all of the packets
transmitted from a particular source receive the same packet marking
(or in the case of the schemes describe in Sec. 3.3, the individual
flows of the source are not distinguished). Per-source
differentiation is particularly suitable when there is no need for
per-application differentiation (for example when all of the source's
flows have homogeneous service requirements). Note that the
distinction between per-packet services and per-flow/source services
is not crisp.
Per-network differentiated services act on the aggregate of flows
from a particular cluster of nodes, from a particular subnet, or from
a particular site (e.g., VPN service). As is the case for per-source
services, per-network services are appropriate whenever the
aggregated flows have homogeneous service requirements.
Finally, we include per-receiver services such as that described in
[Clark97] (and also RSVP aggregation). This class of services would
require tight integration with host congestion control or network
policing mechanisms to ensure appropriate behavior (i.e., reduction
in transmission rate due to congestion experienced by out-of-profile
packets).
5.2 Service Invocation
Another dimension of service categorization is the point of service
invocation. The earliest point of possible invocation is at the
source (at the application layer). Examples of possible source-
invoked services are explicit delay/drop/network control priority,
explicit service class indication, and Integrated Services conformant
packet indication. Source-invoked services are particular useful
where end-to-end differentiated service is required, since this
exposes the service interface to the application [QOSP].
An alternative service invocation point is at some point(s) within
the network. Examples here may include the services described in
Sec. 3.2. and 3.3, Integrated Services non-conformant packet
indication and aggregation, and FECN. The service may be invoked on
any granularity of traffic (see Sec. 5.1) and requires configuration
within the network to identify the flows or aggregate of flows to
which the service should be applied. The scope of such a service is
usually intermediate (across a network or network-to-network [QOSP]).
Network-invoked services are useful whenever network authorization
and policing are required, or whenever a set of flows with
Blake Expires: June 1998 [Page 23]
INTERNET-DRAFT Packet Marking December 1997
homogeneous service requirements can be aggregated.
A hybrid invocation model would permit the source to set the PH field
to request a particular differentiated service, while allowing the
network to authorize and police the traffic from any source which is
allowed to utilize the service. Such a service model might permit
less granular configuration and authorization state in the network
(i.e., no per-flow and only per-source state).
Receiver-invocation of differentiated service is also possible, but
requires some signaling mechanism to allow the receiver to control
the sending rate of a source or the packet markings used across the
network. The canonical example of a receiver-invoked service is
the Integrated Services via RSVP signaling. Another example is ECN
via a receiver-generated BECN (which could be influenced by a
receiver profile meter as describe in [Clark97]). A receiver-to-
network signaling protocol similar to RSVP which did not rely on the
appropriate behavior of the source to enable the differentiated
service (or make it deployable) is conceivable, although the author
is not aware of a proposal for such a signaling mechanism in the
context of packet marking differentiated services.
Service invocation can also be characterized in time as well as in
space. An application, source, or group of sources may have
negotiated on ongoing arrangement with a service provider to provide
a differentiated service for marked packets. This type of static
service allocation may involve a variety of time- and destination-
specific constraints which limits service availability, but it does
not require signaling or any form of immediate configuration to
permit utilization of the service; sources meeting these constraints
as well as constraints on traffic levels may begin to utilize the
service immediately by marking packets (or the network may
automatically mark them). In contrast, dynamic service allocation
involves some form of pre-negotiation (i.e., via signaling) between
the source(s)/receiver(s) and the network for service prior to
availability. This negotiation may involve service start/stop times,
traffic levels, service characteristics, pricing, etc. The network
will be required to dynamically configure authorization and policing
policy mechanisms to instantiate the service, and may also have to
dynamically provision resources within the network interior.
5.3 Service Behavior
A differentiated service's behavior can be categorized by whether it
is biased or offers a substitute best-effort service. Biased
services can be further categorized as to whether they are
provisioned or non-provisioned. For a more detailed discussion of
the differences see Sec. 4.3
A major implementation issue with biased services is the need to
regulate the amount of traffic which can invoke them, usually via
source traffic shaping, or network authorization and traffic
Blake Expires: June 1998 [Page 24]
INTERNET-DRAFT Packet Marking December 1997
policing, as well as appropriate provisioning within the network.
This is required to satisfy whatever delay/loss/throughput
performance guarantees are associated with the service. The behavior
of the service in the presence of non-conformant traffic can be
characterized as to how the non-conformant traffic is handled. Such
traffic could be discarded automatically by the network [TWOBIT], or
it could be handled with lower priority and suffer a higher
probability of loss or delay [Clark97]. The effect of the presence
of non-conformant traffic on the conformant subset is also relevant.
In general, a provisioned biased differentiated service could be
defined as a set of probability density functions of packet delay,
loss, and flow throughput relative to some statistical traffic model
(and time interval). In practice, however, determining this level of
information detail for each differentiated services customer would be
difficult if not impossible. From these density functions service
assurance levels, measured as the probability of service
availability, could be inferred, although without stationarity
assumptions the service failure modes could not be predicted. A
differentiated services user will be primarily interested in the
degradation behavior of the service. Differentiated services
implementations can be characterized by whether service failure
(e.g., due to under-provisioning or network infrastructure failure)
results in a soft degradation in the delay/loss/throughput metrics,
a complete degradation to traditional best-effort service, or total
interconnectivity failure. Furthermore, implementations can be
characterized by the promised maximal duration and frequency of
service failure.
5.4 Direction of Value
An important means of categorizing differentiated services is by
examining in which direction value flows when a differentiated
service is provided for a packet flow (either to the source or to the
receiver). In any point-to-point exchange of traffic there is
usually a benefit to both ends of the conversation; however, it is
often the case that the level of direct benefit to one party exceeds
the level to the other. This is important to a service provider as
it might be more appropriate to implement pricing policies which
target the primary beneficiary.
Most of the packet marking applications examined provide benefit to
a source of traffic by preferentially handling that source's packets
for improved transport performance. Since these mechanisms are not
necessarily destination-specific, they can be viewed as primarily
benefiting the source. As such one would expect that the source (or
his proxy) would be the entity charged for the differentiated service
(e.g., source-purchased traffic profile). When the traffic profile
is associated with a particular set of destinations, and when
reverse-path services are utilized, we can consider the value to be
bi-directional, and the charges for the service can be distributed
between the end-points (often the same organizational entity).
Blake Expires: June 1998 [Page 25]
INTERNET-DRAFT Packet Marking December 1997
The model of source-pricing of differentiated service may not suit
WWW-based information delivery, since the value of service flows
primarily to the receiver of information and there is no incentive
for the information source to request service differentiation for
a particular subset of receivers (this changes if there is some
charge to the receiver associated with information retrieval). For
such applications a receiver-invoked service such as describe in
[Clark97] may be most appropriate. Signaling across the network to
(or near) the source to initiate differentiation may permit more
sophisticated receiver-invoked services (e.g., RSVP). However, there
must likely be some associated settlement mechanism to incent the
service provider or source to deploy such a protocol, and scalability
and interoperability factors must also be weighed.
An interesting problem arises for multicast applications where the
receiver is the main beneficiary (arguably commercial broadcast
applications are an example). When packet marking is invoked at or
near the traffic source, all receivers of the transmission receive
the benefit of the differentiated service (if we assume that the
network does not remark the packet along different branches of the
multicast path). This may deliver greater value to some receivers
than they are willing to pay for. Conversely, if the service
provider charges more for differentiated multicast service, this may
make it difficult for the source to provide the desired service to
one or more particular receivers (in an efficient way). Receiver-
invoked service mechanisms such as described in [Clark97] may scale
poorly in a multicast environment due to BECN implosion at the
source. Also, Integrated Services aggregation suffers a variety of
scaling and heterogeneity problems for IP multicast reservations,
since the granularity of service is often too coarse (and due to
control-plane scaling problems).
6. Fairness and Congestion Control Considerations
In the absence of traffic regulation and associated network
provisioning, the stability of the Internet still depends on the use
of cooperative congestion control by all applications [CCBES,
Floyd97]. This is true even for application flows (or packet
subsets) which specify (non-provisioned) drop-preference service. A
form of congestion collapse can occur in the Internet if applications
rely on the network to discard excess packets and do not implement
closed-loop congestion control, because packets which will later be
discarded downstream could utilize bandwidth on a link which could be
better utilized by non-drop-preference congestion-controlled flows
(e.g., normal TCP) [Floyd97]. The normal router mechanisms which
would allow the non-drop-preference traffic to ramp up to the link-
rate (e.g., weighted RED) may not function effectively if the drop-
preference traffic is not well-behaved. One solution to this problem
might be to introduce aggregate bounds on the amount of drop-
preference traffic transmitted which would incent the application not
to abuse the service. However, if a drop-preference marked packet
Blake Expires: June 1998 [Page 26]
INTERNET-DRAFT Packet Marking December 1997
has such a low probability of being delivered (due to aggregate
constraints), the drop-preference facility is not very useful.
A similar form of congestion collapse can occur if badly behaved
applications which advertise ECN-capability do not respond to a
receiver BECN. This is because the router gives these packets
preferential drop priority. This could allow non-conformant
transport protocols to achieve better throughput than conformant ECN-
capable transports and non-ECN-capable transports. Although it is
not clear whether ECN introduces a fairness problem that is any worse
than the existing problem of badly behaved transports, its deployment
should be approached cautiously. One way to alleviate these fairness
problems might be to implement fairness enforcement mechanisms such
as described in [Floyd97] and [FRED] (note that these mechanisms
might contradict the scalability objectives addressed by packet
marking).
In Sec. 4.3 we introduced the concept of a substitute best-effort
service. Because substitute best-effort differentiation mechanisms
provide short-term biasing at the expense of long-term throughput,
delay, or loss, the router implementation must take active measures
to ensure that these mechanisms do not jeopardize network fairness.
The mechanisms should be engineered to ensure that sources cannot
achieve an unfair share of network resources by modulating between
substitute best-effort services. Badly behaved applications should
not be able to achieve better throughput (or to further degrade
the service of other flows) by selecting a substitute best-effort
service. One possible means of achieving these objectives is to
make the mechanisms non-work-conserving, thereby incenting the
application to select these substitute services only if the trade-off
they provide is absolutely beneficial, and to penalize badly behaved
applications which select these services.
One of the scalability objectives of packet marking differentiated
services is to eliminate per-flow state in the core of the network.
We have examined a hypothetical per-flow router transmission system
to highlight how differentiation might be provided. However, in a
scalable system, only aggregated state would be maintained (or per-
flow state would only be maintained for a small subset of the
active flows). Aggregated buffer management and queueing
implementations may suffer the same fairness problems between flows
within a service class as is exhibited today with best-effort traffic
and single-queue FIFO routers. Provisioning and traffic regulation
might alleviate these problems, but techniques such as described in
[Floyd97] and [FRED] might also be required in some circumstances.
7. Provisioning Considerations
We have used the term "provisioning" in this document to describe the
deployment and assignment of network resources for the exclusive or
preferential use by certain (sets of) traffic flows. Aggregate
Blake Expires: June 1998 [Page 27]
INTERNET-DRAFT Packet Marking December 1997
differentiation mechanisms in and of themselves cannot deliver a
quantifiable service without constraints on the aggregate amount of
traffic which invokes those mechanisms. The resource allocation
policy can be implemented in each interior router, for example by
service class-specific queues with provisioned minimal bandwidth
levels and buffer quotas. Alternatively, resource allocation can be
implemented more globally, relying on traffic authorization and
policing at the network edge and stateless differentiation mechanisms
in the network interior. Whichever choice is preferred depends on
scalability concerns, the aggregate amount of traffic utilizing a
particular differentiated service, as well as the level of
statistical performance assurance associated with the service.
A basic motivation of differentiated services is to provide tiered
levels of statistical assurance of service for a particular traffic
load, with tiered pricing to match the service provider cost and
customer utility associated with each level. Service assurance was
discussed in detail in Sec. 5.3. To elaborate on that discussion,
the statistical assurance of a differentiated service depends upon
uncertainty about the interior transit path taken by appropriately
marked packets, on the statistical multiplexing gain assumed in the
service allocation policy, and on the instantaneous behavior of other
differentiated services users. As such, there is potentially a
strong time-dependence on service quality.
When provisioning a differentiated service, a provider must take into
account the dimensioning of the network, as well as statistical
models of customer activity and traffic levels. Defining a service
allocation policy which satisfies a particular statistical assurance
level is equivalent to an admission control problem. The primary
design choices in a service allocation policy are static vs. dynamic
allocation, and domain-wide vs. hop-by-hop admission control. When
using a static allocation policy, a provider must provision more
resources than when using a dynamic allocation policy to achieve the
same level of statistical assurance since the admission control
decision must be made on historical data or traffic models and not on
instantaneous measurements of network activity. However, dynamic
admission control requires some means of signaling and/or dynamic
configuration to convey the service request to the network (and its
affected elements). This dynamic admission control decision could be
made by a centralized administrative entity (e.g., the Bandwidth
Broker in [TWOBIT]) which bases its decision on a domain-wide view of
existing service allocations (and possibly a coarse view of the
instantaneous traffic activity). Alternatively, the decision could
be made at each node along the hop-by-hop path taken by the affected
packets (e.g., the RSVP/Integrated Services model). It is difficult
to provide a strict guarantee of service along an unspecified path
at an unspecified time [Clark97], which implies that services which
promise strict performance guarantees will usually include
constraints on the available destinations or network egresses as well
as the interval of service availability. Note also that the choices
between static vs. dynamic allocation and domain-wide vs. hop-by-hop
Blake Expires: June 1998 [Page 28]
INTERNET-DRAFT Packet Marking December 1997
dynamic admission control are not mutually exclusive.
Implementation of a dynamic, domain-wide admission control policy, as
well as long term service planning, depends on the availability of
statistics on service utilization and performance. Means of
capturing the characteristics of marked traffic, such as the
utilization of a particular service class or differentiation
mechanism, packet discard distributions, queue delay distributions,
etc., are required (e.g., via new router MIB variables). Service
providers and customers may need to deploy test and measurement
applications to characterize and validate the assurance level of a
service. These mechanisms may also be needed to facilitate inter-
provider monitoring and settlement.
Service provisioning must also take into account the scalability of
the mechanisms used to provide the service. Scalability may be
affected by the amount of configuration state, network monitoring
state, router processing and transmission state, and dynamic service
signaling traffic levels and state which is required for a particular
service implementation.
8. Authorization Considerations
Authorization of use of packet marking-based biased differentiated
services is required to permit any level of service assurance.
Authorization is required at whichever point(s) in the network where
the service is invoked (see Sec. 5.2). We break the authorization
problem down into the components of packet classification, traffic
policing, and PH marking.
The packet classification component matches received packets to
statically or dynamically allocated service profiles, based on any
combination of per-source address/subnet, per-destination address/
subnet, or per-flow packet header filters. The classification
function may be deployed on site subnet boundaries, on site backbone
boundaries, on the site border to a service provider, on provider
ingress boundaries, and/or on inter-provider boundaries. The
granularity of packet classification will generally be relaxed as
the classification component moves into the interior of the network
to facilitate scalability. The classification component may also
honor source service requests (based on the PH value set by the
source).
The traffic policing component measures the instantaneous load of
packets matching a classification entry relative to some traffic
profile. This traffic profile is configured based on some source/
network or network/network agreement on the amounts and signature of
marked traffic. The traffic policing component could be based on a
simple token bucket filter [TWOBIT], or on a more sophisticated
monitoring function which takes into account the congestion-control
behavior of TCP [Feng97, Clark97]. Non-conformant packets may be
Blake Expires: June 1998 [Page 29]
INTERNET-DRAFT Packet Marking December 1997
discarded, depending on the service policy.
The PH marking component sets the PH field of each service profile-
conformant packet to invoke the differentiation mechanism(s) deployed
within the network to instantiate the appropriate service. Non-
conformant packets may be re-marked, depending on the service policy.
The classification, policing, and PH marking components within a
network must be configured whenever a service is allocated to a (set
of) sources. This may involve manual configuration for statically
allocated services, or dynamic signaling (e.g., SNMP, RSVP) for
dynamically allocated services.
Interoperability of packet marking differentiated services between
service providers depends on a joint agreement on PH semantics,
traffic profiles, and authorization policies for each service
supported.
9. Routing Considerations
Service assurance may depend on the stability of the routing system
(e.g, the prevalence of routing flap or the frequency of routing
melt-down). The deployment of service-based routing (Sec. 3.2.4)
introduces a variety of additional routing considerations. One issue
involves the existence of an incomplete service-specific path between
a source and destination (or across a domain which deploys service-
specific routing). This incomplete path might exist due to router
misconfiguration, due to different policy decisions among service
providers, or due to routing transients. In the event that a
service-specific route is not available at a router along the transit
path, we assume that the default routing entry is followed [RFC1583].
The problem occurs when the service-specific and default routes are
calculated using a different set of metrics. It may be possible that
if the default route is followed, then the packet may loop back to
a node which has a matching service-specific route entry, and a
stable routing loop may form. Although the effects of router
misconfiguration and routing transients are hard to mitigate, this
behavior may be particularly onerous since it could be hard to
detect. This problem could be avoided if, whenever a router which
implements service-specific routing has to forward a packet (with a
PH marking associated with a service-specific routing class) using
the default routing entry, the PH field is reset to indicate the
default routing class. This solution avoids the stable routing loop
problem; however, if the same PH bits are overloaded to specify both
service class queueing and route selection (based on some request
such as "Minimize Delay"), then whenever these PH bits are reset, the
service class queueing indicators are erased, and the service
provided to the flow may be degraded at downstream nodes.
Another issue is the choice of behavior in the event that a matching
service-specific routing entry is not available. The basic choices,
Blake Expires: June 1998 [Page 30]
INTERNET-DRAFT Packet Marking December 1997
"Strong TOS", "Weak TOS", and "Very Weak TOS", are defined in
[RFC1349]. The "Strong TOS" model requires that a router only
forward a packet if a matching service-specific route is a available
(otherwise the packet is discarded). The "Weak TOS" model requires
that the router use the best-matching default routing entry if a
matching service-specific route is not available (this is the
behavior assumed in the example above). The "Very Weak TOS" model
requires that the router attempt to utilize the best-matching,
numerically lowest TOS entry if neither a matching service-specific
nor a matching default entry are available. The "Very Weak TOS"
model only makes sense if the services are somehow ranked in
numerical order of precedence. Historically, the "Weak TOS" model
has been favored, since the "Strong TOS" model may penalize packets
utilizing a service class when service-specific routing is not
deployed or a particular service-specific path is broken [RFC1349].
The introduction of service-specific routing introduces an additional
criterion to the route lookup algorithm (the service class match).
The route lookup algorithm may choose to prefer an exact matching
service class value above a longest-prefix address match which does
not support the specific service (e.g., the default service class).
Whichever criterion is preferred must be standardized to prevent the
formation of routing loops amongst routers which implement contrary
policies. However, both [RFC1349] and [RFC1583] mandate the use of
the longest-prefix match as the preferred criterion, as this appears
to be the more robust option.
Whenever service-specific routing is deployed, interoperability
between service providers must be considered. There must exist some
compatibility between service-specific route calculation mechanisms
in the deployed IGP and EGP routing protocols to prevent interdomain
routing loops, and peering service providers must agree to implement
compatible policies (including the resetting of routing-sensitive PH
bits) to avoid routing loops or sub-optimal routing paths.
PH bits which affect route selection should not be modified
dynamically within a flow (on a per-packet basis) since this may
affect packet ordering and RTT estimation. Best-effort service
allocation mechanisms such as described in Sec. 3.3 should not
utilize routing-sensitive PH bit combinations to indicate the
conformance of a packet.
Deployment of service-specific routing may introduce scalability
issues due to the increased amount of routing protocol state
maintained in, as well as the increased amount of routing table
computations performed by, the network routers.
10. System Implementation Considerations
We reiterate that the goal of packet marking is to provide a
simplified, scalable mechanism for invoking service differentiation
Blake Expires: June 1998 [Page 31]
INTERNET-DRAFT Packet Marking December 1997
which avoids per-flow state in the interior of the network to the
maximum extent possible, as this appears to be a more scalable
approach than alternatives such as the RSVP/Integrated Services
model. Marked packets are handled as an aggregate. Note that the
link-fairness model described in Sec. 4.3.1 is an idealized example
which implies far too much dynamic per-flow state for practical
deployment on high-speed nodes. Note also that aggregate
differentiation mechanisms may suffer fairness problems within a
service class (see Sec. 6).
Various differentiation mechanisms may introduce performance and
scalability problems within a router implementation. One particular
example is the impact on router forwarding implementations which
rely on dynamic per-flow caching of forwarding state (e.g., IPv6
Source Address/Flow Label caching as described in [RFC1883]). Such
implementations may enjoy a performance advantage since the first
packet of a flow is searched using the traditional router forwarding
and classification algorithms to determine the next outgoing link,
the appropriate service class, the appropriate delay/drop priority,
etc., while subsequent packets of the flow can be forwarded using a
cache lookup which can usually be performed using an O(1) algorithm
(although this advantage may come at the cost of scalability in terms
of the number of simultaneous flows supported at wire-speed). The
caching algorithm as described in [RFC1883] assumes that the PH field
of a packet remains constant within the six second caching window of
a flow. If the PH field is used to affect the delay or drop priority
of a packet, and if the PH field is modified dynamically to indicate
conformance of the packet to some service profile (see Sec. 3.3),
then the caching algorithm may prevent the router from taking this
indication into account. This problem can be avoided if the PH field
is defined as part of the cache lookup key. Modified values in the
PH field signify a separate "flow" which will require traditional
classification (at least for the first modified packet header).
Alternatively, any differentiation mechanism which is determined by
the PH value may be excluded from the set of cached state and checked
for each individual packet.
Another example is the requirement to recompute the IPv4 header
checksum whenever the PH field is modified. This would be required
for profile meters as defined for example in [Clark97]. [SIMA], and
[TWOBIT]. This would also be required for IPv4 routers which deploy
FECN. IPv4 routers already recompute the IPv4 header checksum
whenever they decrement the TTL of a packet. However, FECN
introduces a potential implementation constraint for routers which
utilize distributed forwarding across a switching fabric, since the
processing component which performs routing and packet classification
and which decrements the packet's TTL may lie across a switching
fabric from the output interface queues. In general, the processing
component which recomputes the IPv4 header checksum must have
knowledge of the state of the targeted output queue whenever FECN is
implemented.
Blake Expires: June 1998 [Page 32]
INTERNET-DRAFT Packet Marking December 1997
Router implementation complexity and performance scalability will be
affected by the number of output interface queues which are
implemented to provide service differentiation, as well as by the
complexity of the scheduling algorithms used. In addition, per-class
memory requirements and the processing requirements to maintain per-
class state will also have an impact. Maintenance of configuration
parameters (e.g., for flow/source/destination classification) and
network management counters (e.g., for service performance
monitoring) may increase memory requirements and introduce additional
performance constraints.
Compatibility with application expectations for network behavior is
critical. Routers may implement aggregated service differentiation
mechanisms using multiple queues. As a consequence, modulating
between different PH markings may cause different packets of a flow
to be serviced using different queues, which may result in packet
reordering. Applications which modulate between PH markings (e.g.,
to signify drop priority for multiple layers of a video signal)
should expect that the packet ordering be maintained. Consequently,
application-visible differentiation mechanisms, as well as network-
invoked differentiation mechanisms should utilize sets of PH markings
which are guaranteed to be serviced within the same queue (and with
the same routing metrics).
11. Standardization Considerations
Packet marked differentiated services cannot be deployed within the
public Internet without some level of standardization. In
particular, the semantics of some of the PH bits must be defined to
allow deployment of interoperable routers, authorization components,
admission control components, and network management agents (we
assume that some of the available PH bits may be reserved for
network-specific use). Specification of those PH bits which may be
changed dynamically in-flight is needed to avoid packet reordering
problems (see Sec. 10). Specification of the PH bits which are
allowed to affect route selection is required for interoperability of
routing protocol implementations. Specification of the PH bits which
may be set by the application or source host (e.g., for substitute
best-effort services) and which are not likely to be changed or reset
in-flight is required for interoperable application development.
Network MIB variables and dynamic signaling protocols necessary for
service configuration and monitoring must be specified. Furthermore,
basic implementation requirements which are essential for the stable
operation of the network should also be specified (e.g., thou shalt
prevent drop-preference traffic from starving normal best-effort
traffic).
Incremental deployment strategies for packet marked differentiated
services may be required if the IPv4 Precedence/TOS field semantics
are redefined from their specification in [RFC795] and [RFC1349].
This may be needed for example to preserve routing protocol traffic
Blake Expires: June 1998 [Page 33]
INTERNET-DRAFT Packet Marking December 1997
prioritization based on the IPv4 Precedence field in networks where
the new PH semantics are incrementally honored. This may also be
required where a "worse-than Routine" drop priority level must be
defined to implement a particular differentiated service, and packets
arrive to the network which are not sourced or re-marked to use the
new PH semantics and are instead marked using the "Routine"
Precedence value (the routers may interpret the "Routine" Precedence
value to indicate "worse-than Routine" drop priority).
Another area of potential standardization is the interaction and
compatibility between packet marked differentiated services and the
traditional Integrated Services. Also, service measurement
methodologies may be defined and specified as a Best Current
Practice.
Interoperability of packet marked differentiated services between
different service providers may require the standardization of the
semantics and expected behavior of a small set of differentiation
mechanisms and/or service classes to allow compatible exchange of
traffic.
Those aspects of packet marking which should remain implementation-
dependent include the particular buffer management, scheduling, and
authorization mechanisms and policies used to instantiate a set of
differentiated services.
12. Security Considerations
As discussed in Sec. 2, the wide-spread deployment of IP Security
obscures the header fields which are traditionally used for per-flow
packet classification. Therefore, deployment of packet marking
differentiated services eliminates a disincentive to the deployment
of IP Security.
Because the differentiation mechanisms which are deployed will likely
introduce service bias, new denial-of-service attacks may be
introduced. As examples, host transport protocols which advertise
ECN capability but which do not respond appropriately to a BECN may
degrade the performance of other users and applications, as may
unauthorized use of priority or service class indications.
Unauthorized use of a network control priority indication may permit
an attacker to severely degrade the performance of the network.
Furthermore, an attack on the differentiated services authorization,
signaling, or configuration mechanisms may permit theft-of-service or
may enable a severe denial-of-service attack. As a consequence,
authorization, signaling, and configuration mechanisms must be
strongly protected (e.g., by authentication). Access to provisioned
biased services must always be authorized, and routers must implement
active measures (or intrinsic mechanism design) to enforce fairness
amongst users of substitute best-effort services. Network control
priority in particular must be authorized, for example by always
Blake Expires: June 1998 [Page 34]
INTERNET-DRAFT Packet Marking December 1997
resetting the associated PH bit(s) on host access links (this may be
difficult to implement on shared-media subnets), or by only honoring
the network control priority indication from configured peers.
The IP Security Authentication Header (AH) does not cover the IPv4
Precedence/TOS field in the integrity check value computation [AH].
This behavior is in fact essential for the deployment of network-
invoked differentiated services where the source host is unaware of
the PH value which will be delivered to the destination host, since
it may be changed in-flight. In the case where the source host is
authorized to select the PH value, this AH behavior does not provide
end-to-end authentication and integrity of the PH value. The AH
header format and integrity check value computation could be
redefined to incorporate an application-selectable mask on the PH
field which would allow the application to specify the particular PH
bits which might require end-to-end authentication (so as to help
determine denial-of-service attacks within the network). However,
end-to-end integrity of the PH field does not guarantee that a
differentiated service has been delivered, since the network is free
to ignore the PH field. Separate measurement and assurance
mechanisms are needed to ensure that any negotiated differentiated
services are being provided.
13. Acknowledgements
The issues examined in this memo have been topics of discussion
within the Internet community for many years. As such, the author
does not claim credit for the originality of any of the ideas herein,
and has made an earnest attempt to reference their original
proponents. Assistance from the community in documenting the origins
of these ideas is appreciated.
The author would like to specifically acknowledge the assistance of
Janet Andersen, Ed Bowen, Charles Burton, Ed Ellesson, Brian
Haberman, and Hal Sandick. The author would also like to thank Fred
Baker and Steve Deering for insights obtained during both public and
private conversations.
14. References
[ACTIVE] B. Braden et. al., "Recommendations on Queue Management
and Congestion Avoidance in the Internet", Internet Draft
<draft-irtf-e2e-queue-mgt-recs.txt>, March 1997.
[AH] S. Kent and R. Atkinson, "IP Authentication Header",
Internet Draft <draft-ietf-ipsec-auth-header-02.txt>,
October 1997.
Blake Expires: June 1998 [Page 35]
INTERNET-DRAFT Packet Marking December 1997
[Bohn93] R. Bohn, H. Braun, K. Claffy, and S. Wolff, "Mitigating
the coming Internet crunch: multiple service levels via
Precedence", submitted for publication, November 1993,
ftp://ftp.sdsc.edu/pub/sdsc/anr/papers/precedence.ps.Z.
[CBQ] S. Floyd and V. Jacobson, "Link-sharing and Resource
Management Models for Packet Networks", IEEE/ACM
Transactions on Networking, Vol. 3 no. 4, pp. 365-386,
August 1995.
[CCBES] C. Lefelhocz, B. Lyles, S. Shenker, and L. Zhang,
"Congestion Control for Best-Effort Service: Why We Need a
New Paradigm", IEEE Network, Vol. 10, no. 1, January 1996.
[Clark97] D. Clark and J. Wroclawski, "An Approach to Service
Allocation in the Internet", Internet Draft
<draft-clark-diff-svc-alloc-00.txt>, July 1997.
[CLASSY] S. Berson and S. Vincent, "A "Classy" Approach to
Aggregation for Integrated Services", Internet Draft
<draft-berson-classy-approach-00.txt>, March 1997.
[Crow97] J. Crowcroft, "All You Need Is Just One Bit", keynote
presentation, IFIP Conf. on Protocols for High Speed
Networks, October 1996,
http://www.cs.ucl.ac.uk/staff/jon/hipparch/dollarbit.
[ECN94] S. Floyd, "TCP and Explicit Congestion Notification",
ACM Computer Communications Review, Vol. 24 no. 5, pp.
10-23, October 1994.
[ECN97] K. Ramakrishnan and S. Floyd, "A Proposal to Add Explicit
Congestion Notification (ECN) to IPv6 and to TCP",
Internet Draft <draft-kksjf-ecn-00.txt>, November 1997.
[ESP] S. Kent and R. Atkinson, "IP Encapsulating Security
Payload", Internet Draft <draft-ietf-ipsec-esp-v2-01.txt>,
October 1997.
[Feng97] W. Feng, D. Kandlur, D. Saha, and K. Shin, "Adaptive
Packet Marking for Providing Differentiated Services in
the Internet", Univ. Michigan Technical Report
CSE-TR-347-97, October 1997,
http://www.eecs.umich.edu/~wuchang/work/pmg.ps.Z.
[Ferg97] P. Ferguson, "Simple Differential Services: IP TOS and
Precedence, Delay Indication, and Drop Preference,
Internet Draft <draft-ferguson-delay-drop-00.txt>,
November 1997.
Blake Expires: June 1998 [Page 36]
INTERNET-DRAFT Packet Marking December 1997
[Floyd91] S. Floyd, "Connections with Multiple Congested Gateways
in Packet-Switched Networks Part 1: One-way Traffic",
Computer Communications Review, Vol.21, No.5, October
1991, p. 30-47, ftp://ftp.ee.lbl.gov/papers/gates1.ps.Z.
[Floyd97] S. Floyd and K. Fall, "Router Mechanisms to Support End-
to-End Congestion Control", LBNL Technical Report,
February 1997, http://ftp.ee.lbl.gov/papers/collapse.ps.
[FRED] D. Lin and R. Morris, "Dynamics of Random Early
Detection", Proc. ACM SIGCOMM 1997, September 1997.
[GBH97] R. Guerin, S. Blake, and S. Herzog, "Aggregating RSVP-
based QoS Requests", Internet Draft
<draft-guerin-aggreg-rsvp-00.txt>, November 1997.
[HFSC] I. Stoica, H. Zhang, and T. Ng, "A Hierarchical Fair
Service Curve Algorithm for Link-Sharing, Real-Time and
Priority Services", Proc. ACM SIGCOMM 97, September 1997.
[HPFQA] J. Bennett and Hui Zhang, "Hierarchical Packet Fair
Queueing Algorithms", Proc. ACM SIGCOMM 96, August 1996.
[IPv6] S. Deering and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", Internet Draft
<draft-ietf-ipngwg-ipv6-spec-v2-01.txt>, November 1997.
[IS802] M. Seaman, A. Smith, and E. Crawley, "Integrated Services
Mappings on IEEE 802 Networks", Internet Draft
<draft-ietf-issll-is802-svc-mapping-00.txt>, July 1997.
[May97] M. May, J. Bolot, C. Diot, and A. Jean-Marie, "1-Bit
Schemes for Service Discrimination in the Internet:
Analysis and Evaluation", INRIA Research Report, August
1997,
http://www.inria.fr/rodeo/personnel/mmay/papers/rr_1bit.ps.
[McCanne] S. McCanne, V. Jacobson, and M. Vetterli, "Receiver-driven
Layered Multicast", Proc. ACM SIGCOMM 96, August 1996.
[MPLS] R. Callon, P. Doolan, N. Feldman, A. Fredette, G. Swallow,
and A. Viswanathan, "A Framework for Multiprotocol Label
Switching", Internet Draft
<draft-ietf-mpls-framework-01.txt>, July, 1997.
[QOSP] S. Bradner, editor, "Internet Protocol Quality of Service
Problem Statement", Internet Draft
<draft-bradner-qos-problem-00.txt>, September 1997.
[RED] S. Floyd and V. Jacobson, "Random Early Detection Gateways
for Congestion Avoidance", IEEE/ACM Transactions on
Networking, August 1993.
Blake Expires: June 1998 [Page 37]
INTERNET-DRAFT Packet Marking December 1997
[RFC795] J. Postel, "Service Mappings", Internet RFC 795, September
1981.
[RFC1046] W. Prue and J. Postel, "A Queuing Algorithm to Provide
Type-of-Service for IP Links", Internet RFC 1046, February
1988.
[RFC1349] P. Almquist, "Type of Service in the Internet Protocol
Suite", Internet RFC 1349, July 1992.
[RFC1583] J. Moy, "OSPF Version 2", Internet RFC 1583, March 1994.
[RFC1633] R. Braden, D. Clark, and S. Shenker, "Integrated Services
in the Internet Architecture: An Overview", Internet RFC
1633, July 1994.
[RFC1812] F. Baker, editor, "Requirements for IP Version 4 Routers",
Internet RFC 1812, June 1995.
[RFC1883] S. Deering and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", Internet RFC 1883, December 1995.
[RSVP] B. Braden et. al., "Resource ReSerVation Protocol (RSVP)
-- Version 1 Functional Specification", Internet RFC 2205,
September 1997.
[Shenker] S. Shenker, "Fundamental Design Issues for the Future
Internet", IEEE/ACM Trans. on Networking, vol. 13, no. 7,
Sep. 1995.
[SIMA] K. Kilkki, "Simple Integrated Media Access (SIMA)",
Internet Draft <draft-kalevi-simple-media-access-01.txt>,
June 1997.
[TWOBIT] K. Nichols, V. Jacobson, and L. Zhang, "A Two-bit
Differentiated Services Architecture for the Internet",
Internet Draft <draft-nichols-diff-svc-arch-00.txt>,
November 1997.
Blake Expires: June 1998 [Page 38]
INTERNET-DRAFT Packet Marking December 1997
Author's Address
Steven Blake
E95/664
IBM Corporation
800 Park Offices Drive
Research Triangle Park, NC 27709
Phone: +1-919-254-2030
Fax: +1-919-254-5483
E-mail: slblake@raleigh.ibm.com
Blake Expires: June 1998 [Page 39]