Network Working Group                          Jerry Perser
   INTERNET-DRAFT                                 Spirent
   Expires in: August 2001                          David Newman
                                                  Network Test
                                                  Sumit Khurana
                                                  Telcordia
                                                  Shobha Erramilli
                                                  Telcordia
                                                  Scott Poretsky
                                                  Avici
                                                    February 2001


                Terminology for Benchmarking Network-layer
                         Traffic Control Mechanisms

                       <draft-ietf-bmwg-dsmterm-00.txt>


   Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force  (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other documents
   at any time.  It is inappropriate to use Internet- Drafts as
   reference material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   Table of Contents

     1. Introduction  ............................................... 2
     2. Existing definitions  ....................................... 3
     3. Term definitions .............................................3
       3.1 Channel Capacity ..........................................3
       3.2 Classification ............................................4
       3.3 Code Point Forwarding Rate ................................4
       3.4 Conforming ................................................5
       3.5 Congestion ................................................5
       3.6 Congestion Management .....................................6
       3.7 Delay .....................................................6
       3.8 Expected Output Vector ....................................7

Perser, Newman, & Poretsky                                    [Page 1]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

       3.9 Flow ......................................................8
       3.10 Jitter ...................................................8
       3.11 Nonconforming ............................................9
       3.12 Offered Load Vector .....................................10
       3.13 Policy ..................................................10
       3.14 Provision ...............................................11
       3.15 Sequence number .........................................11
       3.16 Shaping .................................................12
       3.17 Stream ..................................................12
       3.18 Tail dropping ...........................................13
       3.19 Unburdened Response .....................................14
     4. Security Considerations .....................................14
     5. References ..................................................14
     6. Author's Address ............................................15
     7. Full Copyright Statement ....................................16


   1. Introduction

   Driven by Internet economics, service providers and enterprises
   alike have shown strong interest in adding traffic-control
   capabilities to network devices. These capabilities would enable
   network operators to define and deliver minimum or maximum levels of
   bandwidth, delay, and jitter for multiple classes of traffic.
   Perhaps more importantly, network operators would be able to set
   pricing according to the level of service delivered.

   Networking device manufacturers have responded with a bewildering
   array of mechanisms for controlling network traffic. While there are
   numerous _policy management_ and _quality of service_ frameworks,
   many of them rely on one of two network-layer mechanisms for the
   actual control of forwarding rate, delay, and jitter. These two
   mechanisms are the IP precedence setting in the IP header and the
   diff-serve code point (DSCP) defined in [3].

   This document describes the various terms to be used in benchmarking
   devices that implement traffic control based on IP precedence or
   DSCP criteria. This document is narrowly focused, in that it
   describes only terms for measuring behavior of a device or a group
   of devices using one of these two mechanisms. End-to-end and
   service-level measurements are beyond the scope of this document.

   This document introduces several new terms that will allow
   measurements to be taken during periods of congestion. New
   terminology is needed because most existing benchmarking terms
   assume the absence of congestion. For example, throughput is one of
   the most widely used measurements _- yet RFC 1242 defines throughput
   as a rate in the absence of loss. As a result, throughput is not a
   meaningful measurement where congestion exists.

   Another key difference from existing benchmarking terminology is the
   definition of measurements as observed on egress as well as ingress
   of a device/system under test. Again, the existence of congestion

Perser, Newman, Khurana, Erramilli, Poretsky                  [Page 2]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

   requires the addition of egress measurements as well as those taken
   on ingress; without observing traffic leaving a device it is not
   possible to say whether traffic-control mechanisms effectively dealt
   with congestion.

   The principal measurements introduced in this document are
   forwarding rate, latency, and jitter, all of which can be observed
   with or without congestion of the DUT/SUT.


   2.  Existing definitions

   RFC 1242 "Benchmarking Terminology for Network Interconnect Devices"
   and RFC 2285 " Benchmarking Terminology for LAN Switching Devices"
   should be consulted before attempting to make use of this document.

   RFC 2474 " Definition of the Differentiated Services Field (DS
   Field) in the IPv4 and IPv6 Headers" section 2, contains discussions
   of a number of terms relevant to network-layer traffic control
   mechanisms and should also be consulted.

   For the sake of clarity and continuity this RFC adopts the template
   for definitions set out in Section 2 of RFC 1242.  Definitions are
   indexed and grouped together in sections for ease of reference.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC 2119.


   3. Term definitions

   3.1 Channel Capacity

      Definition:
        The maximum forwarding rate of a link or set of aggregated
        links at which none of the offered packets are dropped by the
        DUT/SUT.

      Discussion:
        Channel capacity measures the data rate at the egress
        interface(s) of the DUT/SUT. In contrast, throughput as defined
        in RFC 1242 measures the data rate is based on the ingress
        interface(s) of the DUT/SUT.

        Ingress-based measurements do not account for congestion of the
        DUT/SUT. Channel capacity, as an egress measurement, does take
        congestion into account.

        Understanding channel capacity is a necessary precursor to any
        measurement involving congestion.  Throughput numbers can be
        higher than channel capacity because of queueing.


Perser, Newman, Khurana, Erramilli, Poretsky                  [Page 3]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

      Measurement units:

         N-octet packets per second

      Issues:

      See Also:
        Throughput [1]


   3.2 Classification

      Definition:
        Local mapping of Policies to the IP Precedence or DSCP value in
        each frame.

      Discussion:
        Could be based on the DS field or IP Precedence in the packet
        header.  Could be based on MF criteria such as IP Source and
        destination addresses, protocol and port number.

        Covers reclassification.  A Hop does not know if a previous Hop
        classified the packets.  Since the scope of the document is a
        per-hop-behavior, we avoid end-to-end discussions.

        Classification should cover all cases defined in policy. This
        may include rules mandating packet drops, not just forwarded
        packets.


      Measurement units:

         n/a

      Issues:


      See Also:


   3.3 Code Point Forwarding Rate

      Definition:
        The number of packets per second in for all packets containing
        a single DSCP (or IP precedence) that a device can be observed
        to successfully transmit to the correct destination interface
        in response to a specified offered load.

      Discussion:
        - CP based, not port nor IP stream based.
        - Per-hop measurement.  A DUT may change the DSCP for a
        multiple-hop measurement.


Perser, Newman, Khurana, Erramilli, Poretsky                  [Page 4]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

      Measurement units:

        N-octet packets per second

      Issues:


      See Also:


   3.4 Conforming

      Definition:
        A packet meeting a certain policy.

      Discussion:
        Consider a policy that allows a given application to consume
        only X bit/s of channel capacity and no more.

        In a congestion scenario, some individual packets will be
        conforming and others will not.

        It cannot be said that an entire stream or flow is conforming,
        since an offered load that exceeds policy boundaries will
        necessarily carry some nonconforming traffic.


      Measurement units:

         n/a

      Issues:

      See Also:

   3.5 Congestion

      Definition:
        A condition in which one or more egress interfaces are offered
        more packets than can be forwarded at any given instant.

      Discussion:


        This condition is a superset of the overload definition [2].
        That definition assumes the overload is introduced by strictly
        by the tester on ingress of a DUT/SUT. That may or may not be
        the case here.

        Another difference is that with multiple-DUT measurements,
        congestion may occur at multiple points. For example, multiple
        edge devices collectively may congest a core device. In
        contrast, overload [1] deals only with overload on ingress.

Perser, Newman, Khurana, Erramilli, Poretsky                  [Page 5]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001


        Ingress observations alone are not sufficient to cover all
        cases in which congestion may occur. A device with an infinite
        amount of memory could buffer an infinite amount of packets,
        and eventually forward all of them. However, these packets may
        or may not be forwarded during the test duration. Even though
        ingress interfaces accept all packets without loss, this
        hypothetical device may still be congested.

        The definition presented here explicitly defines congestion as
        an event observable on egress interfaces. Regardless of
        internal architecture, any device that cannot forward packets
        on one or more egress interfaces is congested.

      Measurement units:

         n/a

      Issues:

      See Also:


   3.6 Congestion Management

      Definition:
        An implementation of one or more per-hop-behaviors to avoid or
        minimize the condition of congestion.

      Discussion:
        Congestion management may seek either to control congestion or
        avoid it altogether. Such mechanisms classify packets based
        upon IP Precedence or DSCP settings in a packet's IP header.

        Congestion avoidance mechanisms seek to prevent congestion
        before it actually occurs.

        Congestion control mechanisms gives one or more service classes
        preferential treatment over other classes during periods of
        congestion.

      Measurement units:

         n/a

      Issues:

      See Also:


   3.7 Delay

      Definition:

Perser, Newman, Khurana, Erramilli, Poretsky                  [Page 6]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

        The time interval starting when the last bit of the input frame
        reaches the input port of the DUT/SUT and ending when the last
        bit of the output frame is seen on the output port of the
        DUT/SUT.

      Discussion:
        Delay measurement is measured the same regardless of the type
        of DUT/SUT.  Latency [1] require some knowledge of whether the
        DUT/SUT is a "store and forward" or a "bit forwarding" device.
        The fact that a DUT/SUT's technology has a lower delay than
        another technology should be visible in the measurement.

        The measurement point of the last bit is more like the way an
        IP datagram is processed.  A datagram is not passed up or down
        the stack unless it is complete.  Competition happens at the
        last bit of the datagram.

        Delay can be run at any offered load.  Recommend at or below
        the channel capacity for non-congested delay.  For congested
        delay, run the offered load above the channel capacity.

      Measurement units:

         Seconds.

      Issues:


      See Also:


   3.8 Expected Output Vector

      Definition:
        A vector describing the expected output rate of packets having
        a specific code-point for each code-point in the code-point set
        depending on the offered load vector and device PHB
        configurations of the DUT/SUT.

      Discussion:
        The DUT is configured in a certain way in order that service
        discrimination happens for behavior aggregates when a specific
        traffic mix consisting of multiple behavior aggregates is
        applied. This term attempts to capture the expected behavior
        for which the device is configured given a certain load.

        The actual algorithms or mechanism, that the DUT uses to
        achieve service discrimination, is not important in describing
        the expected output vector.

      Measurement units:
        bits per second
        N-octets per second

Perser, Newman, Khurana, Erramilli, Poretsky                  [Page 7]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001


      See Also:
        Offered Load Vector


   3.9 Flow

      Definition:
        A flow is a one or more of packets sharing a common intended
        pair of source and destination interfaces.

      Discussion:
        Packets are grouped by which interface they ingress and egress
        the DUT/SUT.

        A flow can contain multiple source IP addresses and/or
        destination IP addresses.  All packets in a flow must enter on
        the same ingress interface and exit on the same egress
        interface, and have some common network layer content.

        Microflows [3] are a subset of flows.  As defined in [3],
        microflows require application-to-application measurement. In
        contrast, flows use lower-layer classification criteria. Since
        this document focuses on network-layer classification criteria,
        we concentrate here on the use of network-layer identifiers in
        describing a flow. Flow identifiers also may reside at the
        data-link, transport, or application layers of the ISO model.
        However, identifiers other than those at the network layer are
        out of scope for this document.

        A flow may contain a single code point/IP precedence or may
        contain multiple values destined for a single egress interface.
        This is determined by the test methodology.


      Measurement units:

         n/a

      Issues:


      See Also:
        Microflow [3]
        Streams

   3.10 Jitter

      Definition:
        Variation in delay.

      Discussion:


Perser, Newman, Khurana, Erramilli, Poretsky                  [Page 8]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

        Jitter as the absolute value of the difference between the
        delay measurement of two adjacent packets |D(i) - D(i+1)|.  The
        jitter between two packets is reported as the "instantaneous
        jitter".  Packets lost are not counted in the jitter
        measurement.

        Average Jitter is the average of the instantaneous jitter
        measured during the test duration.

        Peak-to-peak jitter is the maximum delay minus the minimum
        delay of the packets forwarded by the DUT/SUT.


      Measurement units:

        Seconds (instantaneous)
        Seconds P-P (peak to peak)
        Seconds avg (average)

      Issues:


      See Also:



   3.11 Nonconforming

      Definition:
        A packet in violation of a given policy.

      Discussion:
        A packet can be said to be nonconforming when it violates the
        terms of a given policy. For example, a policy may set an upper
        bound on bandwidth for a given class of traffic. When a DUT/SUT
        is offered packets of that class in excess of the terms allowed
        by the policy, the DUT/SUT must discard some packets of that
        class. The discarded packets are nonconforming to policy.

        Note that the decision to drop or forward packets is
        essentially time-based.  Even though two packets may share a
        common DSCP or IP precedence value, one may be forwarded and
        one may be dropped depending on when the packets are offered to
        the DUT. It is the combination of classification and policy
        that determines whether packets are conforming or
        nonconforming.

      Measurement units:

         n/a

      Issues:


Perser, Newman, Khurana, Erramilli, Poretsky                  [Page 9]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

      See Also:


   3.12 Offered Load Vector

      Definition:
        A vector describing the rate at which packets having a specific
        code-point are offered to the DUT/SUT, for each code-point in a
        code-point set.

      Discussion:
        Offered loads across the different code-point classes,
        constituting a code-point set, determine the metrics associated
        with a specific code-point traffic class.

      Measurement Units:

        bits per second
        N-octets per second

      Issues:
        Packet size.
        Traffic descriptors such as peak rate, average rate etc.

      See Also:
        Expected output vector


   3.13 Policy

      Definition:
        A rule or set of rules used to classify packets.

      Discussion:
        In the context of data networking, policies consist of one or
        (usually) more rules to administer, manage, and control access
        to network resources.  These resources include applications,
        devices, bandwidth, and any other elements attached to a common
        network.

        In the context of network devices, PHBs are the mechanisms by
        which a policy may be enforced. Packets offered to a DUT/SUT
        may be conforming or nonconforming to a given policy. A PHB may
        cause some packets to be dropped or forwarded based on the
        rules laid down by a given policy.

      Measurement units:

         n/a

      Issues:

      See Also:

Perser, Newman, Khurana, Erramilli, Poretsky                 [Page 10]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

        Conforming
        Nonconforming
        PHB

   3.14 Provision

      Definition:
        Configuration of the DUT/SUT to match a specified rule or rules
        in the policy.

      Discussion:
        A DUT/SUT must be configured in accordance with the rules of a
        policy for that policy's rules to be enforced. Provision may be
        done on a dynamic or a static basis.

      Measurement units:

         n/a

      Issues:

      See Also:
        Conforming
        Nonconforming
        PHB
        Policy


   3.15 Sequence number

      Definition:
        A field in the IP payload portion of the packet that is used to
        verify the order of the packets on the egress of the DUT/SUT.

      Discussion:
        The traffic generator sets the sequence number value and the
        traffic receiver checks the value upon receipt of the packet.
        The traffic generator changes the value on each packet
        transmitted based on an algorithm agreed to by the traffic
        receiver.

        The traffic receiver keeps track of the sequence numbers on a
        per stream basis.  In addition to number of received packets,
        the traffic receiver may also report number of in-sequence
        packets, number of out-sequence packets and number of duplicate
        packets.

        The recommended algorithm to use to change the sequence number
        on sequential packets is an incrementing value.


      Measurement units:


Perser, Newman, Khurana, Erramilli, Poretsky                 [Page 11]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

         n/a

      Issues:

      See Also:
        Stream

   3.16 Shaping

      Definition:
        DUT/SUT control of code point forwarding rate so that a flow or
        stream does not exceed or drop below a specified rate set in
        the policy.

      Discussion:
        [3] defines a behavior aggregate in which a DUT/SUT performs a
        specific forwarding treatment on a group of packets. Shaping is
        one such treatment: A device may drop or forward packets within
        a stream or flow depending on whether the packets are in or out
        of the terms of the policy.

      Measurement units:

         n/a

      Issues:

      See Also:
        Conforming
        Nonconforming
        Policy


   3.17 Stream

      Definition:
        A group of packets tracked as a single entity by the traffic
        receiver.  A stream shares a common content such as type (IP,
        UDP), frame size, or payload.

      Discussion:
        Streams are tracked by "sequence number" or "unique signature
        field" (RFC 2889).  Streams define how individual packet's
        statistics are grouped together to form an intelligible
        summary.

        Common stream groupings would be by egress interface,
        destination address, source address, DCSP, or IP precedence.  A
        stream using sequence numbers can track the ordering of packets
        as they transverse the DUT/SUT.

        Streams are not restricted to a pair of source and destination
        interfaces as long as all packets are tracked as a single

Perser, Newman, Khurana, Erramilli, Poretsky                 [Page 12]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

        entity.  A mulitcast stream can be forward to multiple
        destination interfaces.

      Measurement units:

         n/a

      Issues:

      See Also:
        Flow
        MicroFlow [3]
        Sequence number


   3.18 Tail dropping

      Definition:
        The condition in which a DUT/SUT discards newly arriving
        packets.

      Discussion:
        Every DUT/SUT has a finite amount of traffic it can forward,
        beyond which congestion occurs. Once the offered load crosses
        the congestion threshold, the device may discard any additional
        traffic that arrives until congestion clears.

        Tail dropping is typically a function of offered load exceeding
        a DUT/SUT's buffer capacity, but other factors internal to the
        DUT/SUT may also come into play. In terms of what is externally
        observable, tail dropping can be said to occur only when
        offered load exceeds channel capacity. Since a DUT/SUT may
        buffer traffic on ingress, the actual threshold for tail
        dropping may be higher than channel capacity.

      Measurement units:

         n/a

      Issues:
   Some congestion management mechanisms seek to avoid tail dropping by
   discarding packets before offered load exceeds channel capacity. In
   the presence of such mechanisms, neither congestion nor tail
   dropping should occur.

      See Also:
        Channel capacity
        Congestion






Perser, Newman, Khurana, Erramilli, Poretsky                 [Page 13]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

   3.19 Unburdened Response

      Definition:
        A performance measure obtained when mechanisms used to support
        DiffServ are disabled.

      Discussion:
        Enabling Diffserv mechanisms such as scheduling algorithms may
        impose an additional processing overhead for packets, which may
        cause the aggregate response to suffer even when traffic
        belonging to only one class, the best effort class, is offered
        to the device. Comparisons with "unburdened performance" may
        thus be in order when obtaining metrics to ensure that enabling
        Diffserv mechanisms doesn't impose an excessive performance
        penalty.

      Measurement units:
        TBD.



   4. Security Considerations

        Documents of this type do not directly effect the security of
        the Internet or of corporate networks as long as benchmarking
        is not performed on devices or systems connected to operating
        networks.

   5. References

      [1]   Bradner, S., Editor, "Benchmarking Terminology for Network
            Interconnection Devices", RFC 1242, July 1991.

      [2]   Mandeville, R., "Benchmarking Terminology for LAN Switching
            Devices", RFC 2285, February 1998.

      [3]   K. Nichols, S. Blake, F. Baker, D. Black,"Definition of the
            Differentiated Services Field (DS Field) in the IPv4 and
            IPv6 Headers", RFC 2474, December 1998.

      [4]   S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, W.
            Weiss, "An Architecture for Differentiated Services", RFC
            2475, December 1998.











Perser, Newman, Khurana, Erramilli, Poretsky                 [Page 14]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001

   6. Author's Address

        Jerry Perser
        Spirent Communications
        26750 Agoura Road
        Calabasas, CA 91302
        USA

        Phone: + 1 818 676 2300
        EMail: jerry.perser@spirentcom.com


        David Newman
        Network Test
        321 Newark St., Suite 301
        Hoboken, NJ 07030
        USA

        Phone: + 1 201 798 9999
        EMail: dnewman@networktest.com


        Sumit Khurana
        Telcordia Technologies
        445 South Street
        Morristown, NJ 07960
        USA

        Phone: + 1 973 829 3170
        EMail: sumit@research.telcordia.com



        Shobha Erramilli
        Telcordia Technologies
        331 Newman Springs Road,
        Navesink, NJ 07701
        USA

        Phone: + 1 732 758 5508
        EMail: shobha@research.telcordia.com


        Scott Poretsky
        Avici Systems
        101 Billerica Ave_Building #6
        N. Billerica, MA 01862
        USA

        Phone: + 1 978 964 2287
        EMail: sporetsky@avici.com



Perser, Newman, Khurana, Erramilli, Poretsky                 [Page 15]


INTERNET-DRAFT    Differentiated Services Termology          Feb. 2001



   7.  Full Copyright Statement

        Copyright (C) The Internet Society (1998).  All Rights
        Reserved.

        This document and translations of it may be copied and
        furnished to others, and derivative works that comment on or
        otherwise explain it or assist in its implementation may be
        prepared, copied, published and distributed, in whole or in
        part, without restriction of any kind, provided that the above
        copyright notice and this paragraph are included on all such
        copies and derivative works.  However, this document itself may
        not be modified in any way, such as by removing the copyright
        notice or references to the Internet Society or other Internet
        organizations, except as needed for the purpose of developing
        Internet standards in which case the procedures for copyrights
        defined in the Internet Standards process must be followed, or
        as required to translate it into languages other than English.

        The limited permissions granted above are perpetual and will
        not be revoked by the Internet Society or its successors or
        assigns.  This document and the information contained herein is
        provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE
        INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY
        THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY
        RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
        FOR A PARTICULAR PURPOSE.
























Perser, Newman, Khurana, Erramilli, Poretsky                 [Page 16]