Metrics and Methods for IP Capacity
draft-morton-ippm-capacity-metric-method-01

Document Type Active Internet-Draft (ippm WG)
Last updated 2019-12-09 (latest revision 2019-11-04)
Replaces draft-morton-ippm-capcity-metric-method
Stream IETF
Intended RFC status (None)
Formats plain text xml pdf htmlized bibtex
Stream WG state Adopted by a WG
Document shepherd No shepherd assigned
IESG IESG state I-D Exists
Consensus Boilerplate Unknown
Telechat date
Responsible AD (None)
Send notices to (None)
Network Working Group                                          A. Morton
Internet-Draft                                                 AT&T Labs
Updates: ???? (if approved)                                      R. Geib
Intended status: Standards Track                        Deutsche Telekom
Expires: May 7, 2020                                       L. Ciavattone
                                                               AT&T Labs
                                                        November 4, 2019

                  Metrics and Methods for IP Capacity
              draft-morton-ippm-capacity-metric-method-01

Abstract

   This memo revisits the problem of Network Capacity metrics first
   examined in RFC 5136.  The memo specifies a more practical Maximum
   IP-layer Capacity metric definition catering for measurement
   purposes, and outlines the corresponding methods of measurement.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14[RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on May 7, 2020.

Morton, et al.             Expires May 7, 2020                  [Page 1]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Scope and Goals . . . . . . . . . . . . . . . . . . . . . . .   4
   3.  Motivation  . . . . . . . . . . . . . . . . . . . . . . . . .   4
   4.  General Parameters and Definitions  . . . . . . . . . . . . .   5
   5.  IP-Layer Capacity Singleton Metric Definitions  . . . . . . .   6
     5.1.  Formal Name . . . . . . . . . . . . . . . . . . . . . . .   6
     5.2.  Parameters  . . . . . . . . . . . . . . . . . . . . . . .   6
     5.3.  Metric Definitions  . . . . . . . . . . . . . . . . . . .   6
     5.4.  Related Round-Trip Delay and Loss Definitions . . . . . .   8
     5.5.  Discussion  . . . . . . . . . . . . . . . . . . . . . . .   8
     5.6.  Reporting the Metric  . . . . . . . . . . . . . . . . . .   8
   6.  Maximum IP-Layer Capacity Metric Definitions (Statistic)  . .   8
     6.1.  Formal Name . . . . . . . . . . . . . . . . . . . . . . .   8
     6.2.  Parameters  . . . . . . . . . . . . . . . . . . . . . . .   8
     6.3.  Metric Definitions  . . . . . . . . . . . . . . . . . . .   9
     6.4.  Related Round-Trip Delay and Loss Definitions . . . . . .  10
     6.5.  Discussion  . . . . . . . . . . . . . . . . . . . . . . .  10
     6.6.  Reporting the Metric  . . . . . . . . . . . . . . . . . .  11
   7.  IP-Layer Sender Bit Rate Singleton Metric Definitions . . . .  11
     7.1.  Formal Name . . . . . . . . . . . . . . . . . . . . . . .  11
     7.2.  Parameters  . . . . . . . . . . . . . . . . . . . . . . .  11
     7.3.  Metric Definition . . . . . . . . . . . . . . . . . . . .  12
     7.4.  Discussion  . . . . . . . . . . . . . . . . . . . . . . .  12
     7.5.  Reporting the Metric  . . . . . . . . . . . . . . . . . .  12
   8.  Method of Measurement . . . . . . . . . . . . . . . . . . . .  12
     8.1.  Load Rate Adjustment Algorithm (from udpst) . . . . . . .  13
     8.2.  Measurement Qualification or Verification . . . . . . . .  14
     8.3.  Measurement Considerations  . . . . . . . . . . . . . . .  15
   9.  Reporting . . . . . . . . . . . . . . . . . . . . . . . . . .  16
   10. Security Considerations . . . . . . . . . . . . . . . . . . .  17
   11. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  17

Morton, et al.             Expires May 7, 2020                  [Page 2]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   12. Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  17
   13. References  . . . . . . . . . . . . . . . . . . . . . . . . .  17
     13.1.  Normative References . . . . . . . . . . . . . . . . . .  17
     13.2.  Informative References . . . . . . . . . . . . . . . . .  19
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  20

1.  Introduction

   The IETF's efforts to define Network and Bulk Transport Capacity have
   been chartered and progressed for over twenty years.  Over that time,
   the performance community has seen development of Informative
   definitions in [RFC3148] for Framework for Bulk Transport Capacity
   (BTC), RFC 5136 for Network Capacity and Maximum IP-layer Capacity,
   and the Experimental metric definitions and methods in [RFC8337],
   Model-Based Metrics for BTC.

   This memo revisits the problem of Network Capacity metrics examined
   first in [RFC3148] and later in [RFC5136].  Maximum IP-Layer Capacity
   and [RFC3148] Bulk Transfer Capacity (goodput) are different metrics.
   Max IP-layer Capacity is like the theoretical goal for goodput.
   There are many metrics in [RFC5136], such as Available Capacity.
   Measurements depend on the network path under test and the use case.
   Here, the main use case is to assess the maximum capacity of the
   access network, with specific performance criteria used in the
   measurement.

   This memo recognizes the importance of a definition of a Maximum IP-
   layer Capacity Metric at a time when access speeds have increased
   dramatically; a definition that is both practical and effective for
   the performance community's needs, including Internet users.  The
   metric definition is intended to use Active Methods of Measurement
   [RFC7799], and a method of measurement is included.

   The most direct active measurement of IP-layer Capacity would use IP
   packets, but in practice a transport header is needed to traverse
   address and port translators.  UDP offers the most direct assessment
   possibility, and in the [copycat][copycat] measurement study to
   investigate whether UDP is viable as a general Internet transport
   protocol, the authors found that a high percentage of paths tested
   support UDP transport.  A number of liaisons have been exchanged on
   this topic [ refs to ITU-T SG12, ETSI STQ, BBF liaisons ], discussing
   the laboratory and field tests that support the UDP-based approach to
   IP-layer Capacity measurement.

   This memo also recognizes the many updates to the IP Performance
   Metrics Framework [RFC2330] published over twenty years, and makes
   use of [RFC7312] for Advanced Stream and Sampling Framework, and RFC
   8468 [RFC8468]IPv4, IPv6, and IPv4-IPv6 Coexistence Updates.

Morton, et al.             Expires May 7, 2020                  [Page 3]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   NOTE: The text contains a few Author comments, in brackets [RG: ,
   acm: ]

2.  Scope and Goals

   The scope of this memo is to define a metric and corresponding method
   to unambiguously perform Active measurements of Maximum IP-Layer
   Capacity, along with related metrics and methods.

   The main goal is to harmonize the specified metric and method across
   the industry, and this memo is the vehicle through which working
   group (and eventually, IETF) consensus will be captured and
   communicated to achieve broad agreement, and possibly result in
   changes in the specifications of other Standards Development
   Organizations (SDO) (through the SDO's normal contribution process,
   or through liaison exchange).

   A local goal is to aid efficient test procedures where possible, and
   to recommend reporting with additional interpretation of the results.
   Also, to foster the development of protocol support for this metric
   and method of measurement (all active testing protocols currently
   defined by the IPPM WG are UDP-based, meeting a key requirement of
   these methods).

3.  Motivation

   As with any problem that has been worked for many years in various
   SDOs without any special attempts at coordination, various solutions
   for metrics and methods have emerged.

   There are five factors that have changed (or begun to change) in the
   2013-2019 time frame, and the presence of any one of them on the path
   requires features in the measurement design to account for the
   changes:

   1.  Internet access is no longer the bottleneck for many users.

   2.  Both speed and latency are important to user's satisfaction.

   3.  UDP's growing role in Transport, in areas where TCP once
       dominated.

   4.  Content and applications moving physically closer to users.

   5.  Less emphasis on ISP gateway measurements, possibly due to less
       traffic crossing ISP gateways in future.

Morton, et al.             Expires May 7, 2020                  [Page 4]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

4.  General Parameters and Definitions

   This section lists the REQUIRED input factors to specify a Sender or
   Receiver metric.

   o  Src, the address of a host (such as the globally routable IP
      address).

   o  Dst, the address of a host (such as the globally routable IP
      address).

   o  i, the limit on the number of Hops a specific packet may visit as
      it traverses from the host at Src to the host at Dst (such as the
      TTL or Hop Limit).

   o  MaxHops, the maximum value of i used, (i=1,2,3,...MaxHops).

   o  T0, the time at the start of measurement interval, when packets
      are first transmitted from the Source.

   o  I, the duration of a measurement interval (default 10 sec)

   o  dt, the duration of N equal sub-intervals in I (default 1 sec)

   o  Tmax, a maximum waiting time for test packets to arrive at the
      destination, set sufficiently long to disambiguate packets with
      long delays from packets that are discarded (lost), such that the
      distribution of one-way delay is not truncated.

   o  F, the number of different flows synthesized by the method
      (default 1 flow)

   o  flow, the stream of packets with the same n-tuple of designated
      header fields that (when held constant) result in identical
      treatment in a multi-path decision (such as the decision taken in
      load balancing).  Note: The IPv6 flow label MAY be included in the
      flow definition when routers have complied with [RFC6438]
      guidelines at the Tunnel End Points (TEP), and the source of the
      measurement is a TEP.

   o  Type-P, the complete description of the packets for which this
      assessment applies (including the flow-defining fields).  Note
      that the UDP transport layer is one requirement specified below.
      Type-P is a parallel concept to "population of interest" defined
      in ITU-T Rec. Y.1540.

   o  PM, a list of fundamental metrics, such as loss, delay, and
      reordering, and corresponding Target performance threshold.  At

Morton, et al.             Expires May 7, 2020                  [Page 5]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

      least one fundamental metric and Target performance threshold MUST
      be supplied (such as One-way IP Packet Loss [RFC7680] equal to
      zero).

   A non-Parameter which is required for several metrics is defined
   below:

   o  T, the host time of the *first* test packet's *arrival* as
      measured at MP(Dst).  There may be other packets sent between
      source and destination hosts that are excluded, so this is the
      time of arrival of the first packet used for measurement of the
      metric.

   Note that time stamps, sequnce numbers, etc. will be established by
   the test protocol.

5.  IP-Layer Capacity Singleton Metric Definitions

   This section sets requirements for the following components to
   support the Maximum IP-layer Capacity Metric.

5.1.  Formal Name

   Type-P-IP-Capacity, or informally called IP-layer Capacity.

   Note that Type-P depends on the chosen method.

5.2.  Parameters

   This section lists the REQUIRED input factors to specify the metric,
   beyond those listed in Section 4.

   No additional Parameters are needed.

5.3.  Metric Definitions

   This section defines the REQUIRED aspects of the measureable IP-layer
   Capacity metric (unless otherwise indicated) for measurements between
   specified Source and Destination hosts:

   Define the IP-layer capacity, C(T,I,PM), to be the number of IP-layer
   bits (including header and data fields) in packets that can be
   transmitted from the Src host and correctly received by the Dst host
   during one contiguous sub-interval, dt.

   The number of these IP-layer bits is designated n0[dtn-1,dtn] for a
   specific dtn.

Morton, et al.             Expires May 7, 2020                  [Page 6]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   When the packet size is known and of fixed size, the packet count
   during a single sub-interval dt multiplied by the total bits in IP
   header and data fields is equal to n0[dtn-1,dtn].

   Anticipating a Sample of Singletons, the interval dt SHOULD be set to
   a natural number m so that T+I = T + m*dt with dtn - dtn-1 = dt and
   with 0 < n <= m.

   Parameter PM represents other performance metrics [see section
   Related Round-Trip Delay and Loss Definitions below]; their
   measurement results SHALL be collected during measurement of IP-layer
   Capacity and associated with the corresponding dtn for further
   evaluation and reporting.

   Mathematically, this definition can be represented as:

                                       ( n0[dtn-1,dtn] )
                       C(T,I,PM) = -------------------------
                                              dt

                      Equation for IP-Layer Capacity

   and:

   o  n0 is the total number of IP-layer header and payload bits that
      can be transmitted in Standard Formed packets from the Src host
      and correctly received by the Dst host during one contiguous sub-
      interval, dt in length, during the interval [T, T+I],

   o  C(T,I,PM) the IP-Layer Capacity, corresponds to the value of n0
      measured in any sub-interval ending at dtn (meaning T + n*dt),
      divided by the length of sub-interval, dt.

   o  all sub-intervals SHOULD be of equal duration.  Choosing dt as
      non-overlapping consecutive time intervals allows for a simple
      implementation.

   o  The bit rate of the physical interface of the measurement device
      must be higher than that of the link whose C(T,I,PM) is to be
      measured.

   Measurements according to these definitions SHALL use UDP transport
   layer.

Morton, et al.             Expires May 7, 2020                  [Page 7]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

5.4.  Related Round-Trip Delay and Loss Definitions

   RTD[dtn-1,dtn] is defined as a sample of the [RFC2681] Round-trip
   Delay between the Src host and the Dst host over the interval
   [T,T+I].  The statistics used to to summarize RTD[dtn-1,dtn] MAY
   include the minimum, maximum, and mean.

   RTL[dtn-1,dtn] is defined as a sample of the [RFC6673] Round-trip
   Loss between the Src host and the Dst host over the interval [T,T+I].
   The statistics used to to summarize RTL[dtn-1,dtn] MAY include the
   lost packet count and the lost packet ratio.

5.5.  Discussion

   See the corresponding section for Maximum IP-Layer Capacity.

5.6.  Reporting the Metric

   The IP-Layer Capacity SHALL be reported with meaningful resolution,
   in units of Megabits per second (Mbps).

   The Related Round Trip Delay and/or Loss metric measurements for the
   same Singleton SHALL be reported, also with meaningful resolution for
   the values measured.

   Individual Capacity measurements MAY be reported in a manner
   consistent with the Maximum IP-Layer Capacity, see Section 9.

6.  Maximum IP-Layer Capacity Metric Definitions (Statistic)

   This section sets requirements for the following components to
   support the Maximum IP-layer Capacity Metric.

6.1.  Formal Name

   Type-P-Max-IP-Capacity, or informally called Maximum IP-layer
   Capacity.

   Note that Type-P depends on the chosen method.

6.2.  Parameters

   This section lists the REQUIRED input factors to specify the metric,
   beyond those listed in Section 4.

   No additional Parameters or definitions are needed.

Morton, et al.             Expires May 7, 2020                  [Page 8]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

6.3.  Metric Definitions

   This section defines the REQUIRED aspects of the Maximum IP-layer
   Capacity metric (unless otherwise indicated) for measurements between
   specified Source and Destination hosts:

   Define the Maximum IP-layer capacity, Maximum_C(T,I,PM), to be the
   maximum number of IP-layer bits n0[dtn-1,dtn] that can be transmitted
   in packets from the Src host and correctly received by the Dst host,
   over all dt length intervals in [T, T+I], and meeting the PM
   criteria.  Equivalently the Maximum of a Sample of size m of
   C(T,I,PM) collected during the interval [T, T+I] and meeting the PM
   criteria.

   The interval dt SHOULD be set to a natural number m so that T+I = T +
   m*dt with dtn - dtn-1 = dt and with 0 < n <= m.

   Parameter PM represents the other performance metrics [see section
   Related Round-Trip Delay and Loss Definitions below] and their
   measurement results for the maximum IP-layer capacity.  At least one
   target performance threshold (PM criterion) MUST be defined.  If more
   than one target performance threshold is defined, then the sub-
   interval with maximum number of bits transmitted MUST meet all the
   target performance thresholds.

   Mathematically, this definition can be represented as:

                                     max  ( n0[dtn-1,dtn] )
                                    [T,T+I]
               Maximum_C(T,I,PM) = -------------------------
                                              dt
              where:
                 T                                      T+I
                 _________________________________________
                 |   |   |   |   |   |   |   |   |   |   |
             dtn=0   1   2   3   4   5   6   7   8   9   m=10

                       Equation for Maximum Capacity

   and:

   o  n0 is the total number of IP-layer header and payload bits that
      can be transmitted in Standard Formed packets from the Src host
      and correctly received by the Dst host during one contiguous sub-
      interval, dt in length, during the interval [T, T+I],

Morton, et al.             Expires May 7, 2020                  [Page 9]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   o  Maximum _C(T,I,PM) the Maximum IP-Layer Capacity, corresponds to
      the maximum value of n0 measured in any sub-interval ending at dtn
      (meaning T + n*dt), divided by the constant length of all sub-
      intervals, dt.

   o  all sub-intervals SHOULD be of equal duration.  Choosing dt as
      non-overlapping consecutive time intervals allows for a simple
      implementation.

   o  The bit rate of the physical interface of the measurement systems
      must be higher than that of the link whose Maximum _C(T,I,PM) is
      to be measured (the bottleneck link).

   In this definition, the m sub-intervals can be viewed as trials when
   the Src host varies the transmitted packet rate, searching for the
   maximum n0 that meets the PM criteria measured at the Dst host in a
   test of duration, I.  When the transmitted packet rate is held
   constant at the Src host, the m sub-intervals may also be viewed as
   trials to evaluate the stability of n0 and metric(s) in the PM list
   over all dt-length intervals in I.

   Measurements according to these definitions SHALL use UDP transport
   layer.

6.4.  Related Round-Trip Delay and Loss Definitions

   RTD[dtn-1,dtn] is defined as a sample of the [RFC2681] Round-trip
   Delay between the Src host and the Dst host over the interval
   [T,T+I], and corresponds to the dt interval containing
   Maximum_C(T,I,PM).  The statistics used to to summarize RTD[dtn-
   1,dtn] MAY include the minimum, maximum, and mean.

   RTL[dtn-1,dtn] is defined as a sample of the [RFC6673] Round-trip
   Loss between the Src host and the Dst host over the interval [T,T+I]
   and corresponds to the dt interval containing Maximum_C(T,I,PM).  The
   statistics used to to summarize RTL[dtn-1,dtn] MAY include the lost
   packet count and the lost packet ratio.

6.5.  Discussion

   If traffic conditioning applies along a path for which Maximum
   _C(T,I,PM) is to be determined, different values for dt SHOULD be
   picked and measurements be executed during multiple intervals [T,
   T+I].  Any single interval dt SHOULD be chosen so that is an integer
   multiple of increasing values k times serialisation delay of a path
   MTU at the physical interface speed where traffic conditioning is
   expected.  This should avoid taking configured burst tolerance
   singletons as a valid Maximum _C(T,I,PM) result.

Morton, et al.             Expires May 7, 2020                 [Page 10]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   A Maximum_C(T,I,PM) without any indication of bottleneck congestion,
   be that an increasing latency, packet loss or ECN marks during a
   measurement interval I, is likely to underestimate Maximum_C(T,I,PM).

6.6.  Reporting the Metric

   The Maximum IP-Layer Capacity SHALL be reported with meaningful
   resolution, in units of Megabits per second.

   The Related Round Trip Delay and/or Loss metric measurements for the
   same Singleton SHALL be reported, also with meaningful resolution for
   the values measured.

   When there are demonstrated and repeatable modes in the Sample, then
   the Maximum IP-Layer Capacity SHALL be reported for each mode, along
   with the relative time from the beginning of the stream that the mode
   was observed to be present.  Bimodal Maxima have been observed with
   some services, sometimes called a "turbo" mode" intending to deliver
   short transfers more quickly, or reduce the initial buffering time
   for some video streams.

7.  IP-Layer Sender Bit Rate Singleton Metric Definitions

   This section sets requirements for the following components to
   support the IP-layer Sender Bitrate Metric.

7.1.  Formal Name

   Type-P-IP-Sender-Bit-Rate, or informally called IP-layer Sender
   Bitrate.

   Note that Type-P depends on the chosen method.

7.2.  Parameters

   This section lists the REQUIRED input factors to specify the metric,
   beyond those listed in Section 4.

   o  S, the duration of the measurement interval at the Source

   o  st, the nominal duration of N sub-intervals in S (default = 0.05
      seconds)

   S SHALL be longer than I, primarily to account for on-demand
   activation of the path, or any preamble to testing required.

Morton, et al.             Expires May 7, 2020                 [Page 11]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   st SHOULD be much smaller than the sub-interval dt.  The st parameter
   does not have relevance when the Source is transmitting at a fixed
   rate throughout S.

7.3.  Metric Definition

   This section defines the REQUIRED aspects of the IP-layer Sender
   Bitrate metric (unless otherwise indicated) for measurements at the
   specified Source on packets addressed for the intended Destination
   host and matching the required Type-P:

   Define the IP-layer Sender Bit Rate, B(S,st), to be the number of IP-
   layer bits (including header and data fields) that are transmitted
   from the Source during one contiguous sub-interval, st, during the
   test interval S (where S SHALL be longer than I), and where the
   fixed-size packet count during that single sub-interval st also
   provides the number of IP-layer bits in any interval: n0[stn-1,stn].

   Measurements according to these definitions SHALL use UDP transport
   layer.  Any feedback from Dst host to Src host received by Src host
   during an interval [stn-1,stn] MUST NOT result in an adaptation of
   the Src host traffic conditioning during this interval.

7.4.  Discussion

   Both the Sender and Receiver or (source and destination) bit rates
   SHOULD be assessed as part of a measurement.

7.5.  Reporting the Metric

   The IP-Layer Sender Bit Rate SHALL be reported with meaningful
   resolution, in units of Megabits per second.

   Individual IP-Layer Sender Bit Rate measurements are discussed
   further in Section 9.

8.  Method of Measurement

   The duration of a test, I, MUST be constrained in a production
   network, since this is an active test method and it will likely cause
   congestion on the Src to Dst host path during a test.

   Additional Test methods and configurations may be provided in this
   section, after review and further testing.

Morton, et al.             Expires May 7, 2020                 [Page 12]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

8.1.  Load Rate Adjustment Algorithm (from udpst)

   A table is pre-built defining all the offered load rates that will be
   supported (R1 - Rn, in ascending order).  Each rate is defined as
   datagrams of size S, sent as a burst of count C, every time interval
   T.  While it is advantageous to use datagrams of as large a size as
   possible, it may be prudent to use a slightly smaller maximum that
   allows for secondary protocol headers and/or tunneling without
   resulting in IP-layer fragmentation.

   At the beginning of a test, the sender begins sending at rate R1 and
   the receiver starts a feedback timer at interval F (while awaiting
   inbound datagrams).  As datagrams are received they are checked for
   sequence number anomalies (loss, out-of-order, duplication, etc.) and
   the delay variation is measured (one-way or round-trip).  This
   information is accumulated until the feedback timer F expires and a
   status feedback message is sent from the receiver back to the sender,
   to communicate this information.  The accumulated statistics are then
   reset by the receiver for the next feedback interval.  As feedback
   messages are received back at the sender, they are evaluated to
   determine how to adjust the current offered load rate (Rx).

   If the feedback indicates that there were no sequence number
   anomalies AND the delay variation was below the lower threshold, the
   offered load rate is increased.  If congestion has not been confirmed
   up to this point, the offered load rate is increased by more than one
   rate (e.g., Rx+10).  This allows the offered load to quickly reach a
   near-maximum rate.  Conversely, if congestion has been previously
   confirmed, the offered load rate is only increased by one (Rx+1).

   If the feedback indicates that sequence number anomalies were
   detected OR the delay variation was above the upper threshold, the
   offered load rate is decreased.  If congestion is confirmed by the
   current feedback message being processed, the offered load rate is
   decreased by more than one rate (e.g., Rx-30).  This one-time
   reduction is intended to compensate for the fast initial ramp-up.  In
   all other cases, the offered load rate is only decreased by one (Rx-
   1).

   If the feedback indicates that there were no sequence number
   anomalies AND the delay variation was above the lower threshold, but
   below the upper threshold, the offered load rate is not changed.
   This allows time for recent changes in the offered load rate to
   stabilize, and the feedback to represent current conditions more
   accurately.

Morton, et al.             Expires May 7, 2020                 [Page 13]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   Lastly, the method for confirming congestion is that there were
   sequence number anomalies OR the delay variation was above the upper
   threshold for two consecutive feedback intervals.

8.2.  Measurement Qualification or Verification

   When assessing a Maximum rate as the metric specifies, artificially
   high (optimistic) values might be measured until some buffer on the
   path is filled.  Other causes include bursts of back-to-back packets
   with idle intervals delivered by a path, while the measurement
   interval (dt) is small and aligned with the bursts.  The artificial
   values might result in an un-sustainable Maximum Capacity observed
   when the method of measurement is searching for the Maximum, and that
   would not do.  This situation is different from the bi-modal service
   rates (discussed under Reporting), which are characterized by a
   multi-second duration (much longer than the measured RTT) and
   repeatable behavior.

   There are many ways that the Method of Measurement could handle this
   false-max issue.  The default value for measurement of singletons (dt
   = 1 second) has proven to a be of practical value during tests of
   this method, allows the bimodal service rates to be characterized,
   and it has an obvious alignment with the reporting units (Mbps).

   Another approach comes from Section 24 of RFC 2544[RFC2544] and its
   discussion of Trial duration, where relatively short trials conducted
   as part of the search are followed by longer trials to make the final
   determination.  In the production network, measurements of singletons
   and samples (the terms for trials and tests of Lab Benchmarking) must
   be limited in duration because they may be service-affecting.  But
   there is sufficient value in repeating a sample with a fixed sending
   rate determined by the previous search for the Max IP-layer Capacity,
   to qualify the result in terms of the other performance metrics
   measured at the same time.

   A qualification measurement for the search result is a subsequent
   measurement, sending at a fixed 99.x % of the Max IP-layer Capacity
   for I, or an indefinite period.  The same Max Capacity Metric is
   applied, and the Qualification for the result is a sample without
   packet loss or a growing minimum delay trend in subsequent singletons
   (or each dt of the measurement interval, I).  Samples exhibiting
   losses or increasing queue occupation require a repeated search and/
   or test at reduced fixed sender rate for qualification.

   Here, as with any Active Capacity test, the test duration must be
   kept short. 10 second tests for each direction of transmission are
   common today.  The default measurement interval specified here is I =
   10 seconds).  In combination with a fast search method and user-

Morton, et al.             Expires May 7, 2020                 [Page 14]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   network coordination, the concerns raised in RFC 6815[RFC6815] are
   alleviated.  The method for assessing Max IP Capacity is different
   from classic [RFC2544] methods: they use short term load adjustment
   and are sensitive to loss and delay, like other congestion control
   algorithms used on the Internet every day.

8.3.  Measurement Considerations

   In general, the wide-spread measurements that this memo encourages
   will encounter wide-spread behaviors.  The bimodal IP Capacity
   behavior is a good example.

   The path measured may be state-full based on many factors, and the
   Parameter "Time of day" when a test starts may not be enough enough
   information.  Repeatable testing may require the time from the
   beginning of a measured flow, and how the flow is constructed
   including how much traffic has already been sent on that flow when a
   state-change is observed, because the state-change may be based on
   time or bytes sent or both.

   Many different traffic shapers and on-demand access technology may be
   encountered, as anticipated in [RFC7312], and play a key role in
   measurement results.  Methods MUST be prepared to provide a short
   preamble transmission to activate on-demand access, and to discard
   the preamble from subsequent test results.

   In general, results depend on the sending stream characteristics; the
   measurement community has known this for a long time, and to keep it
   front of mind.  Although the default is a single flow (F=1) for
   testing, use of multiple flows may be advantageous for the following
   reasons:

   1.  the test hosts may be able to create higher load than with a
       single flow, or parallel test hosts may be used to generate 1
       flow each.

   2.  there may be link aggregation present (flow-based load balancing)
       and multiple flows are need to occupy each member of the
       aggregate.

   Each flow would be controlled using its own implementation of the
   Load Adjustment (Search) Algorithm.

   As testing continues, implementers should expect some evolution in
   the methods.

Morton, et al.             Expires May 7, 2020                 [Page 15]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

9.  Reporting

   The Maximum IP-Layer Capacity results SHOULD be reported in the
   format of a table with a row for each of the test Phases and Number
   of Flows.  There SHOULD be columns for the phases with number of
   flows, and for the resultant Maximum IP-Layer Capacity results for
   the aggregate and each flow tested.

   The PM list metrics corresponding to the sub-interval where the
   Maximum Capacity occurred MUST accompany a report of Maximum IP-Layer
   Capacity results, for each test phase.

   +--------------+----------------------+-----------+-----------------+
   | Phase, #     | Max IP-Layer         | Loss      | RTT min, max,   |
   | Flows        | Capacity, Mbps       | Ratio     | msec            |
   +--------------+----------------------+-----------+-----------------+
   | Search,1     | 967.31               | 0.0002    | 30, 58          |
   | Verify,1     | 966.00               | 0.0000    | 30, 38          |
   +--------------+----------------------+-----------+-----------------+

                     Maximum IP-layer Capacity Results

   Static and configuration parameters:

   The sub-interval time, dt, MUST accompany a report of Maximum IP-
   Layer Capacity results, and the remaining Parameters from Section 4,
   General Parameters.

   The IP-Layer Sender Bit rate results SHOULD be reported in the format
   of a table with a row for each of the test Phases, sub-intervals (st)
   and Number of Flows.  There SHOULD be columns for the phases with
   number of flows, and for the resultant IP-Layer Sender Bit rate
   results for the aggregate and each flow tested.

   +------------------------+-------------+-----------------------+----+
   | Phase, Flow or         | st, sec     | Sender Bit Rate, Mbps | ?? |
   | Aggregate              |             |                       |    |
   +------------------------+-------------+-----------------------+----+
   | Search,1               | 0.00 - 0.05 | 345                   | __ |
   | Search,2               | 0.00 - 0.05 | 289                   | __ |
   | Search,Agg             | 0.00 - 0.05 | 634                   | __ |
   +------------------------+-------------+-----------------------+----+

                     IP-layer Sender Bit Rate Results

   Static and configuration parameters:

Morton, et al.             Expires May 7, 2020                 [Page 16]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   The subinterval time, st, MUST accompany a report of Sender IP-Layer
   Bit Rate results.

   Also, the values of the remaining Parameters from Section 4, General
   Parameters, MUST be reported.

10.  Security Considerations

   Active metrics and measurements have a long history of security
   considerations [add references to LMAP Framework, etc.].

   <There are certainly some new ones for Capacity testing>

11.  IANA Considerations

   This memo makes no requests of IANA.

12.  Acknowledgements

   Thanks to Joachim Fabini, Matt Mathis, and Ignacio Alvarez-Hamelin
   for their extensive comments on the memo and related topics.

13.  References

13.1.  Normative References

   [RFC1242]  Bradner, S., "Benchmarking Terminology for Network
              Interconnection Devices", RFC 1242, DOI 10.17487/RFC1242,
              July 1991, <https://www.rfc-editor.org/info/rfc1242>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC2330]  Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
              "Framework for IP Performance Metrics", RFC 2330,
              DOI 10.17487/RFC2330, May 1998,
              <https://www.rfc-editor.org/info/rfc2330>.

   [RFC2544]  Bradner, S. and J. McQuaid, "Benchmarking Methodology for
              Network Interconnect Devices", RFC 2544,
              DOI 10.17487/RFC2544, March 1999,
              <https://www.rfc-editor.org/info/rfc2544>.

   [RFC2681]  Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
              Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681,
              September 1999, <https://www.rfc-editor.org/info/rfc2681>.

Morton, et al.             Expires May 7, 2020                 [Page 17]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   [RFC2889]  Mandeville, R. and J. Perser, "Benchmarking Methodology
              for LAN Switching Devices", RFC 2889,
              DOI 10.17487/RFC2889, August 2000,
              <https://www.rfc-editor.org/info/rfc2889>.

   [RFC3148]  Mathis, M. and M. Allman, "A Framework for Defining
              Empirical Bulk Transfer Capacity Metrics", RFC 3148,
              DOI 10.17487/RFC3148, July 2001,
              <https://www.rfc-editor.org/info/rfc3148>.

   [RFC5136]  Chimento, P. and J. Ishac, "Defining Network Capacity",
              RFC 5136, DOI 10.17487/RFC5136, February 2008,
              <https://www.rfc-editor.org/info/rfc5136>.

   [RFC5180]  Popoviciu, C., Hamza, A., Van de Velde, G., and D.
              Dugatkin, "IPv6 Benchmarking Methodology for Network
              Interconnect Devices", RFC 5180, DOI 10.17487/RFC5180, May
              2008, <https://www.rfc-editor.org/info/rfc5180>.

   [RFC6201]  Asati, R., Pignataro, C., Calabria, F., and C. Olvera,
              "Device Reset Characterization", RFC 6201,
              DOI 10.17487/RFC6201, March 2011,
              <https://www.rfc-editor.org/info/rfc6201>.

   [RFC6412]  Poretsky, S., Imhoff, B., and K. Michielsen, "Terminology
              for Benchmarking Link-State IGP Data-Plane Route
              Convergence", RFC 6412, DOI 10.17487/RFC6412, November
              2011, <https://www.rfc-editor.org/info/rfc6412>.

   [RFC6438]  Carpenter, B. and S. Amante, "Using the IPv6 Flow Label
              for Equal Cost Multipath Routing and Link Aggregation in
              Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011,
              <https://www.rfc-editor.org/info/rfc6438>.

   [RFC6673]  Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673,
              DOI 10.17487/RFC6673, August 2012,
              <https://www.rfc-editor.org/info/rfc6673>.

   [RFC6815]  Bradner, S., Dubray, K., McQuaid, J., and A. Morton,
              "Applicability Statement for RFC 2544: Use on Production
              Networks Considered Harmful", RFC 6815,
              DOI 10.17487/RFC6815, November 2012,
              <https://www.rfc-editor.org/info/rfc6815>.

   [RFC6985]  Morton, A., "IMIX Genome: Specification of Variable Packet
              Sizes for Additional Testing", RFC 6985,
              DOI 10.17487/RFC6985, July 2013,
              <https://www.rfc-editor.org/info/rfc6985>.

Morton, et al.             Expires May 7, 2020                 [Page 18]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   [RFC7312]  Fabini, J. and A. Morton, "Advanced Stream and Sampling
              Framework for IP Performance Metrics (IPPM)", RFC 7312,
              DOI 10.17487/RFC7312, August 2014,
              <https://www.rfc-editor.org/info/rfc7312>.

   [RFC7799]  Morton, A., "Active and Passive Metrics and Methods (with
              Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799,
              May 2016, <https://www.rfc-editor.org/info/rfc7799>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC8337]  Mathis, M. and A. Morton, "Model-Based Metrics for Bulk
              Transport Capacity", RFC 8337, DOI 10.17487/RFC8337, March
              2018, <https://www.rfc-editor.org/info/rfc8337>.

   [RFC8468]  Morton, A., Fabini, J., Elkins, N., Ackermann, M., and V.
              Hegde, "IPv4, IPv6, and IPv4-IPv6 Coexistence: Updates for
              the IP Performance Metrics (IPPM) Framework", RFC 8468,
              DOI 10.17487/RFC8468, November 2018,
              <https://www.rfc-editor.org/info/rfc8468>.

13.2.  Informative References

   [copycat]  Edleine, K., Kuhlewind, K., Trammell, B., and B. Donnet,
              "copycat: Testing Differential Treatment of New Transport
              Protocols in the Wild (ANRW '17)", July 2017,
              <https://irtf.org/anrw/2017/anrw17-final5.pdf>.

   [RFC8239]  Avramov, L. and J. Rapp, "Data Center Benchmarking
              Methodology", RFC 8239, DOI 10.17487/RFC8239, August 2017,
              <https://www.rfc-editor.org/info/rfc8239>.

   [TST009]   Morton, R. A., "ETSI GS NFV-TST 009 V3.1.1 (2018-10),
              "Network Functions Virtualisation (NFV) Release 3;
              Testing; Specification of Networking Benchmarks and
              Measurement Methods for NFVI"", October 2018,
              <https://www.etsi.org/deliver/etsi_gs/NFV-
              TST/001_099/009/03.01.01_60/gs_NFV-TST009v030101p.pdf>.

   [VSPERF-b2b]
              Morton, A., "Back2Back Testing Time Series (from CI)",
              June 2017, <https://wiki.opnfv.org/display/vsperf/
              Traffic+Generator+Testing#TrafficGeneratorTesting-
              AppendixB:Back2BackTestingTimeSeries(fromCI)>.

Morton, et al.             Expires May 7, 2020                 [Page 19]
Internet-Draft         IP Capacity Metrics/Methods         November 2019

   [VSPERF-BSLV]
              Morton, A. and S. Rao, "Evolution of Repeatability in
              Benchmarking: Fraser Plugfest (Summary for IETF BMWG)",
              July 2018,
              <https://datatracker.ietf.org/meeting/102/materials/
              slides-102-bmwg-evolution-of-repeatability-in-
              benchmarking-fraser-plugfest-summary-for-ietf-bmwg-00>.

Authors' Addresses

   Al Morton
   AT&T Labs
   200 Laurel Avenue South
   Middletown,, NJ  07748
   USA

   Phone: +1 732 420 1571
   Fax:   +1 732 368 1192
   Email: acm@research.att.com

   Ruediger Geib
   Deutsche Telekom
   Heinrich Hertz Str. 3-7
   Darmstadt  64295
   Germany

   Phone: +49 6151 5812747
   Email: Ruediger.Geib@telekom.de

   Len Ciavattone
   AT&T Labs
   200 Laurel Avenue South
   Middletown,, NJ  07748
   USA

   Email: lencia@att.com

Morton, et al.             Expires May 7, 2020                 [Page 20]