Metrics for the Evaluation of Congestion Control Mechanisms
RFC 5166

Network Working Group                                      S. Floyd, Ed.
Request for Comments: 5166                                    March 2008
Category: Informational

      Metrics for the Evaluation of Congestion Control Mechanisms

Status of This Memo

   This memo provides information for the Internet community.  It does
   not specify an Internet standard of any kind.  Distribution of this
   memo is unlimited.

IESG Note

   This document is not an IETF Internet Standard.  It represents the
   individual opinion(s) of one or more members of the TMRG Research
   Group of the Internet Research Task Force.  It may be considered for
   standardization by the IETF or adoption as an IRTF Research Group
   consensus document in the future.

Abstract

   This document discusses the metrics to be considered in an evaluation
   of new or modified congestion control mechanisms for the Internet.
   These include metrics for the evaluation of new transport protocols,
   of proposed modifications to TCP, of application-level congestion
   control, and of Active Queue Management (AQM) mechanisms in the
   router.  This document is the first in a series of documents aimed at
   improving the models that we use in the evaluation of transport
   protocols.

   This document is a product of the Transport Modeling Research Group
   (TMRG), and has received detailed feedback from many members of the
   Research Group (RG).  As the document tries to make clear, there is
   not necessarily a consensus within the research community (or the
   IETF community, the vendor community, the operations community, or
   any other community) about the metrics that congestion control
   mechanisms should be designed to optimize, in terms of trade-offs
   between throughput and delay, fairness between competing flows, and
   the like.  However, we believe that there is a clear consensus that
   congestion control mechanisms should be evaluated in terms of trade-
   offs between a range of metrics, rather than in terms of optimizing
   for a single metric.

Floyd                        Informational                      [Page 1]
RFC 5166                     TMRG, METRICS                    March 2008

Table of Contents

   1. Introduction ....................................................2
   2. Metrics .........................................................3
      2.1. Throughput, Delay, and Loss Rates ..........................4
           2.1.1. Throughput ..........................................5
           2.1.2. Delay ...............................................6
           2.1.3. Packet Loss Rates ...................................6
      2.2. Response Times and Minimizing Oscillations .................7
           2.2.1. Response to Changes .................................7
           2.2.2. Minimizing Oscillations .............................8
      2.3. Fairness and Convergence ...................................9
           2.3.1. Metrics for Fairness between Flows .................10
           2.3.2. Metrics for Fairness between Flows with
                  Different Resource Requirements ....................10
           2.3.3. Comments on Fairness ...............................12
      2.4. Robustness for Challenging Environments ...................13
      2.5. Robustness to Failures and to Misbehaving Users ...........14
      2.6. Deployability .............................................14
      2.7. Metrics for Specific Types of Transport ...................15
      2.8. User-Based Metrics ........................................15
   3. Metrics in the IP Performance Metrics (IPPM) Working Group .....15
   4. Comments on Methodology ........................................16
   5. Security Considerations ........................................16
   6. Acknowledgements ...............................................16
   7. Informative References .........................................17

1.  Introduction

   As a step towards improving our methodologies for evaluating
   congestion control mechanisms, in this document we discuss some of
   the metrics to be considered.  We also consider the relationship
   between metrics, e.g., the well-known trade-off between throughput
   and delay.  This document doesn't attempt to specify every metric
   that a study might consider; for example, there are domain-specific
   metrics that researchers might consider that are over and above the
   metrics laid out here.

   We consider metrics for aggregate traffic (taking into account the
   effect of flows on competing traffic in the network) as well as the
   heterogeneous goals of different applications or transport protocols
   (e.g., of high throughput for bulk data transfer, and of low delay
   for interactive voice or video).  Different transport protocols or