[Search] [txt|pdf|bibtex] [Tracker] [WG] [Email] [Nits]

Versions: 00 01 02 03 04 05 rfc2581                                     
TCP Implementation Working Group                              W. Stevens
INTERNET DRAFT                                                Consultant
File: draft-ietf-tcpimpl-cong-control-00.txt                   M. Allman
                                            NASA Lewis/Sterling Software
                                                               V. Paxson
                                                            August, 1998

                        TCP Congestion Control

Status of this Memo

    This document is an Internet-Draft.  Internet-Drafts are working
    documents of the Internet Engineering Task Force (IETF), its areas,
    and its working groups.  Note that other groups may also distribute
    working documents as Internet-Drafts.

    Internet-Drafts are draft documents valid for a maximum of six
    months and may be updated, replaced, or obsoleted by other documents
    at any time.  It is inappropriate to use Internet-Drafts as
    reference material or to cite them other than as ``work in

    To view the entire list of current Internet-Drafts, please check the
    "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow
    Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern
    Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific
    Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast).


    This document defines TCP's four intertwined congestion control
    algorithms: slow start, congestion avoidance, fast retransmit, and
    fast recovery.  In addition, the document specifies how TCP should
    begin transmission after a relatively long idle period, as well as
    discussing various acknowledgment generation methods.

1   Introduction

    This document specifies four TCP [Pos81] congestion control
    algorithms: slow start, congestion avoidance, fast retransmit and
    fast recovery.  These algorithms were devised in [Jac88] and
    [Jac90].  Their use with TCP is required by [Bra89].

    This document is an update of [Ste97].  In addition to specifying
    the congestion control algorithms, this document specifies what TCP
    connections should do after a relatively long idle period, as well
    as specifying and clarifying some of the issues pertaining to TCP
    ACK generation.

    Note that [Ste94] provides examples of these algorithms in action
    and [WS95] provides an explanation of the source code for the BSD
    implementation of these algorithms.

Expires: February, 1999                                           [Page 1]

draft-ietf-tcpimpl-cong-control-00.txt                       August 1998

    This document is organized as follows.  Section 2 provides various
    definitions which will be used throughout the paper.  Section 3
    provides a specification of the congestion control algorithms.
    Section 4 outlines concerns related to the congestion control
    algorithms and finally, section 5 outlines security considerations.

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
    document are to be interpreted as described in [Bra97].

2   Definitions

    This section provides the definition of several terms that will be
    used throughout the remainder of this document.

        A segment is ANY TCP/IP data or acknowledgment packet (or both).

        The MSS is the largest segment size that can be used.  The size
        does not include the TCP/IP headers and options.

        A segment that contains the maximum number of data bytes
        permitted (i.e., a segment containing MSS bytes of data).

        The most recently advertised receiver window.

        A TCP state variable that limits the amount of data a TCP can
        send.  At any given time, a TCP MUST NOT send data with a
        sequence number higher than the sum of the highest acknowledged
        sequence number and the minimum of cwnd and rwnd.

        The initial window is the size of the sender's congestion window
        when a connection is established.

        The loss window is the size of the congestion window after a TCP
        sender detects loss using its retransmission timer.

        The restart window is the size of the congestion window after a
        TCP restarts transmission after an idle period.

3  Congestion Control Algorithms

    This section defines the four congestion control algorithms: slow
    start, congestion avoidance, fast retransmit and fast recovery,
    developed in [Jac88] and [Jac90].  In some situations it may be
    beneficial for a TCP sender to be more conservative than the
    algorithms allow, however a TCP MUST NOT be more aggressive than the

Expires: February, 1999                                           [Page 2]

draft-ietf-tcpimpl-cong-control-00.txt                       August 1998

    following algorithms allow (that is, MUST NOT send data when the
    value of cwnd computed by the following algorithms would not allow
    the data to be sent).

3.1 Slow Start and Congestion Avoidance

    The slow start and congestion avoidance algorithms MUST be used by a
    TCP sender to control the amount of outstanding data being injected
    into the network.  To implement these algorithms, two variables are
    added to the TCP per-connection state.  The congestion window (cwnd)
    is a sender-side limit on the amount of data the sender can transmit
    into the network before receiving an acknowledgment (ACK), while the
    receiver's advertised window (rwnd) is a receiver-side limit on the
    amount of outstanding data.  The minimum of cwnd and rwnd governs
    data transmission.

    Another state variable, the slow start threshold (ssthresh), is used
    to determine whether the slow start or congestion avoidance
    algorithm is used to control data transmission, as discussed below.

    Beginning transmission into a network with unknown conditions
    requires TCP to slowly probe the network to determine the available
    capacity, in order to avoid congesting the network with an
    inappropriately large burst of data.  The slow start algorithm is
    used for this purpose at the beginning of a transfer, or after
    repairing loss detected by the retransmission timer.

    IW, the initial value of cwnd, MUST be less than or equal to MSS

    We note that a non-standard, experimental TCP extension allows that
    a TCP MAY use a larger initial window (IW), as defined in equation 1

               IW = min (4*MSS, max (2*MSS, 4380 bytes))             (1)

    With this extension, a TCP sender MAY use a 2 segment initial
    window, regardless of the segment size, and 3 and 4 segment initial
    windows MAY be used, provided the combined size of the segments does
    not exceed 4380 bytes.  We do NOT allow this change as part of the
    standard defined by this document.  However, we include discussion
    of (1) in the remainder of this document as a guideline for those
    experimenting with the change, rather than conforming to the present
    standards for TCP congestion control.

    The initial value of ssthresh MAY be arbitrarily high (for example,
    some implementations use the size of the advertised window), but it
    may be reduced in response to congestion.  The slow start algorithm
    is used when cwnd < ssthresh, while the congestion avoidance
    algorithm is used when cwnd > ssthresh.  When cwnd and ssthresh are
    equal the sender may use either slow start or congestion avoidance.

    During slow start, a TCP increments cwnd by at most MSS bytes for
    each ACK received that acknowledges new data.  Slow start ends when

Expires: February, 1999                                           [Page 3]

draft-ietf-tcpimpl-cong-control-00.txt                       August 1998

    cwnd exceeds ssthresh (or, optionally, when it reaches it, as noted
    above); or when cwnd reaches rwnd; or when congestion is observed.

    During congestion avoidance, cwnd is incremented by 1 full-sized
    segment per round-trip time (RTT).  Congestion avoidance continues
    until cwnd reaches the receiver's advertised window or congestion is
    detected.  One formula commonly used to update cwnd during
    congestion avoidance is given in equation 2:

                          cwnd += MSS*MSS/cwnd                       (2)

    This provides an acceptable approximation to the underlying
    principle of increasing cwnd by 1 full-sized segment per RTT.  (Note
    that for a connection in which the receiver acknowledges every data
    segment, (2) proves slightly more aggressive than 1 segment per RTT,
    and for a receiver acknowledging every-other packet, (2) is less

    Implementation Note: Since integer arithmetic is usually used in TCP
    implementations, the formula given in equation 2 can fail to
    increase cwnd when the congestion window is very large (larger than
    MSS*MSS).  If the above formula yields 0, the result SHOULD be
    rounded up to 1 byte.

    Implementation Note: older implementations have an additional
    additive constant on the right-hand side of (2).  This is incorrect
    and can actually lead to diminished performance [PAD+98].

    Another acceptable way to increase cwnd during congestion avoidance
    is to count the number of bytes that have been acknowledged by ACKs
    for new data.  (A drawback of this implementation is that it
    requires maintaining an additional state variable.)  When the number
    of bytes acknowledged reaches cwnd, then cwnd can be incremented by
    up to MSS bytes.  Note that during congestion avoidance, cwnd MUST
    NOT be increased by more than the larger of either 1 full-sized
    segment per RTT, or the value computed using equation 2.

    Implementation Note: some implementations maintain cwnd in units of
    bytes, while others in units of full-sized segments.  The latter
    will find equation (2) difficult to use, and may prefer to use the
    counting approach discussed in the previous paragraph.

    When a TCP sender detects segment loss using the retransmission
    timer, the value of ssthresh MUST be set to no more than the value
    given in equation 3:

              ssthresh = max (min (cwnd, rwnd) / 2, 2*MSS)           (3)

    Implementation Note: an easy mistake to make is to forget the inner
    min() operation and simply use cwnd, which in some implementations
    may incidentally increase well beyond rwnd.

    Furthermore, upon a timeout cwnd MUST be set to no more than the
    loss window, LW, which equals 1 full-sized segment (regardless of

Expires: February, 1999                                           [Page 4]

draft-ietf-tcpimpl-cong-control-00.txt                       August 1998

    the value of IW).  Therefore, after retransmitting the dropped
    segment the TCP sender uses the slow start algorithm to increase the
    window from 1 full-sized segment to the new value of ssthresh, at
    which point congestion avoidance again takes over in a fashion
    identical to that for a connection's initial slow start.

3.3 Fast Retransmit/Fast Recovery

    A TCP receiver SHOULD send an immediate duplicate ACK when an
    out-of-order segment arrives.  The purpose of this ACK is to inform
    the sender that a segment was received out-of-order and which
    sequence number is expected.  From the sender's perspective,
    duplicate ACKs can be caused by a number of network problems.
    First, they can be caused by dropped segments.  In this case, all
    segments after the dropped segment will trigger duplicate ACKs.
    Second, duplicate ACKs can be caused by the re-ordering of data
    segments by the network (not a rare event along some network paths).
    Finally, duplicate ACKs can be caused by replication of ACK or data
    segments by the network.

    The TCP sender SHOULD use the "fast retransmit" algorithm to detect
    and repair loss, based on incoming duplicate ACKs.  The fast
    retransmit algorithm uses the arrival of 3 duplicate ACKs (i.e., 4
    identical ACKs) as an indication that a segment has been lost.
    After receiving 3 duplicate ACKs, TCP performs a retransmission of
    what appears to be the missing segment, without waiting for the
    retransmission timer to expire.

    After the fast retransmit sends what appears to be the missing
    segment, the "fast recovery" algorithm governs the transmission of
    new data until a non-duplicate ACK arrives.  The reason for not
    performing slow start is that the receipt of the duplicate ACKs not
    only tells the TCP that a segment has been lost, but also that
    segments are leaving the network.  In other words, since the
    receiver can only generate a duplicate ACK when a segment has
    arrived, that segment has left the network and is in the receiver's
    buffer, so we know it is no longer consuming network resources.
    Furthermore, since the ACK "clock" [Jac88] is preserved, the TCP
    sender can continue to transmit new segments (although transmission
    must continue using a reduced cwnd).

    The fast retransmit and fast recovery algorithms are usually
    implemented together as follows.

    1.  When the third duplicate ACK is received, set ssthresh to no
        more than the value given in equation 3.

    2.  Retransmit the lost segment and set cwnd to ssthresh plus 3*MSS.
        This artificially "inflates" the congestion window by the number
        of segments (three) that have left the network and which the
        receiver has buffered.

    3.  For each additional duplicate ACK received, increment cwnd by
        MSS.  This artificially inflates the congestion window in order

Expires: February, 1999                                           [Page 5]

draft-ietf-tcpimpl-cong-control-00.txt                       August 1998

        to reflect the additional segment that has left the network.

    4.  Transmit a segment, if allowed by the new value of cwnd and the
        receiver's advertised window.

    5.  When the next ACK arrives that acknowledges new data, set cwnd
        to ssthresh (the value set in step 1).  This is termed
        "deflating" the window.

        This ACK should be the acknowledgment elicited by the
        retransmission from step 1, one RTT after the retransmission
        (though it may arrive sooner in the presence of significant
        out-of-order delivery of data segments at the receiver).
        Additionally, this ACK should acknowledge all the intermediate
        segments sent between the lost segment and the receipt of the
        first duplicate ACK, if none of these were lost.

    Implementing fast retransmit/fast recovery in this manner can lead
    to a phenomenon which allows the fast retransmit algorithm to repair
    multiple dropped segments from a single window of data [Flo94].
    However, in doing so, the size of cwnd is also reduced multiple
    times, which reduces performance.  The following steps MAY be taken
    to reduce the impact of successive fast retransmits on performance.

    A.  After the third duplicate ACK is received follow step 1 above,
        but also record the highest sequence number transmitted

    B.  Instead of reducing cwnd to ssthresh upon receipt of the first
        non-duplicate ACK received (step 5), the sender should wait
        until an ACK covering send_high is received.  In addition, each
        duplicate ACK received should continue to artificially inflate
        cwnd by 1 MSS.

    C.  A non-duplicate ACK that does not cover send_high, followed by 3
        duplicate ACKs should not reduce ssthresh or cwnd but SHOULD
        trigger a retransmission.  A non-duplicate ACK that does not
        cover send_high SHOULD NOT cause any adjustment in cwnd.

4   Additional Considerations

4.1 Re-starting Idle Connections

    A known problem with the TCP congestion control algorithms described
    above is that they allow a potentially inappropriate burst of
    traffic to be transmitted after TCP has been idle for a relatively
    long period of time.  After an idle period, TCP cannot use the ACK
    clock to strobe new segments into the network, as all the ACKs have
    drained from the network.  Therefore, as specified above, TCP can
    potentially send a cwnd-size line-rate burst into the network after
    an idle period.

    [Jac88] recommends that a TCP use slow start to restart transmission
    after a relatively long idle period.  Slow start serves to restart

Expires: February, 1999                                           [Page 6]

draft-ietf-tcpimpl-cong-control-00.txt                       August 1998

    the ACK clock, just as it does at the beginning of a transfer.  This
    mechanism has been widely deployed in the following manner.  When
    TCP has not received a segment for more than one retransmission
    timeout, cwnd is reduced to the value of the restart window (RW)
    before transmission begins.

    For the purposes of this standard, we define RW = IW = 1 full-sized

    We note that the non-standard experimental extension to TCP defined
    in [AFP98] defines RW = min(IW, cwnd), with the definition of IW
    adjusted per equation (1) above.

    Using the last time a segment was received to determine whether or
    not to decrease cwnd fails to deflate cwnd in the common case of
    persistent HTTP connections [HTH98].  In this case, a WWW server
    receives a request before transmitting data to the WWW browser.  The
    reception of the request makes the test for an idle connection fail,
    and allows the TCP to begin transmission with a possibly
    inappropriately large cwnd.

    Therefore, a TCP SHOULD reduce cwnd to no more than RW before
    beginning transmission if the TCP has not sent data in an interval
    exceeding the retransmission timeout.

4.2 Acknowledgment Mechanisms

    The delayed ACK algorithm specified in [Bra89] SHOULD be used by a
    TCP receiver.  When used, a TCP receiver MUST NOT excessively delay
    acknowledgments.  Specifically, an ACK MUST be generated for every
    second full-sized segment.  (This "MUST" is listed in [Bra89] in one
    place as a SHOULD and another as a MUST; here we unambiguously state
    it is a MUST.)  Furthermore, an ACK SHOULD be generated for every
    second segment regardless of size.  Finally, an ACK MUST NOT be
    delayed for more than 500 ms waiting on a second full-sized segment
    to arrive.  Out-of-order data segments SHOULD be acknowledged
    immediately, in order to trigger the fast retransmit algorithm.

    A TCP receiver MUST NOT generate more than one ACK for every
    incoming segment.

    TCP implementations that implement the selective acknowledgment
    (SACK) option [MMFR96] are able to determine which segments have not
    arrived at the receiver.  Consequently, they can retransmit the lost
    segments more quickly than TCPs without SACKs.  This allows a TCP
    sender to repair multiple losses in roughly one RTT after detecting
    loss [FF96,MM96a,MM96b].  While no specific SACK-based recovery
    algorithm has yet been standardized, any SACK-based algorithm should
    follow the general principles embodied by the above algorithms.
    First, as soon as loss is detected, ssthresh should be decreased per
    equation (3).  Second, in the RTT following loss detection, the
    number of segments sent should be no more than half the number
    transmitted in the previous RTT (i.e., before loss occurred).
    Third, after the recovery period is finished, cwnd should be set to

Expires: February, 1999                                           [Page 7]

draft-ietf-tcpimpl-cong-control-00.txt                       August 1998

    the reduced value of ssthresh.  The SACK-based algorithms outlined
    in [FF96,MM96a,MM96b] adhere to these guidelines.

5.  Security Considerations

    This document requires a TCP to diminish its sending rate in the
    presence of retransmission timeouts and the arrival of duplicate
    acknowledgments.  An attacker can therefore impair the performance
    of a TCP connection by either causing data packets or their
    acknowledgments to be lost, or by forging excessive duplicate
    acknowledgments.  Causing two congestion control events back-to-back
    will often cut ssthresh to its minimum value of 2*MSS, causing the
    connection to immediately enter the slower-performing congestion
    avoidance phase.

    The Internet to a considerable degree relies on the correct
    implementation of these algorithms in order to preserve network
    stability and avoid congestion collapse.  An attacker could cause
    TCP endpoints to respond more aggressively in the face of congestion
    by forging excessive duplicate acknowledgments or excessive
    acknowledgments for new data.  Conceivably, such an attack could
    drive a portion of the network into congestion collapse.


    The four algorithms that are described were developed by Van

    Some of the text from this document is taken from "TCP/IP
    Illustrated, Volume 1: The Protocols" by W. Richard Stevens
    (Addison-Wesley, 1994) and "TCP/IP Illustrated, Volume 2: The
    Implementation" by Gary R. Wright and W.  Richard Stevens
    (Addison-Wesley, 1995).  This material is used with the permission
    of Addison-Wesley.

    Sally Floyd devised the algorithm presented in section 3.3 for
    avoiding multiple cwnd cutbacks in the presence of multiple packets
    lost from the same flight.  Craig Partridge and Joe Touch
    contributed a number of helpful suggestions.


    [AFP98] M. Allman, S. Floyd, C. Partridge, Increasing TCP's Initial
        Window Size, Internet-Draft draft-floyd-incr-init-win-03.txt.
        May, 1998.  (Work in progress).

    [Bra89] B. Braden, ed., "Requirements for Internet Hosts --
        Communication Layers," RFC 1122, Oct. 1989.

    [Bra97] S. Bradner, "Key words for use in RFCs to Indicate
        Requirement Levels", BCP 14, RFC 2119, March 1997.

Expires: February, 1999                                           [Page 8]

draft-ietf-tcpimpl-cong-control-00.txt                       August 1998

    [FF96] Kevin Fall and Sally Floyd.  Simulation-based Comparisons of
        Tahoe, Reno and SACK TCP.  Computer Communication Review, July
        1996.  ftp://ftp.ee.lbl.gov/papers/sacks.ps.Z.

    [Flo94] S. Floyd, TCP and Successive Fast Retransmits. Technical
        report, October 1994.

    [HTH98] Amy Hughes, Joe Touch, John Heidemann.  Internet-Draft
        draft-ietf-tcpimpl-restart-00.txt, March 1998.  (Work in

    [Jac88] V. Jacobson, "Congestion Avoidance and Control," Computer
        Communication Review, vol. 18, no. 4, pp. 314-329, Aug. 1988.

    [Jac90] V. Jacobson, "Modified TCP Congestion Avoidance Algorithm,"
        end2end-interest mailing list, April 30, 1990.

    [MM96a] M. Mathis, J. Mahdavi, "Forward Acknowledgment: Refining TCP
        Congestion Control," Proceedings of SIGCOMM'96, August, 1996,
        Stanford, CA.  Available from

    [MM96b] M. Mathis, J. Mahdavi, "TCP Rate-Halving with Bounding
        Parameters" Available from

    [MMFR96] M. Mathis, J. Mahdavi, S. Floyd, A. Romanow, "TCP Selective
        Acknowledgement Options", RFC 2018, October 1996.

    [PAD+98] V. Paxson, M. Allman, S. Dawson, J. Griner, I. Heavens,
        K. Lahey, J. Semke, B. Volz.  Internet-Draft
        draft-ietf-tcpimpl-prob-04.txt, August 1998.  (Work in

    [Pos81] J. Postel, Transmission Control Protocol, September 1981.
        RFC 793.

    [Ste94] W. R. Stevens, "TCP/IP Illustrated, Volume 1: The
        Protocols", Addison-Wesley, 1994.

    [Ste97] W. R. Stevens, "TCP Slow Start, Congestion Avoidance, Fast
        Retransmit, and Fast Recovery Algorithms", RFC 2001, January

    [WS95] G. R. Wright, W. R. Stevens, "TCP/IP Illustrated, Volume 2:
        The Implementation", Addison-Wesley, 1995.

Expires: February, 1999                                           [Page 9]

draft-ietf-tcpimpl-cong-control-00.txt                       August 1998

Author's  Address:

    W. Richard Stevens
    1202 E. Paseo del Zorro
    Tucson, AZ  85718

    Mark Allman
    NASA Lewis Research Center/Sterling Software
    21000 Brookpark Rd.  MS 54-2
    Cleveland, OH  44135

    Vern Paxson
    Network Research Group
    Lawrence Berkeley National Laboratory
    Berkeley, CA 94720

Expires: February, 1999                                          [Page 10]