Identifying Modified Explicit Congestion Notification (ECN) Semantics for Ultra-Low Queuing Delay (L4S)
draft-ietf-tsvwg-ecn-l4s-id-06

Document Type Active Internet-Draft (tsvwg WG)
Last updated 2019-03-11
Replaces draft-briscoe-tsvwg-ecn-l4s-id
Stream IETF
Intended RFC status (None)
Formats plain text xml pdf html bibtex
Stream WG state WG Document
Document shepherd Wesley Eddy
IESG IESG state I-D Exists
Consensus Boilerplate Unknown
Telechat date
Responsible AD (None)
Send notices to Wesley Eddy <wes@mti-systems.com>
Transport Services (tsv)                                  K. De Schepper
Internet-Draft                                           Nokia Bell Labs
Intended status: Experimental                            B. Briscoe, Ed.
Expires: September 12, 2019                                    CableLabs
                                                          March 11, 2019

 Identifying Modified Explicit Congestion Notification (ECN) Semantics
                   for Ultra-Low Queuing Delay (L4S)
                     draft-ietf-tsvwg-ecn-l4s-id-06

Abstract

   This specification defines the identifier to be used on IP packets
   for a new network service called low latency, low loss and scalable
   throughput (L4S).  It is similar to the original (or 'Classic')
   Explicit Congestion Notification (ECN).  'Classic' ECN marking was
   required to be equivalent to a drop, both when applied in the network
   and when responded to by a transport.  Unlike 'Classic' ECN marking,
   for packets carrying the L4S identifier, the network applies marking
   more immediately and more aggressively than drop, and the transport
   response to each mark is reduced and smoothed relative to that for
   drop.  The two changes counterbalance each other so that the
   throughput of an L4S flow will be roughly the same as a 'Classic'
   flow under the same conditions.  However, the much more frequent
   control signals and the finer responses to them result in ultra-low
   queuing delay without compromising link utilization, and low delay is
   maintained during high load.  Examples of new active queue management
   (AQM) marking algorithms and examples of new transports (whether TCP-
   like or real-time) are specified separately.  The new L4S identifier
   is the key piece that enables them to interwork and distinguishes
   them from 'Classic' traffic.  It gives an incremental migration path
   so that existing 'Classic' TCP traffic will be no worse off, but it
   can be prevented from degrading the ultra-low delay and loss of the
   new scalable transports.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any

De Schepper & Briscoe  Expires September 12, 2019               [Page 1]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 12, 2019.

Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.1.  Problem . . . . . . . . . . . . . . . . . . . . . . . . .   4
     1.2.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   6
     1.3.  Scope . . . . . . . . . . . . . . . . . . . . . . . . . .   6
   2.  Consensus Choice of L4S Packet Identifier: Requirements . . .   7
   3.  L4S Packet Identification at Run-Time . . . . . . . . . . . .   8
   4.  Prerequisite Transport Layer Behaviour  . . . . . . . . . . .   8
     4.1.  Prerequisite Codepoint Setting  . . . . . . . . . . . . .   8
     4.2.  Prerequisite Transport Feedback . . . . . . . . . . . . .   8
     4.3.  Prerequisite Congestion Response  . . . . . . . . . . . .   9
   5.  Prerequisite Network Node Behaviour . . . . . . . . . . . . .  11
     5.1.  Prerequisite Classification and Re-Marking Behaviour  . .  11
     5.2.  The Meaning of L4S CE Relative to Drop  . . . . . . . . .  11
     5.3.  Exception for L4S Packet Identification by Network Nodes
           with Transport-Layer Awareness  . . . . . . . . . . . . .  12
     5.4.  Interaction of the L4S Identifier with other Identifiers   13
       5.4.1.  Examples of Other Identifiers Complementing L4S
               Identifiers . . . . . . . . . . . . . . . . . . . . .  13
         5.4.1.1.  Inclusion of Additional Traffic with L4S  . . . .  13
         5.4.1.2.  Exclusion of Traffic From L4S Treatment . . . . .  14
       5.4.2.  Generalized Combination of L4S and Other Identifiers   15
   6.  L4S Experiments . . . . . . . . . . . . . . . . . . . . . . .  16
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  16
   8.  Security Considerations . . . . . . . . . . . . . . . . . . .  16
   9.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  16
   10. References  . . . . . . . . . . . . . . . . . . . . . . . . .  17

De Schepper & Briscoe  Expires September 12, 2019               [Page 2]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

     10.1.  Normative References . . . . . . . . . . . . . . . . . .  17
     10.2.  Informative References . . . . . . . . . . . . . . . . .  17
   Appendix A.  The 'Prague L4S Requirements'  . . . . . . . . . . .  23
     A.1.  Requirements for Scalable Transport Protocols . . . . . .  24
       A.1.1.  Use of L4S Packet Identifier  . . . . . . . . . . . .  24
       A.1.2.  Accurate ECN Feedback . . . . . . . . . . . . . . . .  24
       A.1.3.  Fall back to Reno-friendly congestion control on
               packet loss . . . . . . . . . . . . . . . . . . . . .  24
       A.1.4.  Fall back to Reno-friendly congestion control on
               classic ECN bottlenecks . . . . . . . . . . . . . . .  25
       A.1.5.  Reduce RTT dependence . . . . . . . . . . . . . . . .  25
       A.1.6.  Scaling down to fractional congestion windows . . . .  26
       A.1.7.  Measuring Reordering Tolerance in Time Units  . . . .  27
     A.2.  Scalable Transport Protocol Optimizations . . . . . . . .  29
       A.2.1.  Setting ECT in TCP Control Packets and
               Retransmissions . . . . . . . . . . . . . . . . . . .  29
       A.2.2.  Faster than Additive Increase . . . . . . . . . . . .  29
       A.2.3.  Faster Convergence at Flow Start  . . . . . . . . . .  30
   Appendix B.  Alternative Identifiers  . . . . . . . . . . . . . .  30
     B.1.  ECT(1) and CE codepoints  . . . . . . . . . . . . . . . .  31
     B.2.  ECN Plus a Diffserv Codepoint (DSCP)  . . . . . . . . . .  33
     B.3.  ECN capability alone  . . . . . . . . . . . . . . . . . .  35
     B.4.  Protocol ID . . . . . . . . . . . . . . . . . . . . . . .  37
     B.5.  Source or destination addressing  . . . . . . . . . . . .  37
     B.6.  Summary: Merits of Alternative Identifiers  . . . . . . .  37
   Appendix C.  Potential Competing Uses for the ECT(1) Codepoint  .  38
     C.1.  Integrity of Congestion Feedback  . . . . . . . . . . . .  38
     C.2.  Notification of Less Severe Congestion than CE  . . . . .  39
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  40

1.  Introduction

   This specification defines the identifier to be used on IP packets
   for a new network service called low latency, low loss and scalable
   throughput (L4S).  It is similar to the original (or 'Classic')
   Explicit Congestion Notification (ECN [RFC3168]).  'Classic' ECN
   marking was required to be equivalent to a drop, both when applied in
   the network and when responded to by a transport.  Unlike 'Classic'
   ECN marking, the network applies L4S marking more immediately and
   more aggressively than drop, and the transport response to each mark
   is reduced and smoothed relative to that for drop.  The two changes
   counterbalance each other so that the throughput of an L4S flow will
   be roughly the same as a 'Classic' flow under the same conditions.
   Nonetheless, the much more frequent control signals and the finer
   responses to them result in ultra-low queuing delay without
   compromising link utilization, and low delay is maintained during
   high load.

De Schepper & Briscoe  Expires September 12, 2019               [Page 3]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   An example of a scalable congestion control that would enable the L4S
   service is Data Centre TCP (DCTCP), which until now has been
   applicable solely to controlled environments like data centres
   [RFC8257], because it is too aggressive to co-exist with existing
   TCP.  The DualQ Coupled AQM, which is defined in a complementary
   experimental specification [I-D.ietf-tsvwg-aqm-dualq-coupled], is an
   AQM framework that enables scalable congestion controls like DCTCP to
   co-exist with existing traffic, each getting roughly the same flow
   rate when they compete under similar conditions.  Note that a
   transport such as DCTCP is still not safe to deploy on the Internet
   unless it satisfies the requirements listed in Section 4.  Also note
   that L4S is not only for elastic TCP-like traffic - there are
   scalable congestion controls for real-time media, such as the L4S
   variant of the SCReAM [RFC8298] real-time media congestion avoidance
   technique (RMCAT).

   The new L4S identifier is the key piece that enables L4S hosts and
   L4S network nodes to interwork and distinguishes their traffic from
   'Classic' traffic.  It gives an incremental migration path so that
   existing 'Classic' TCP traffic will be no worse off, but it can be
   prevented from degrading the ultra-low delay and loss of the new
   scalable congestion controls.  The performance improvement is so
   great that it is motivating initial deployment of the separate parts
   of this system.

1.1.  Problem

   Latency is becoming the critical performance factor for many (most?)
   applications on the public Internet, e.g. interactive Web, Web
   services, voice, conversational video, interactive video, interactive
   remote presence, instant messaging, online gaming, remote desktop,
   cloud-based applications, and video-assisted remote control of
   machinery and industrial processes.  In the developed world, further
   increases in access network bit-rate offer diminishing returns,
   whereas latency is still a multi-faceted problem.  In the last decade
   or so, much has been done to reduce propagation time by placing
   caches or servers closer to users.  However, queuing remains a major
   intermittent component of latency.

   The Diffserv architecture provides Expedited Forwarding [RFC3246], so
   that low latency traffic can jump the queue of other traffic.
   However, on access links dedicated to individual sites (homes, small
   enterprises or mobile devices), often all traffic at any one time
   will be latency-sensitive.  Then Diffserv is of little use.  Instead,
   we need to remove the causes of any unnecessary delay.

   The bufferbloat project has shown that excessively-large buffering
   ('bufferbloat') has been introducing significantly more delay than

De Schepper & Briscoe  Expires September 12, 2019               [Page 4]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   the underlying propagation time.  These delays appear only
   intermittently--only when a capacity-seeking (e.g.  TCP) flow is long
   enough for the queue to fill the buffer, making every packet in other
   flows sharing the buffer sit through the queue.

   Active queue management (AQM) was originally developed to solve this
   problem (and others).  Unlike Diffserv, which gives low latency to
   some traffic at the expense of others, AQM controls latency for _all_
   traffic in a class.  In general, AQMs introduce an increasing level
   of discard from the buffer the longer the queue persists above a
   shallow threshold.  This gives sufficient signals to capacity-seeking
   (aka. greedy) flows to keep the buffer empty for its intended
   purpose: absorbing bursts.  However, RED [RFC2309] and other
   algorithms from the 1990s were sensitive to their configuration and
   hard to set correctly.  So, AQM was not widely deployed.

   More recent state-of-the-art AQMs, e.g. fq_CoDel [RFC8290],
   PIE [RFC8033], Adaptive RED [ARED01], are easier to configure,
   because they define the queuing threshold in time not bytes, so it is
   invariant for different link rates.  However, no matter how good the
   AQM, the sawtoothing rate of TCP will either cause queuing delay to
   vary or cause the link to be under-utilized.  Even with a perfectly
   tuned AQM, the additional queuing delay will be of the same order as
   the underlying speed-of-light delay across the network.  Flow-queuing
   can isolate one flow from another, but it cannot isolate a TCP flow
   from the delay variations it inflicts on itself, and it has other
   problems - it overrides the flow rate decisions of variable rate
   video applications, it does not recognise the flows within IPSec VPN
   tunnels and it is relatively expensive to implement.

   Latency is not our only concern: It was known when TCP was first
   developed that it would not scale to high bandwidth-delay products
   [TCP-CA].  Given regular broadband bit-rates over WAN distances are
   already [RFC3649] beyond the scaling range of 'Classic' TCP Reno,
   'less unscalable' Cubic [RFC8312] and
   Compound [I-D.sridharan-tcpm-ctcp] variants of TCP have been
   successfully deployed.  However, these are now approaching their
   scaling limits.  Unfortunately, fully scalable congestion controls
   such as DCTCP [RFC8257] cause 'Classic' TCP to starve itself, which
   is why they have been confined to private data centres or research
   testbeds (until now).

   It turns out that a TCP algorithm like DCTCP that solves the latency
   problem also solves TCP's scalability problem.  The finer sawteeth
   have low amplitude, so they cause very little queuing delay variation
   and the number of sawteeth per round trip remains invariant, which
   maintains constant tight control as flow-rate scales.  A supporting
   paper [DCttH15] gives the full explanation of why the design solves

De Schepper & Briscoe  Expires September 12, 2019               [Page 5]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   both the latency and the scaling problems, both in plain English and
   in more precise mathematical form.  The explanation is summarised
   without the maths in the L4S architecture document
   [I-D.ietf-tsvwg-l4s-arch].

1.2.  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in
   [RFC2119].  In this document, these words will appear with that
   interpretation only when in ALL CAPS.  Lower case uses of these words
   are not to be interpreted as carrying RFC-2119 significance.

   Classic service:  The 'Classic' service is intended for all the
      behaviours that currently co-exist with TCP Reno (e.g.  TCP Cubic,
      Compound, SCTP, etc).

   Low-Latency, Low-Loss and Scalable (L4S) service:  The 'L4S' service
      is intended for traffic from scalable congestion control
      algorithms such as Data Centre TCP.  But it is also more general--
      it allows the set of congestion controls with similar scaling
      properties to DCTCP to evolve (e.g.  Relentless TCP [Mathis09] and
      the L4S variant of SCREAM for real-time media [RFC8298].

      Both Classic and L4S services can cope with a proportion of
      unresponsive or less-responsive traffic as well, as long as it
      does not build a queue (e.g.  DNS, VoIP, game sync datagrams,
      etc).

   Classic ECN:  The original Explicit Congestion Notification (ECN)
      protocol [RFC3168].

1.3.  Scope

   The new L4S identifier defined in this specification is applicable
   for IPv4 and IPv6 packets (as for classic ECN [RFC3168]).  It is
   applicable for the unicast, multicast and anycast forwarding modes.

   The L4S identifier is an orthogonal packet classification to the
   Differentiated Services Code Point (DSCP [RFC2474]).  Section 5.4
   explains what this means in practice.

   This document is intended for experimental status, so it does not
   update any standards track RFCs.  Therefore it depends on [RFC8311],
   which is a standards track specification that:

De Schepper & Briscoe  Expires September 12, 2019               [Page 6]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   o  updates the ECN proposed standard [RFC3168] to allow experimental
      track RFCs to relax the requirement that an ECN mark must be
      equivalent to a drop, both when applied by the network, and when
      responded to by the sender;

   o  changes the status of the experimental ECN nonce [RFC3540] to
      historic;

   o  makes consequent updates to the following additional proposed
      standard RFCs to reflect the above two bullets:

      *  ECN for RTP [RFC6679];

      *  the congestion control specifications of various DCCP
         congestion control identifier (CCID) profiles [RFC4341],
         [RFC4342], [RFC5622].

2.  Consensus Choice of L4S Packet Identifier: Requirements

   This subsection briefly records the process that led to a consensus
   choice of L4S identifier, selected from all the alternatives in
   Appendix B.

   Ideally, the identifier for packets using the Low Latency, Low Loss,
   Scalable throughput (L4S) service ought to meet the following
   requirements:

   o  it SHOULD survive end-to-end between source and destination
      applications: across the boundary between host and network,
      between interconnected networks, and through middleboxes;

   o  it SHOULD be common to IPv4 and IPv6 and transport-agnostic;

   o  it SHOULD be incrementally deployable;

   o  it SHOULD enable an AQM to classify packets encapsulated by outer
      IP or lower-layer headers;

   o  it SHOULD consume minimal extra codepoints;

   o  it SHOULD be consistent on all the packets of a transport layer
      flow, so that some packets of a flow are not served by a different
      queue to others.

   Whether the identifier would be recoverable if the experiment failed
   is a factor that could be taken into account.  However, this has not
   been made a requirement, because that would favour schemes that would
   be easier to fail, rather than those more likely to succeed.

De Schepper & Briscoe  Expires September 12, 2019               [Page 7]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   It is recognised that the chosen identifier is unlikely to satisfy
   all these requirements, particularly given the limited space left in
   the IP header.  Therefore a compromise will be necessary, which is
   why all the requirements are expressed with the word 'SHOULD' not
   'MUST'.  Appendix B discusses the pros and cons of the compromises
   made in various competing identification schemes against the above
   requirements.

   On the basis of this analysis, "ECT(1) and CE codepoints" is the best
   compromise.  Therefore this scheme is defined in detail in the
   following sections, while Appendix B records the rationale for this
   decision.

3.  L4S Packet Identification at Run-Time

   The L4S treatment is an experimental track alternative packet marking
   treatment [RFC4774] to the classic ECN treatment [RFC3168], which has
   been updated by [RFC8311] to allow this experiment (amongst others).
   Like classic ECN, L4S ECN identifies both network and host behaviour:
   it identifies the marking treatment that network nodes are expected
   to apply to L4S packets, and it identifies packets that have been
   sent from hosts that are expected to comply with a broad type of
   sending behaviour.

   For a packet to receive L4S treatment as it is forwarded, the sender
   sets the ECN field in the IP header to the ECT(1) codepoint.  See
   Section 4 for full transport layer behaviour requirements, including
   feedback and congestion response.

   A network node that implements the L4S service normally classifies
   arriving ECT(1) and CE packets for L4S treatment.  See Section 5 for
   full network element behaviour requirements, including
   classification, ECN-marking and interaction of the L4S identifier
   with other identifiers and per-hop behaviours.

4.  Prerequisite Transport Layer Behaviour

4.1.  Prerequisite Codepoint Setting

   A sender that wishes a packet to receive L4S treatment as it is
   forwarded, MUST set the ECN field in the IP header (v4 or v6) to the
   ECT(1) codepoint.

4.2.  Prerequisite Transport Feedback

   In general, a scalable congestion control needs feedback of the
   extent of CE marking on the forward path.  When ECN was added to TCP
   [RFC3168], the feedback method reported no more than one CE mark per

De Schepper & Briscoe  Expires September 12, 2019               [Page 8]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   round trip.  Some transport protocols derived from TCP mimic this
   behaviour while others report the accurate extent of TCP marking.
   This means that some transport protocols will need to be updated as a
   prerequisite for scalable congestion control.  The position for a few
   well-known transport protocols is given below.

   TCP:  Support for accurate ECN feedback (AccECN
      [I-D.ietf-tcpm-accurate-ecn]) by both ends is a prerequisite for
      scalable congestion control.  Therefore, the presence of ECT(1) in
      the IP headers even in one direction of a TCP connection will
      imply that both ends support AccECN.  However, the converse does
      not apply.  So even if both ends support AccECN, either of the two
      ends can choose not to use a scalable congestion control, whatever
      the other end's choice.

   SCTP:  A suitable ECN feedback mechanism for SCTP could add a chunk
      to report the number of received CE marks (e.g.
      [I-D.stewart-tsvwg-sctpecn]), and update the ECN feedback protocol
      sketched out in Appendix A of the standards track specification of
      SCTP [RFC4960].

   RTP over UDP:  A prerequisite for scalable congestion control is for
      both (all) ends of one media-level hop to signal ECN support
      [RFC6679] and use the new generic RTCP feedback format of
      [I-D.ietf-avtcore-cc-feedback-message].  The presence of ECT(1)
      implies that both (all) ends of that hop support ECN.  However,
      the converse does not apply, so each end of a media-level hop can
      independently choose not to use a scalable congestion control,
      even if both ends support ECN.

   QUIC:  Support for sufficiently fine-grained ECN feedback is provided
      by the first IETF QUIC transport [I-D.ietf-quic-transport].

   DCCP:  The ACK vector in DCCP [RFC4340] is already sufficient to
      report the extent of CE marking as needed by a scalable congestion
      control.

4.3.  Prerequisite Congestion Response

   As a condition for a host to send packets with the L4S identifier
   (ECT(1)), it SHOULD implement a congestion control behaviour that
   ensures the flow rate is inversely proportional to the proportion of
   bytes in packets marked with the CE codepoint.  This is termed a
   scalable congestion control, because the number of control signals
   (ECN marks) per round trip remains roughly constant for any flow
   rate.  As with all transport behaviours, a detailed specification
   will need to be defined for each type of transport or application,
   including the timescale over which the proportionality is averaged,

De Schepper & Briscoe  Expires September 12, 2019               [Page 9]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   and control of burstiness.  The inverse proportionality requirement
   above is worded as a 'SHOULD' rather than a 'MUST' to allow
   reasonable flexibility when defining these specifications.

   Data Center TCP (DCTCP [RFC8257]) and the L4S variant of SCReAM
   [RFC8298] are examples of a scalable congestion controls.

   Each sender in a session can use a scalable congestion control
   independently of the congestion control used by the receiver(s) when
   they send data.  Therefore there might be ECT(1) packets in one
   direction and ECT(0) or Not-ECT in the other.

   In order to coexist safely with other Internet traffic, a scalable
   congestion control MUST NOT tag its packets with the ECT(1) codepoint
   unless it complies with the following bulleted requirements.  The
   specification of a particular scalable congestion control MUST
   describe in detail how it satisfies each requirement:

   o  A scalable congestion control MUST react to packet loss in a way
      that will coexist safely with a TCP Reno congestion control
      [RFC5681] (see Appendix A.1.3 for rationale).

   o  A scalable congestion control MUST react to ECN marking from a
      non-L4S but ECN-capable bottleneck in a way that will coexist with
      a TCP Reno congestion control [RFC5681] (see Appendix A.1.4 for
      rationale).

      Note that a scalable congestion control is not expected to change
      to setting ECT(0) while it temporarily falls back to coexist with
      Reno . However an implementer who believes this would be
      beneficial if fall-back persists, can choose to do so,

   o  A scalable congestion control MUST reduce or eliminate RTT bias
      over as wide a range of RTTs as possible, or at least over the
      typical range of RTTs that will interact in the intended
      deployment scenario (see Appendix A.1.5 for rationale).

   o  A scalable congestion control MUST remain responsive to congestion
      when the RTT is significantly smaller than in the current public
      Internet (see Appendix A.1.6 for rationale).

   o  A scalable congestion control MUST detect loss by counting in
      time-based units, which is scalable, as opposed to counting in
      units of packets (as in the 3 DupACK rule of traditional TCP),
      which is not scalable (see Appendix A.1.7 for rationale).

De Schepper & Briscoe  Expires September 12, 2019              [Page 10]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

5.  Prerequisite Network Node Behaviour

5.1.  Prerequisite Classification and Re-Marking Behaviour

   A network node that implements the L4S service MUST classify arriving
   ECT(1) packets for L4S treatment and, other than in the exceptional
   case referred to next, it MUST classify arriving CE packets for L4S
   treatment as well.  CE packets might have originated as ECT(1) or
   ECT(0), but the above rule to classify them as if they originated as
   ECT(1) is the safe choice (see Appendix B.1 for rationale).  The
   exception is where some flow-aware in-network mechanism happens to be
   available for distinguishing CE packets that originated as ECT(0), as
   described in Section 5.3, but there is no implication that such a
   mechanism is necessary.

   An L4S AQM treatment follows similar codepoint transition rules to
   those in RFC 3168.  Specifically, the ECT(1) codepoint MUST NOT be
   changed to any other codepoint than CE, and CE MUST NOT be changed to
   any other codepoint.  An ECT(1) packet is classified as ECN-capable
   and, if congestion increases, an L4S AQM algorithm will mark the ECN
   field as CE for an increasing proportion of packets, otherwise
   forwarding packets unchanged as ECT(1).  Necessary conditions for an
   L4S marking treatment are defined in Section 5.2.  Under persistent
   overload an L4S marking treatment SHOULD turn off ECN marking, using
   drop as a congestion signal until the overload episode has subsided,
   as recommended for all AQMs in [RFC7567] (Section 4.2.1), which
   follows the similar advice in RFC 3168 (Section 7).

   For backward compatibility in uncontrolled environments, a network
   node that implements the L4S treatment MUST also implement a classic
   AQM treatment.  It MUST classify arriving ECT(0) and Not-ECT packets
   for treatment by the Classic AQM (see the discussion of the
   classifier for the dual-queue coupled AQM in
   [I-D.ietf-tsvwg-aqm-dualq-coupled]).  Classic treatment means that
   the AQM will mark ECT(0) packets under the same conditions as it
   would drop Not-ECT packets [RFC3168].

5.2.  The Meaning of L4S CE Relative to Drop

   The likelihood that an AQM drops a Not-ECT Classic packet (p_C) MUST
   be roughly proportional to the square of the likelihood that it would
   have marked it if it had been an L4S packet (p_L) {ToDo cross-ref to
   new section in l4s-arch that explains the rationale for the square}.
   That is

      p_C ~= (p_L / k)^2

De Schepper & Briscoe  Expires September 12, 2019              [Page 11]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   The constant of proportionality (k) does not have to be standardised
   for interoperability, but a value of 2 is RECOMMENDED.

   [I-D.ietf-tsvwg-aqm-dualq-coupled] specifies the essential aspects of
   an L4S AQM, as well as recommending other aspects.  It gives example
   implementations in appendices.

   The term 'likelihood' is used above to allow for marking and dropping
   to be either probabilistic or deterministic.  The example AQMs in
   [I-D.ietf-tsvwg-aqm-dualq-coupled] drop and mark probabilistically,
   so the drop probability is arranged to be the square of the marking
   probability.  Nonetheless, an alternative AQM that dropped and marked
   deterministically would be valid, as long as the dropping frequency
   was proportional to the square of the marking frequency.

   Note that, contrary to RFC 3168, a Dual AQM implementing the L4S and
   Classic treatments does not mark an ECT(1) packet under the same
   conditions that it would have dropped a Not-ECT packet, as allowed by
   [RFC8311], which updates RFC 3168.  However, it does mark an ECT(0)
   packet under the same conditions that it would have dropped a Not-ECT
   packet.

5.3.  Exception for L4S Packet Identification by Network Nodes with
      Transport-Layer Awareness

   To implement the L4S treatment, a network node does not need to
   identify transport-layer flows.  Nonetheless, if an implementer is
   willing to identify transport-layer flows at a network node, and if
   the most recent ECT packet in the same flow was ECT(0), the node MAY
   classify CE packets for classic ECN [RFC3168] treatment.  In all
   other cases, a network node MUST classify all CE packets for L4S
   treatment.  Examples of such other cases are: i) if no ECT packets
   have yet been identified in a flow; ii) if it is not desirable for a
   network node to identify transport-layer flows; or iii) if the most
   recent ECT packet in a flow was ECT(1).

   If an implementer uses flow-awareness to classify CE packets, to
   determine whether the flow is using ECT(0) or ECT(1) it only uses the
   most recent ECT packet of a flow (this advice will need to be
   verified as part of L4S experiments).  This is because a sender might
   switch from sending ECT(1) (L4S) packets to sending ECT(0) (Classic)
   packets, or back again, in the middle of a transport-layer flow (e.g.
   it might manually switch its congestion control module mid-
   connection, or it might be deliberately attempting to confuse the
   network).

De Schepper & Briscoe  Expires September 12, 2019              [Page 12]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

5.4.  Interaction of the L4S Identifier with other Identifiers

5.4.1.  Examples of Other Identifiers Complementing L4S Identifiers

5.4.1.1.  Inclusion of Additional Traffic with L4S

   In a typical case for the public Internet a network element that
   implements L4S might want to classify some low-rate but unresponsive
   traffic (e.g.  DNS, voice, game sync packets) into the low latency
   queue to mix with L4S traffic.  Such non-ECN-based packet types MUST
   be safe to mix with L4S traffic without harming the low latency
   service, where 'safe' is explained in Section 5.4.1.1.1 below.

   In this case it would not be appropriate to call the queue an L4S
   queue, because it is shared by L4S and non-L4S traffic.  Instead it
   will be called the low latency or L queue.  The L queue then offers
   two different treatments:

   o  The L4S treatment, which is a combination of the L4S AQM treatment
      and a priority scheduling treatment;

   o  The low latency treatment, which is solely the priority scheduling
      treatment, without ECN-marking by the AQM.

   To identify packets for just the scheduling treatment, it would be
   inappropriate to use the L4S ECT(1) identifier, because such traffic
   is unresponsive to ECN marking.  Therefore, a network element that
   implements L4S MAY classify additional packets into the L queue if
   they carry certain non-ECN identifiers.  For instance:

   o  addresses of specific applications or hosts configured to be safe
      (but for example cannot set the ECN field for some temporary
      reason);

   o  certain protocols that are usually lightweight (e.g.  ARP, DNS);

   o  specific Diffserv codepoints that indicate traffic with limited
      burstiness such as the EF (Expedited Forwarding) and Voice-Admit
      service classes or equivalent local-use DSCPs (see
      [I-D.briscoe-tsvwg-l4s-diffserv]).

   Of course, a packet that carried both the ECT(1) codepoint and a
   relevant non-ECN identifier would also be classified into the L
   queue.

   For clarity, non-ECN identifiers, such as the examples itemized
   above, might be used by some network operators who believe they
   identify non-L4S traffic that would be safe to mix with L4S traffic.

De Schepper & Briscoe  Expires September 12, 2019              [Page 13]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   They are not alternative ways for a host to indicate that it is
   sending L4S packets.  Only the ECT(1) ECN codepoint indicates to a
   network element that a host is sending L4S packets (and CE indicates
   that it could be).  Specifically ECT(1) indicates that the host
   claims its behaviour satisfies the per-requisite transport
   requirements in Section 4.

5.4.1.1.1.  'Safe' Unresponsive Traffic

   The above section requires unresponsive traffic to be 'safe' to mix
   with L4S traffic.  Ideally this means that the sender never sends any
   sequence of packets at a data rate that exceeds the available
   capacity of the bottleneck link.  However, typically an unresponsive
   transport does not even know the bottleneck capacity of the path, let
   alone its available capacity.  Nonetheless, an application can be
   considered safe enough if it paces packets out (not necessarily
   completely regularly) such that its maximum instantaneous data rate
   from packet to packet stays well below a typical broadband access
   rate.

   This is a vague but useful definition, because it encompasses many
   low latency applications of interest, such as DNS, voice, game sync
   packets, RPC, ACKs, keep-alives, etc.

5.4.1.2.  Exclusion of Traffic From L4S Treatment

   To extend the above example, an operator might want to exclude some
   traffic from the L4S treatment for policy reason, e.g.  security
   (traffic from malicious sources) or commercial (initially the
   operator may wish to confine the benefits of L4S to business
   customers).

   In this exclusion case, the operator MUST classify on the relevant
   locally-used identifiers (e.g. source addresses) before classifying
   the non-matching traffic on the end-to-end L4S ECN identifier.

   The operator MUST NOT re-mark the end-to-end L4S identifier, because
   its decision to exclude certain traffic from L4S treatment is local-
   only.  The end-to-end L4S identifier then survives for other
   operators to use, or indeed, they can apply their own policy,
   independently based on their own choice of locally-used identifiers.
   This approach also allows any operator to remove its locally-applied
   exclusions in future, e.g. if it wishes to widen the benefit of the
   L4S treatment to all its customers.

De Schepper & Briscoe  Expires September 12, 2019              [Page 14]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

5.4.2.  Generalized Combination of L4S and Other Identifiers

   L4S concerns low latency, which it can provide for all traffic
   without differentiation and without affecting bandwidth allocation.
   Diffserv provides for differentiation of both bandwidth and low
   latency, but its control of latency depends on its control of
   bandwidth.  The two can be combined if a network operator wants to
   control bandwidth allocation but it also wants to provide low latency
   - for any amount of traffic within one of these allocations of
   bandwidth (rather than only providing low latency by limiting
   bandwidth) [I-D.briscoe-tsvwg-l4s-diffserv].

   The examples above were framed in the context of providing the
   default Best Efforts Per-Hop Behaviour (PHB) using two queues - a Low
   Latency (L) queue and a Classic (C) Queue.  This single DualQ
   structure is expected to be by far the most common and useful
   arrangement.  But, more generally, an operator might choose to
   control bandwidth allocation through a hierarchy of Diffserv PHBs at
   a node, and to offer one (or more) of these PHBs with a low latency
   and a classic variant.

   In the first case, if we assume that there are no other PHBs except
   the DualQ, if a packet carries ECT(1) or CE, a network element would
   classify it for the L4S treatment irrespective of its DSCP.  And, if
   a packet carried (say) the EF DSCP, the network element could
   classify it into the L queue irrespective of its ECN codepoint.
   However, where the DualQ is in a hierarchy of other PHBs, the
   classifier would classify some traffic into other PHBs based on DSCP
   before classifying between the latency and classic queues (based on
   ECT(1), CE and perhaps also the EF DSCP or other identifiers as in
   the above example).  [I-D.briscoe-tsvwg-l4s-diffserv] gives a number
   of examples of such arrangements to address various requirements.

   [I-D.briscoe-tsvwg-l4s-diffserv] describes how an operator might use
   L4S to offer low latency for all L4S traffic as well as using
   Diffserv for bandwidth differentiation.  It identifies two main types
   of approach, which can be combined: the operator might split certain
   Diffserv PHBs between L4S and a corresponding Classic service.  Or it
   might split the L4S and/or the Classic service into multiple Diffserv
   PHBs.  In any of these cases, a packet would have to be classified on
   its Diffserv and ECN codepoints.

   In summary, there are numerous ways in which the L4S ECN identifier
   (ECT(1) and CE) could be combined with other identifiers to achieve
   particular objectives.  The following categorization articulates
   those that are valid, but it is not necessarily exhaustive.  Those
   tagged 'Global-use' could be set by the sending host or a network.
   Those tagged 'Local-use' would only be set by a network:

De Schepper & Briscoe  Expires September 12, 2019              [Page 15]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   1.  Identifiers Complementing the L4S Identifier

       A.  Including More Traffic in the L Queue
           (Global-use or Local-use)

       B.  Excluding Certain Traffic from the L Queue
           (Local-use only)

   2.  Identifiers to place L4S classification in a PHB Hierarchy
       (Global-use or Local-use)

       A.  PHBs Before L4S ECN Classification

       B.  PHBs After L4S ECN Classification

6.  L4S Experiments

   [I-D.ietf-tsvwg-aqm-dualq-coupled] sets operational and management
   requirements for experiments with DualQ Coupled AQMs.  General
   operational and management requirements for experiments with L4S
   congestion controls are given in Section 4 and Section 5 above, e.g.
   co-existence and scaling requirements, incremental deployment
   arrangements.  The specification of each scalable congestion control
   will need to include protocol-specific requirements for configuration
   and monitoring performance during experiments.  Appendix A of
   [RFC5706] provides a helpful checklist.

7.  IANA Considerations

   This specification contains no IANA considerations.

8.  Security Considerations

   Approaches to assure the integrity of signals using the new identifer
   are introduced in Appendix C.1.

   The requirement to detect loss in time units prevents the ACK-
   splitting attacks described in [Savage-TCP].

9.  Acknowledgements

   Thanks to Richard Scheffenegger, John Leslie, David Taeht, Jonathan
   Morton, Gorry Fairhurst, Michael Welzl, Mikael Abrahamsson and Andrew
   McGregor for the discussions that led to this specification.  Ing-jyh
   (Inton) Tsang was a contributor to the early drafts of this document.
   And thanks to Mikael Abrahamsson, Lloyd Wood, Nicolas Kuhn, Greg
   White, David Black and Gorry Fairhurst for providing help and
   reviewing this draft and to Ingemar Johansson for reviewing and

De Schepper & Briscoe  Expires September 12, 2019              [Page 16]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   providing substantial text.  Appendix A listing the Prague L4S
   Requirements is based on text authored by Marcelo Bagnulo Braun that
   was originally an appendix to [I-D.ietf-tsvwg-l4s-arch].  That text
   was in turn based on the collective output of the attendees listed in
   the minutes of a 'bar BoF' on DCTCP Evolution during IETF-94
   [TCPPrague].

   The authors' contributions were part-funded by the European Community
   under its Seventh Framework Programme through the Reducing Internet
   Transport Latency (RITE) project (ICT-317700).  Bob Briscoe was also
   part-funded by the Research Council of Norway through the TimeIn
   project.  The views expressed here are solely those of the authors.

10.  References

10.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC3168]  Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
              of Explicit Congestion Notification (ECN) to IP",
              RFC 3168, DOI 10.17487/RFC3168, September 2001,
              <https://www.rfc-editor.org/info/rfc3168>.

   [RFC4774]  Floyd, S., "Specifying Alternate Semantics for the
              Explicit Congestion Notification (ECN) Field", BCP 124,
              RFC 4774, DOI 10.17487/RFC4774, November 2006,
              <https://www.rfc-editor.org/info/rfc4774>.

   [RFC6679]  Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P.,
              and K. Carlberg, "Explicit Congestion Notification (ECN)
              for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August
              2012, <https://www.rfc-editor.org/info/rfc6679>.

10.2.  Informative References

   [Alizadeh-stability]
              Alizadeh, M., Javanmard, A., and B. Prabhakar, "Analysis
              of DCTCP: Stability, Convergence, and Fairness", ACM
              SIGMETRICS 2011 , June 2011.

   [ARED01]   Floyd, S., Gummadi, R., and S. Shenker, "Adaptive RED: An
              Algorithm for Increasing the Robustness of RED's Active
              Queue Management", ACIRI Technical Report , August 2001,
              <http://www.icir.org/floyd/red.html>.

De Schepper & Briscoe  Expires September 12, 2019              [Page 17]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   [DCttH15]  De Schepper, K., Bondarenko, O., Briscoe, B., and I.
              Tsang, "'Data Centre to the Home': Ultra-Low Latency for
              All", RITE Project Technical Report , 2015,
              <http://riteproject.eu/publications/>.

   [I-D.briscoe-tsvwg-l4s-diffserv]
              Briscoe, B., "Interactions between Low Latency, Low Loss,
              Scalable Throughput (L4S) and Differentiated Services",
              draft-briscoe-tsvwg-l4s-diffserv-02 (work in progress),
              November 2018.

   [I-D.ietf-avtcore-cc-feedback-message]
              Sarker, Z., Perkins, C., Singh, V., and M. Ramalho, "RTP
              Control Protocol (RTCP) Feedback for Congestion Control",
              draft-ietf-avtcore-cc-feedback-message-03 (work in
              progress), December 2018.

   [I-D.ietf-quic-transport]
              Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed
              and Secure Transport", draft-ietf-quic-transport-18 (work
              in progress), January 2019.

   [I-D.ietf-tcpm-accurate-ecn]
              Briscoe, B., Kuehlewind, M., and R. Scheffenegger, "More
              Accurate ECN Feedback in TCP", draft-ietf-tcpm-accurate-
              ecn-08 (work in progress), March 2019.

   [I-D.ietf-tcpm-generalized-ecn]
              Bagnulo, M. and B. Briscoe, "ECN++: Adding Explicit
              Congestion Notification (ECN) to TCP Control Packets",
              draft-ietf-tcpm-generalized-ecn-03 (work in progress),
              October 2018.

   [I-D.ietf-tcpm-rack]
              Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "RACK:
              a time-based fast loss detection algorithm for TCP",
              draft-ietf-tcpm-rack-04 (work in progress), July 2018.

   [I-D.ietf-tsvwg-aqm-dualq-coupled]
              Schepper, K., Briscoe, B., Bondarenko, O., and I. Tsang,
              "DualQ Coupled AQMs for Low Latency, Low Loss and Scalable
              Throughput (L4S)", draft-ietf-tsvwg-aqm-dualq-coupled-08
              (work in progress), November 2018.

De Schepper & Briscoe  Expires September 12, 2019              [Page 18]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   [I-D.ietf-tsvwg-ecn-encap-guidelines]
              Briscoe, B., Kaippallimalil, J., and P. Thaler,
              "Guidelines for Adding Congestion Notification to
              Protocols that Encapsulate IP", draft-ietf-tsvwg-ecn-
              encap-guidelines-11 (work in progress), November 2018.

   [I-D.ietf-tsvwg-l4s-arch]
              Briscoe, B., Schepper, K., and M. Bagnulo, "Low Latency,
              Low Loss, Scalable Throughput (L4S) Internet Service:
              Architecture", draft-ietf-tsvwg-l4s-arch-03 (work in
              progress), October 2018.

   [I-D.sridharan-tcpm-ctcp]
              Sridharan, M., Tan, K., Bansal, D., and D. Thaler,
              "Compound TCP: A New TCP Congestion Control for High-Speed
              and Long Distance Networks", draft-sridharan-tcpm-ctcp-02
              (work in progress), November 2008.

   [I-D.stewart-tsvwg-sctpecn]
              Stewart, R., Tuexen, M., and X. Dong, "ECN for Stream
              Control Transmission Protocol (SCTP)", draft-stewart-
              tsvwg-sctpecn-05 (work in progress), January 2014.

   [Mathis09]
              Mathis, M., "Relentless Congestion Control", PFLDNeT'09 ,
              May 2009, <http://www.hpcc.jp/pfldnet2009/
              Program_files/1569198525.pdf>.

   [Paced-Chirping]
              Misund, J., "Rapid Acceleration in TCP Prague", Masters
              Thesis , May 2018,
              <https://riteproject.files.wordpress.com/2018/07/
              misundjoakimmastersthesissubmitted180515.pdf>.

   [QV]       Briscoe, B. and P. Hurtig, "Up to Speed with Queue View",
              RITE Technical Report D2.3; Appendix C.2, August 2015,
              <https://riteproject.files.wordpress.com/2015/12/
              rite-deliverable-2-3.pdf>.

   [RFC2309]  Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering,
              S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G.,
              Partridge, C., Peterson, L., Ramakrishnan, K., Shenker,
              S., Wroclawski, J., and L. Zhang, "Recommendations on
              Queue Management and Congestion Avoidance in the
              Internet", RFC 2309, DOI 10.17487/RFC2309, April 1998,
              <https://www.rfc-editor.org/info/rfc2309>.

De Schepper & Briscoe  Expires September 12, 2019              [Page 19]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   [RFC2474]  Nichols, K., Blake, S., Baker, F., and D. Black,
              "Definition of the Differentiated Services Field (DS
              Field) in the IPv4 and IPv6 Headers", RFC 2474,
              DOI 10.17487/RFC2474, December 1998,
              <https://www.rfc-editor.org/info/rfc2474>.

   [RFC2983]  Black, D., "Differentiated Services and Tunnels",
              RFC 2983, DOI 10.17487/RFC2983, October 2000,
              <https://www.rfc-editor.org/info/rfc2983>.

   [RFC3246]  Davie, B., Charny, A., Bennet, J., Benson, K., Le Boudec,
              J., Courtney, W., Davari, S., Firoiu, V., and D.
              Stiliadis, "An Expedited Forwarding PHB (Per-Hop
              Behavior)", RFC 3246, DOI 10.17487/RFC3246, March 2002,
              <https://www.rfc-editor.org/info/rfc3246>.

   [RFC3540]  Spring, N., Wetherall, D., and D. Ely, "Robust Explicit
              Congestion Notification (ECN) Signaling with Nonces",
              RFC 3540, DOI 10.17487/RFC3540, June 2003,
              <https://www.rfc-editor.org/info/rfc3540>.

   [RFC3649]  Floyd, S., "HighSpeed TCP for Large Congestion Windows",
              RFC 3649, DOI 10.17487/RFC3649, December 2003,
              <https://www.rfc-editor.org/info/rfc3649>.

   [RFC4340]  Kohler, E., Handley, M., and S. Floyd, "Datagram
              Congestion Control Protocol (DCCP)", RFC 4340,
              DOI 10.17487/RFC4340, March 2006,
              <https://www.rfc-editor.org/info/rfc4340>.

   [RFC4341]  Floyd, S. and E. Kohler, "Profile for Datagram Congestion
              Control Protocol (DCCP) Congestion Control ID 2: TCP-like
              Congestion Control", RFC 4341, DOI 10.17487/RFC4341, March
              2006, <https://www.rfc-editor.org/info/rfc4341>.

   [RFC4342]  Floyd, S., Kohler, E., and J. Padhye, "Profile for
              Datagram Congestion Control Protocol (DCCP) Congestion
              Control ID 3: TCP-Friendly Rate Control (TFRC)", RFC 4342,
              DOI 10.17487/RFC4342, March 2006,
              <https://www.rfc-editor.org/info/rfc4342>.

   [RFC4960]  Stewart, R., Ed., "Stream Control Transmission Protocol",
              RFC 4960, DOI 10.17487/RFC4960, September 2007,
              <https://www.rfc-editor.org/info/rfc4960>.

De Schepper & Briscoe  Expires September 12, 2019              [Page 20]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   [RFC5562]  Kuzmanovic, A., Mondal, A., Floyd, S., and K.
              Ramakrishnan, "Adding Explicit Congestion Notification
              (ECN) Capability to TCP's SYN/ACK Packets", RFC 5562,
              DOI 10.17487/RFC5562, June 2009,
              <https://www.rfc-editor.org/info/rfc5562>.

   [RFC5622]  Floyd, S. and E. Kohler, "Profile for Datagram Congestion
              Control Protocol (DCCP) Congestion ID 4: TCP-Friendly Rate
              Control for Small Packets (TFRC-SP)", RFC 5622,
              DOI 10.17487/RFC5622, August 2009,
              <https://www.rfc-editor.org/info/rfc5622>.

   [RFC5681]  Allman, M., Paxson, V., and E. Blanton, "TCP Congestion
              Control", RFC 5681, DOI 10.17487/RFC5681, September 2009,
              <https://www.rfc-editor.org/info/rfc5681>.

   [RFC5706]  Harrington, D., "Guidelines for Considering Operations and
              Management of New Protocols and Protocol Extensions",
              RFC 5706, DOI 10.17487/RFC5706, November 2009,
              <https://www.rfc-editor.org/info/rfc5706>.

   [RFC5925]  Touch, J., Mankin, A., and R. Bonica, "The TCP
              Authentication Option", RFC 5925, DOI 10.17487/RFC5925,
              June 2010, <https://www.rfc-editor.org/info/rfc5925>.

   [RFC6077]  Papadimitriou, D., Ed., Welzl, M., Scharf, M., and B.
              Briscoe, "Open Research Issues in Internet Congestion
              Control", RFC 6077, DOI 10.17487/RFC6077, February 2011,
              <https://www.rfc-editor.org/info/rfc6077>.

   [RFC6660]  Briscoe, B., Moncaster, T., and M. Menth, "Encoding Three
              Pre-Congestion Notification (PCN) States in the IP Header
              Using a Single Diffserv Codepoint (DSCP)", RFC 6660,
              DOI 10.17487/RFC6660, July 2012,
              <https://www.rfc-editor.org/info/rfc6660>.

   [RFC7560]  Kuehlewind, M., Ed., Scheffenegger, R., and B. Briscoe,
              "Problem Statement and Requirements for Increased Accuracy
              in Explicit Congestion Notification (ECN) Feedback",
              RFC 7560, DOI 10.17487/RFC7560, August 2015,
              <https://www.rfc-editor.org/info/rfc7560>.

   [RFC7567]  Baker, F., Ed. and G. Fairhurst, Ed., "IETF
              Recommendations Regarding Active Queue Management",
              BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015,
              <https://www.rfc-editor.org/info/rfc7567>.

De Schepper & Briscoe  Expires September 12, 2019              [Page 21]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   [RFC7713]  Mathis, M. and B. Briscoe, "Congestion Exposure (ConEx)
              Concepts, Abstract Mechanism, and Requirements", RFC 7713,
              DOI 10.17487/RFC7713, December 2015,
              <https://www.rfc-editor.org/info/rfc7713>.

   [RFC8033]  Pan, R., Natarajan, P., Baker, F., and G. White,
              "Proportional Integral Controller Enhanced (PIE): A
              Lightweight Control Scheme to Address the Bufferbloat
              Problem", RFC 8033, DOI 10.17487/RFC8033, February 2017,
              <https://www.rfc-editor.org/info/rfc8033>.

   [RFC8257]  Bensley, S., Thaler, D., Balasubramanian, P., Eggert, L.,
              and G. Judd, "Data Center TCP (DCTCP): TCP Congestion
              Control for Data Centers", RFC 8257, DOI 10.17487/RFC8257,
              October 2017, <https://www.rfc-editor.org/info/rfc8257>.

   [RFC8290]  Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys,
              J., and E. Dumazet, "The Flow Queue CoDel Packet Scheduler
              and Active Queue Management Algorithm", RFC 8290,
              DOI 10.17487/RFC8290, January 2018,
              <https://www.rfc-editor.org/info/rfc8290>.

   [RFC8298]  Johansson, I. and Z. Sarker, "Self-Clocked Rate Adaptation
              for Multimedia", RFC 8298, DOI 10.17487/RFC8298, December
              2017, <https://www.rfc-editor.org/info/rfc8298>.

   [RFC8311]  Black, D., "Relaxing Restrictions on Explicit Congestion
              Notification (ECN) Experimentation", RFC 8311,
              DOI 10.17487/RFC8311, January 2018,
              <https://www.rfc-editor.org/info/rfc8311>.

   [RFC8312]  Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and
              R. Scheffenegger, "CUBIC for Fast Long-Distance Networks",
              RFC 8312, DOI 10.17487/RFC8312, February 2018,
              <https://www.rfc-editor.org/info/rfc8312>.

   [Savage-TCP]
              Savage, S., Cardwell, N., Wetherall, D., and T. Anderson,
              "TCP Congestion Control with a Misbehaving Receiver", ACM
              SIGCOMM Computer Communication Review 29(5):71--78,
              October 1999.

   [TCP-CA]   Jacobson, V. and M. Karels, "Congestion Avoidance and
              Control", Laurence Berkeley Labs Technical Report ,
              November 1988, <http://ee.lbl.gov/papers/congavoid.pdf>.

De Schepper & Briscoe  Expires September 12, 2019              [Page 22]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   [TCP-sub-mss-w]
              Briscoe, B. and K. De Schepper, "Scaling TCP's Congestion
              Window for Small Round Trip Times", BT Technical Report
              TR-TUB8-2015-002, May 2015,
              <http://www.bobbriscoe.net/projects/latency/
              sub-mss-w.pdf>.

   [TCPPrague]
              Briscoe, B., "Notes: DCTCP evolution 'bar BoF': Tue 21 Jul
              2015, 17:40, Prague", tcpprague mailing list archive ,
              July 2015, <https://www.ietf.org/mail-
              archive/web/tcpprague/current/msg00001.html>.

   [VCP]      Xia, Y., Subramanian, L., Stoica, I., and S. Kalyanaraman,
              "One more bit is enough", Proc. SIGCOMM'05, ACM CCR
              35(4)37--48, 2005,
              <http://doi.acm.org/10.1145/1080091.1080098>.

Appendix A.  The 'Prague L4S Requirements'

   This appendix is informative, not normative.  It gives a list of
   modifications to current scalable congestion controls so that they
   can be deployed over the public Internet and coexist safely with
   existing traffic.  The list complements the normative requirements in
   Section 4 that a sender has to comply with before it can set the L4S
   identifier in packets it sends into the Internet.  As well as
   necessary safety improvements (requirements) this appendix also
   includes preferable performance improvements (optimizations).

   These recommendations have become know as the Prague L4S
   Requirements, because they were originally identified at an ad hoc
   meeting during IETF-94 in Prague [TCPPrague].  The wording has been
   generalized to apply to all scalable congestion controls, not just
   TCP congestion control specifically.  They were originally called the
   'TCP Prague Requirements', but they are not solely applicable to TCP,
   so the name has been generalized, and TCP Prague is now used for a
   specific implementation of the requirements.

   DCTCP [RFC8257] is currently the most widely used scalable transport
   protocol.  In its current form, DCTCP is specified to be deployable
   only in controlled environments.  Deploying it in the public Internet
   would lead to a number of issues, both from the safety and the
   performance perspective.  The modifications and additional mechanisms
   listed in this section will be necessary for its deployment over the
   global Internet.  Where an example is needed, DCTCP is used as a
   base, but it is likely that most of these requirements equally apply
   to other scalable congestion controls.

De Schepper & Briscoe  Expires September 12, 2019              [Page 23]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

A.1.  Requirements for Scalable Transport Protocols

A.1.1.  Use of L4S Packet Identifier

   Description: A scalable congestion control needs to distinguish the
   packets it sends from those sent by classic congestion controls.

   Motivation: It needs to be possible for a network node to classify
   L4S packets without flow state into a queue that applies an L4S ECN
   marking behaviour and isolates L4S packets from the queuing delay of
   classic packets.

A.1.2.  Accurate ECN Feedback

   Description: The transport protocol for a scalable congestion control
   needs to provide timely, accurate feedback about the extent of ECN
   marking experienced by all packets.

   Motivation: Classic congestion controls only need feedback about the
   existence of a congestion episode within a round trip, not precisely
   how many packets were marked with ECN or dropped.  Therefore, in
   2001, when ECN feedback was added to TCP [RFC3168], it could not
   inform the sender of more than one ECN mark per RTT.  Since then,
   requirements for more accurate ECN feedback in TCP have been defined
   in [RFC7560] and [I-D.ietf-tcpm-accurate-ecn] specifies an
   experimental change to the TCP wire protocol to satisfy these
   requirements.  Most other transport protocols already satisfy this
   requirement.

A.1.3.  Fall back to Reno-friendly congestion control on packet loss

   Description: A scalable congestion control needs to react to packet
   loss in a way that will coexist safely with a TCP Reno congestion
   control [RFC5681].

   Motivation: Part of the safety conditions for deploying a scalable
   congestion control on the public Internet is to make sure that it
   behaves properly when it builds a queue at a network bottleneck that
   has not been upgraded to support L4S.  Packet loss can have many
   causes, but it usually has to be conservatively assumed that it is a
   sign of congestion.  Therefore, on detecting packet loss, a scalable
   congestion control will need to fall back to classic congestion
   control behaviour.  If it does not comply with this requirement it
   could starve classic traffic.

   A scalable congestion control can be used for different types of
   transport, e.g. for real-time media or for reliable bulk transport
   like TCP.  Therefore, the particular classic congestion control

De Schepper & Briscoe  Expires September 12, 2019              [Page 24]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   behaviour to fall back on will need to be part of the congestion
   control specification of the relevant transport.  In the particular
   case of DCTCP, the current DCTCP specification states that "It is
   RECOMMENDED that an implementation deal with loss episodes in the
   same way as conventional TCP."  For safe deployment of a scalable
   congestion control in the public Internet, the above requirement
   would need to be defined as a "MUST".

   Packet loss might (rarely) occur in the case that the bottleneck is
   L4S capable.  In this case, the sender may receive a high number of
   packets marked with the CE bit set and also experience a loss.
   Current DCTCP implementations react differently to this situation.
   At least one implementation reacts only to the drop signal (e.g. by
   halving the CWND) and at least another DCTCP implementation reacts to
   both signals (e.g. by halving the CWND due to the drop and also
   further reducing the CWND based on the proportion of marked packet).
   We believe that further experimentation is needed to understand what
   is the best behaviour for the public Internet, which may or not be
   one of these existing approaches.

A.1.4.  Fall back to Reno-friendly congestion control on classic ECN
        bottlenecks

   Description: A scalable congestion control needs to react to ECN
   marking from a non-L4S but ECN-capable bottleneck in a way that will
   coexist with a TCP Reno congestion control [RFC5681].

   Motivation: Similarly to the requirement in Appendix A.1.3, this
   requirement is a safety condition to ensure a scalable congestion
   control behaves properly when it builds a queue at a network
   bottleneck that has not been upgraded to support L4S.  On detecting
   classic ECN marking (see below), a scalable congestion control will
   need to fall back to classic congestion control behaviour.  If it
   does not comply with this requirement it could starve classic
   traffic.

   It would take time for endpoints to distinguish classic and L4S ECN
   marking.  An increase in queuing delay or in delay variation would be
   a tell-tale sign, but it is not yet clear where a line would be drawn
   between the two behaviours.  It might be possible to cache what was
   learned about the path to help subsequent attempts to detect the type
   of marking.

A.1.5.  Reduce RTT dependence

   Description: A scalable congestion control needs to reduce or
   eliminate RTT bias over as wide a range of RTTs as possible, or at

De Schepper & Briscoe  Expires September 12, 2019              [Page 25]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   least over the typical range of RTTs that will interact in the
   intended deployment scenario.

   Motivation: Classic TCP's throughput is known to be inversely
   proportional to RTT, so one would expect flows over very low RTT
   paths to nearly starve flows over larger RTTs.  However, Classic TCP
   has never allowed a very low RTT path to exist because it induces a
   large queue.  For instance, consider two paths with base RTT 1ms and
   100ms.  If Classic TCP induces a 100ms queue, it turns these RTTs
   into 101ms and 200ms leading to a throughput ratio of about 2:1.
   Whereas if a Scalable TCP induces only a 1ms queue, the ratio is
   2:101, leading to a throughput ratio of about 50:1.

   Therefore, with very small queues, long RTT flows will essentially
   starve, unless scalable congestion controls comply with this
   requirement.

A.1.6.  Scaling down to fractional congestion windows

   Description: A scalable congestion control needs to remain responsive
   to congestion when RTTs are significantly smaller than in the current
   public Internet.

   Motivation: As currently specified, the minimum required congestion
   window of TCP (and its derivatives) is set to 2 maximum segment sizes
   (MSS) (see equation (4) in [RFC5681]).  Once the congestion window
   reaches this minimum, all current TCP algorithms become unresponsive
   to congestion signals.  No matter how much drop or ECN marking, the
   congestion window no longer reduces.  Instead, TCP forces the queue
   to grow, overriding any AQM and increasing queuing delay.

   L4S mechanisms significantly reduce queueing delay so, over the same
   path, the RTT becomes lower.  Then this problem becomes surprisingly
   common [TCP-sub-mss-w].  This is because, for the same link capacity,
   smaller RTT implies a smaller window.  For instance, consider a
   residential setting with an upstream broadband Internet access of 8
   Mb/s, assuming a max segment size of 1500 B.  Two upstream flows will
   each have the minimum window of 2 MSS if the RTT is 6ms or less,
   which is quite common when accessing a nearby data centre.  So, any
   more than two such parallel TCP flows will become unresponsive and
   increase queuing delay.

   Unless scalable congestion controls are required to comply with this
   requirement from the start, they will frequently become unresponsive,
   negating the low latency benefit of L4S, for themselves and for
   others.  One possible sub-MSS window mechanism is described in
   [TCP-sub-mss-w], and other approaches are likely to be feasible.

De Schepper & Briscoe  Expires September 12, 2019              [Page 26]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

A.1.7.  Measuring Reordering Tolerance in Time Units

   Description: A scalable congestion control needs to detect loss by
   counting in time-based units, which is scalable, rather than counting
   in units of packets, which is not.

   Motivation: If it is known that all L4S senders using a link obey
   this rule, then link technologies that support L4S can remove the
   head-of-line blocking delay they have to introduce while trying to
   keep packets in tight order to avoid triggering loss detection based
   on counting packets.

   End-systems cannot know whether a missing packet is due to loss or
   reordering, except in hindsight - if it appears later.  If senders
   deem that loss has occurred by counting reordered packets (e.g. the 3
   Duplicate ACK rule of Classic TCP), the time over which the network
   has to keep packets in order scales down as packet rates scale up
   over the years.  In contrast, if senders allow a reordering window in
   time-based units before they deem there has been a loss, the time
   over which the network has to keep packets in order stays constant.

   The potential benefit for links comes in two parts: i) switching the
   unit from packet count to time-based; ii) potentially relaxing the
   amount of time available for re-ordering.  The initial switch to
   time-based units offers no immediate benefit, but as the years
   progress it stops the reordering requirement becoming tighter.  The
   secondary relaxation might be possible where some transport protocols
   find they can tolerate more re-ordering (e.g. more than the 3 DupACK
   rule, perhaps because it was reasonable when packet rates were low,
   but it is now far too tight).

   Tolerance of reordering over a small duration could allow parallel
   (e.g. bonded-channel) link technologies to relax their need to
   deliver packets strictly in order.  Such links typically give
   arriving packets a link-level sequence number and introduce delay
   while buffering packets at the receiving end until they can be
   delivered in the same order.  For radio links, this delay usually
   includes the time allowed for link-layer retransmissions.

   Note, however, that a link technology will only be able to relax its
   ordering requirement if it is certain that it will not degrade the
   transport /most/ sensitive to reordering that might use the link.
   Also note that in some controlled environments no reordering is
   tolerated by some transports (e.g.  RoCEv2 ussed for RDMA), therefore
   a switch to time-based units could not be exploited to relax
   reordering.

De Schepper & Briscoe  Expires September 12, 2019              [Page 27]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   For receivers that need their packets in order, it would seem that
   relaxing network ordering would simply shift this reordering delay
   from the network to the receiver.  However, that is not true in the
   general case because links generally do not recognize transport layer
   flows and often cannot even see application layer streams within the
   flows (as in SCTP, HTTP/2 or QUIC).  So a link will often be holding
   back packets from one flow or stream while waiting for those from
   another.  Relaxing strict ordering in the network will remove this
   head-of-line blocking delay. {ToDo: this is being quantified
   experimentally - will need to add the figures here.}

   Classic TCP implementations are switching over to the time-based
   approach of RACK (Recent ACKnowledgements [I-D.ietf-tcpm-rack]).
   However, it will be many years (decades?) before networks no longer
   have to allow for the presence of traditional TCP senders still using
   the 3 DupACK rule.  This specification (Section 4.3) says that
   senders are not entitled to identify packets as L4S in the IP/ECN
   field unless they use the time-based approach.  Then networks that
   identify L4S traffic separately (e.g. using
   [I-D.ietf-tsvwg-aqm-dualq-coupled]) can know for certain that all L4S
   traffic is using the scalable time-based approach.

   This will allow networks to remove the head-of-line blocking delay of
   resequencing straight away, but only for L4S traffic.  Classic
   traffic will have to wait for many years until incremental deployment
   of RACK has become near-universal.  Nonetheless, experience with RACK
   will determine how much reordering tolerance networks will be
   reasonable for L4S traffic.

   Performance Optimization as well as Safety Improvement: The delay
   benefit would be lost if any L4S sender did not follow the time-based
   approach.  Therefore, the time-based approach is made a normative
   requirement (a necessary safety improvement).  Nonetheless, the time-
   based approach also enables a throughput benefit that a flow can
   enjoy independently of others (a performance optimization), explained
   next.

   Given the requirement for a scalable congestion control to fall-back
   to Reno or Cubic on a loss (see Appendix A.1.3), it is important that
   a scalable congestion control does not deem that a loss has occurred
   too soon.  If, later within the same round trip, an out-of-order
   acknowledgement fills the gap, the sender would have halved its rate
   spuriously (as well as retransmitting spuriously).  With a RACK-like
   approach, allowing longer before a loss is deemed to have occurred
   maintains higher throughput in the presence of reordering {ToDo:
   Quantify this statement}.

De Schepper & Briscoe  Expires September 12, 2019              [Page 28]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   On the other hand, it is also important not to wait too long before
   deeming that a gap is due to a loss (termed a long reordering
   window), otherwise loss recovery would be slow.

   The speed of loss recovery is much more significant for short flows
   than long, therefore a good compromise would adapt the reordering
   window; from a small fraction of the RTT at the start of a flow, to a
   larger fraction of the RTT for flows that continue for many round
   trips.  This is the approach adopted by TCP RACK (Recent
   ACKnowledgements) [I-D.ietf-tcpm-rack] and recommended for all L4S
   senders, whether using TCP or another transport protocol.

   The requirement to detect loss in time units also prevents the ACK-
   splitting attacks described in [Savage-TCP].

A.2.  Scalable Transport Protocol Optimizations

A.2.1.  Setting ECT in TCP Control Packets and Retransmissions

   Description: This item only concerns TCP and its derivatives (e.g.
   SCTP), because the original specification of ECN for TCP precluded
   the use of ECN on control packets and retransmissions.  To improve
   performance, scalable transport protocols ought to enable ECN at the
   IP layer in TCP control packets (SYN, SYN-ACK, pure ACKs, etc.) and
   in retransmitted packets.  The same is true for derivatives of TCP,
   e.g.  SCTP.

   Motivation: RFC 3168 prohibits the use of ECN on these types of TCP
   packet, based on a number of arguments.  This means these packets are
   not protected from congestion loss by ECN, which considerably harms
   performance, particularly for short flows.
   [I-D.ietf-tcpm-generalized-ecn] counters each argument in RFC 3168 in
   turn, showing it was over-cautious.  Instead it proposes experimental
   use of ECN on all types of TCP packet as long as AccECN feedback
   [I-D.ietf-tcpm-accurate-ecn] is available (which is itself a
   prerequisite for using a scalable congestion control).

A.2.2.  Faster than Additive Increase

   Description: It would improve performance if scalable congestion
   controls did not limit their congestion window increase to the
   traditional additive increase of 1 MSS per round trip [RFC5681]
   during congestion avoidance.  The same is true for derivatives of TCP
   congestion control, including similar approaches used for real-time
   media.

   Motivation: As currently defined, DCTCP uses the traditional TCP Reno
   additive increase in congestion avoidance phase.  When the available

De Schepper & Briscoe  Expires September 12, 2019              [Page 29]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   capacity suddenly increases (e.g. when another flow finishes, or if
   radio capacity increases) it can take very many round trips to take
   advantage of the new capacity.  In the steady state, DCTCP induces
   about 2 ECN marks per round trip, so it should be possible to quickly
   detect when these signals have disappeared and seek available
   capacity more rapidly.  It will of course be necessary to minimize
   the impact on other flows (classic and scalable).

   TCP Cubic was designed to solve this problem, but as flow rates have
   continued to increase, the delay accelerating into available capacity
   has become prohibitive.  For instance, with RTT=20 ms, to increase
   flow rate from 100Mb/s to 200Mb/s Cubic takes between 50 and 100
   round trips.  Every 8x increase in flow rate leads to 2x more
   acceleration delay.

A.2.3.  Faster Convergence at Flow Start

   Description: Particularly when a flow starts, scalable congestion
   controls need to converge (reach their steady-state share of the
   capacity) at least as fast as classic TCP and preferably faster.
   This does not just affect TCP Prague, but also the flow start
   behaviour of any L4S congestion control derived from a Classic
   transport that uses TCP slow start, including those for real-time
   media.

   Motivation: As an example, a new DCTCP flow takes longer than classic
   TCP to obtain its share of the capacity of the bottleneck when there
   are already ongoing flows using the bottleneck capacity.  In a data
   centre environment DCTCP takes about a factor of 1.5 to 2 longer to
   converge due to the much higher typical level of ECN marking that
   DCTCP background traffic induces, which causes new flows to exit slow
   start early [Alizadeh-stability].  In testing for use over the public
   Internet the convergence time of DCTCP relative to regular TCP is
   even less favourable [Paced-Chirping]).  It is exacerbated by the
   typically greater mismatch between the link rate of the sending host
   and typical Internet access bottlenecks, in combination with the
   shallow ECN marking threshold needed for L4S.  This problem is
   detrimental in general, but would particularly harm the performance
   of short flows relative to classic TCP.

Appendix B.  Alternative Identifiers

   This appendix is informative, not normative.  It records the pros and
   cons of various alternative ways to identify L4S packets to record
   the rationale for the choice of ECT(1) (Appendix B.1) as the L4S
   identifier.  At the end, Appendix B.6 summarises the distinguishing
   features of the leading alternatives.  It is intended to supplement,
   not replace the detailed text.

De Schepper & Briscoe  Expires September 12, 2019              [Page 30]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   The leading solutions all use the ECN field, sometimes in combination
   with the Diffserv field.  Both the ECN and Diffserv fields have the
   additional advantage that they are no different in either IPv4 or
   IPv6.  A couple of alternatives that use other fields are mentioned
   at the end, but it is quickly explained why they are not serious
   contenders.

B.1.  ECT(1) and CE codepoints

   Definition:

      Packets with ECT(1) and conditionally packets with CE would
      signify L4S semantics as an alternative to the semantics of
      classic ECN [RFC3168], specifically:

      *  The ECT(1) codepoint would signify that the packet was sent by
         an L4S-capable sender;

      *  Given shortage of codepoints, both L4S and classic ECN sides of
         an AQM would have to use the same CE codepoint to indicate that
         a packet had experienced congestion.  If a packet that had
         already been marked CE in an upstream buffer arrived at a
         subsequent AQM, this AQM would then have to guess whether to
         classify CE packets as L4S or classic ECN.  Choosing the L4S
         treatment would be a safer choice, because then a few classic
         packets might arrive early, rather than a few L4S packets
         arriving late;

      *  Additional information might be available if the classifier
         were transport-aware.  Then it could classify a CE packet for
         classic ECN treatment if the most recent ECT packet in the same
         flow had been marked ECT(0).  However, the L4S service ought
         not to need tranport-layer awareness;

   Cons:

   Consumes the last ECN codepoint:  The L4S service is intended to
      supersede the service provided by classic ECN, therefore using
      ECT(1) to identify L4S packets could ultimately mean that the
      ECT(0) codepoint was 'wasted' purely to distinguish one form of
      ECN from its successor;

   ECN hard in some lower layers:  It is not always possible to support
      ECN in an AQM acting in a buffer below the IP layer
      [I-D.ietf-tsvwg-ecn-encap-guidelines].  In such cases, the L4S
      service would have to drop rather than mark frames even though
      they might contain an ECN-capable packet.  However, such cases
      would be unusual.

De Schepper & Briscoe  Expires September 12, 2019              [Page 31]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   Risk of reordering classic CE packets:  Having to classify all CE
      packets as L4S risks some classic CE packets being wrongly
      classified as L4S and arriving early, which is a form of
      reordering.  Reordering can cause the TCP sender to retransmit
      spuriously.  However, the risk of spurious retransmissions would
      be extremely low, because:

      1.  it is quite unusual to experience more than one bottleneck
          queue on a path.

      2.  It would be even more unusual for the first bottleneck to
          support classic ECN marking and for the second to support L4S
          ECN marking

      3.  even then, reordering would only occur if there was
          simultaneous mixing of classic and L4S traffic, which would be
          more unlikely in an access link, which is where most
          bottlenecks are located;

      4.  even then, spurious retransmissions would only occur if a
          contiguous sequence of three or more packets in one classic
          ECN flow were all CE-marked at the first bottleneck;

      5.  even then, a spurious retransmission would only occur if the
          source did not support RACK [I-D.ietf-tcpm-rack], which is
          already widely supported.  Otherwise a whole reordering window
          within one classic ECN flow would have to be marked CE at the
          first bottleneck to cause a spurious retransmission.

      It is extremely unlikely that a set of 5 eventualities that are
      each unusual in themselves would all happen simultaneously.  But,
      even if they did, it would only cause spurious retransmission of a
      packet.

   Non-L4S service for control packets:  The classic ECN RFCs [RFC3168]
      and [RFC5562] require a sender to clear the ECN field to Not-ECT
      for retransmissions and certain control packets specifically pure
      ACKs, window probes and SYNs.  When L4S packets are classified by
      the ECN field alone, these control packets would not be classified
      into an L4S queue, and could therefore be delayed relative to the
      other packets in the flow.  This would not cause re-ordering
      (because retransmissions are already out of order, and the control
      packets carry no data).  However, it would make critical control
      packets more vulnerable to loss and delay.  To address this
      problem, [I-D.ietf-tcpm-generalized-ecn] proposes an experiment in
      which all TCP control packets and retransmissions are ECN-capable
      as long as ECN feedback is available.

De Schepper & Briscoe  Expires September 12, 2019              [Page 32]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   Pros:

   Should work e2e:  The ECN field generally works end-to-end across the
      Internet.  Unlike the DSCP, the setting of the ECN field is at
      least forwarded unchanged by networks that do not support ECN, and
      networks rarely clear it to zero;

   Should work in tunnels:  Unlike Diffserv, ECN is defined to always
      work across tunnels.  However, tunnels do not always implement ECN
      processing as they should do, particularly because IPsec tunnels
      were defined differently for a few years.

   Could migrate to one codepoint:  If all classic ECN senders
      eventually evolve to use the L4S service, the ECT(0) codepoint
      could be reused for some future purpose, but only once use of
      ECT(0) packets had reduced to zero, or near-zero, which might
      never happen.

B.2.  ECN Plus a Diffserv Codepoint (DSCP)

   Definition:

      For packets with a defined DSCP, all codepoints of the ECN field
      (except Not-ECT) would signify alternative L4S semantics to those
      for classic ECN [RFC3168], specifically:

      *  The L4S DSCP would signifiy that the packet came from an L4S-
         capable sender;

      *  ECT(0) and ECT(1) would both signify that the packet was
         travelling between transport endpoints that were both ECN-
         capable;

      *  CE would signify that the packet had been marked by an AQM
         implementing the L4S service.

   Use of a DSCP is the only approach for alternative ECN semantics
   given as an example in [RFC4774].  However, it was perhaps considered
   more for controlled environments than new end-to-end services;

   Cons:

   Consumes DSCP pairs:  A DSCP is obviously not orthogonal to Diffserv.
      Therefore, wherever the L4S service is applied to multiple
      Diffserv scheduling behaviours, it would be necessary to replace
      each DSCP with a pair of DSCPs.

De Schepper & Briscoe  Expires September 12, 2019              [Page 33]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   Uses critical lower-layer header space:  The resulting increased
      number of DSCPs might be hard to support for some lower layer
      technologies, e.g. 802.1p and MPLS both offer only 3-bits for a
      maximum of 8 traffic class identifiers.  Although L4S should
      reduce and possibly remove the need for some DSCPs intended for
      differentiated queuing delay, it will not remove the need for
      Diffserv entirely, because Diffserv is also used to allocate
      bandwidth, e.g. by prioritising some classes of traffic over
      others when traffic exceeds available capacity.

   Not end-to-end (host-network):  Very few networks honour a DSCP set
      by a host.  Typically a network will zero (bleach) the Diffserv
      field from all hosts.  Sometimes networks will attempt to identify
      applications by some form of packet inspection and, based on
      network policy, they will set the DSCP considered appropriate for
      the identified application.  Network-based application
      identification might use some combination of protocol ID, port
      numbers(s), application layer protocol headers, IP address(es),
      VLAN ID(s) and even packet timing.

   Not end-to-end (network-network):  Very few networks honour a DSCP
      received from a neighbouring network.  Typically a network will
      zero (bleach) the Diffserv field from all neighbouring networks at
      an interconnection point.  Sometimes bilateral arrangements are
      made between networks, such that the receiving network remarks
      some DSCPs to those it uses for roughly equivalent services.  The
      likelihood that a DSCP will be bleached or ignored depends on the
      type of DSCP:

      Local-use DSCP:  These tend to be used to implement application-
         specific network policies, but a bilateral arrangement to
         remark certain DSCPs is often applied to DSCPs in the local-use
         range simply because it is easier not to change all of a
         network's internal configurations when a new arrangement is
         made with a neighbour;

      Global-use DSCP:  These do not tend to be honoured across network
         interconnections more than local-use DSCPs.  However, if two
         networks decide to honour certain of each other's DSCPs, the
         reconfiguration is a little easier if both of their globally
         recognised services are already represented by the relevant
         global-use DSCPs.

         Note that today a global-use DSCP gives little more assurance
         of end-to-end service than a local-use DSCP.  In future the
         global-use range might give more assurance of end-to-end
         service than local-use, but it is unlikely that either

De Schepper & Briscoe  Expires September 12, 2019              [Page 34]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

         assurance will be high, particularly given the hosts are
         included in the end-to-end path.

   Not all tunnels:  Diffserv codepoints are often not propagated to the
      outer header when a packet is encapsulated by a tunnel header.
      DSCPs are propagated to the outer of uniform mode tunnels, but not
      pipe mode [RFC2983], and pipe mode is fairly common.

   ECN hard in some lower layers::  Because this approach uses both the
      Diffserv and ECN fields, an AQM wil only work at a lower layer if
      both can be supported.  If individual network operators wished to
      deploy an AQM at a lower layer, they would usually propagate an IP
      Diffserv codepoint to the lower layer, using for example IEEE
      802.1p.  However, the ECN capability is harder to propagate down
      to lower layers because few lower layers support it.

   Pros:

   Could migrate to e2e:  If all usage of classic ECN migrates to usage
      of L4S, the DSCP would become redundant, and the ECN capability
      alone could eventually identify L4S packets without the
      interconnection problems of Diffserv detailed above, and without
      having permanently consumed more than one codepoint in the IP
      header.  Although the DSCP does not generally function as an end-
      to-end identifier (see above), it could be used initially by
      individual ISPs to introduce the L4S service for their own locally
      generated traffic;

B.3.  ECN capability alone

   Definition:

      This approach uses ECN capability alone as the L4S identifier.  It
      is only feasible if classic ECN is not widely deployed.  The
      specific definition of codepoints would be:

      *  Any ECN codepoint other than Not-ECT would signify an L4S-
         capable sender;

      *  ECN codepoints would not be used for classic [RFC3168] ECN, and
         the classic network service would only be used for Not-ECT
         packets.

      This approach would only be feasible if

      A.  it was generally agreed that there was little chance of any
          classic [RFC3168] ECN deployment in any network nodes;

De Schepper & Briscoe  Expires September 12, 2019              [Page 35]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

      B.  it was generally agreed that there was little chance of any
          client devices being deployed with classic [RFC3168] TCP-ECN
          on by default (note that classic TCP-ECN is already on-by-
          default on many servers);

      C.  for TCP connections, developers of client OSs would all have
          to agree not to encourage further deployment of classic ECN.
          Specifically, at the start of a TCP connection classic ECN
          could be disabled during negotation of the ECN capability:

          +  an L4S-capable host would have to disable ECN if the
             corresponding host did not support accurate ECN feedback
             [RFC7560], which is a prerequisite for the L4S service;

          +  developers of operating systems for user devices would only
             enable ECN by default for TCP once the stack implemented
             L4S and accurate ECN feedback [RFC7560] including
             requesting accurate ECN feedback by default.

   Cons:

   Near-infeasible deployment constraints:  The constraints for
      deployment above represent a highly unlikely, but not completely
      impossible, set of circumstances.  If, despite the above measures,
      a pair of hosts did negotiate to use classic ECN, their packets
      would be classified into the same queue as L4S traffic, and if
      they had to compete with a long-running L4S flow they would get a
      very small capacity share;

   ECN hard in some lower layers:  See the same issue with "ECT(1) and
      CE codepoints" (Appendix B.1);

   Non-L4S service for control packets:  See the same issue with "ECT(1)
      and CE codepoints" (Appendix B.1).

   Pros:

   Consumes no additional codepoints:  The ECT(1) codepoint and all
      spare Diffserv codepoints would remain available for future use;

   Should work e2e:  As with "ECT(1) and CE codepoints" (Appendix B.1);

   Should work in tunnels:  As with "ECT(1) and CE codepoints"
      (Appendix B.1).

De Schepper & Briscoe  Expires September 12, 2019              [Page 36]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

B.4.  Protocol ID

   It has been suggested that a new ID in the IPv4 Protocol field or the
   IPv6 Next Header field could identify L4S packets.  However this
   approach is ruled out by numerous problems:

   o  A new protocol ID would need to be paired with the old one for
      each transport (TCP, SCTP, UDP, etc.);

   o  In IPv6, there can be a sequence of Next Header fields, and it
      would not be obvious which one would be expected to identify a
      network service like L4S;

   o  A new protocol ID would rarely provide an end-to-end service,
      because It is well-known that new protocol IDs are often blocked
      by numerous types of middlebox;

   o  The approach is not a solution for AQMs below the IP layer;

B.5.  Source or destination addressing

   Locally, a network operator could arrange for L4S service to be
   applied based on source or destination addressing, e.g. packets from
   its own data centre and/or CDN hosts, packets to its business
   customers, etc.  It could use addressing at any layer, e.g.  IP
   addresses, MAC addresses, VLAN IDs, etc.  Although addressing might
   be a useful tactical approach for a single ISP, it would not be a
   feasible approach to identify an end-to-end service like L4S.  Even
   for a single ISP, it would require packet classifiers in buffers to
   be dependent on changing topology and address allocation decisions
   elsewhere in the network.  Therefore this approach is not a feasible
   solution.

B.6.  Summary: Merits of Alternative Identifiers

   Table 1 provides a very high level summary of the pros and cons
   detailed against the schemes described respectively in Appendix B.2,
   Appendix B.3 and Appendix B.1, for six issues that set them apart.

De Schepper & Briscoe  Expires September 12, 2019              [Page 37]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   +--------------+--------------------+---------+--------------------+
   | Issue        |     DSCP + ECN     |   ECN   |    ECT(1) + CE     |
   +--------------+--------------------+---------+--------------------+
   |              | initial   eventual | initial | initial   eventual |
   |              |                    |         |                    |
   | end-to-end   |  N . .      . ? .  |  . . Y  |  . . Y      . . Y  |
   | tunnels      |  . O .      . O .  |  . . ?  |  . . ?      . . Y  |
   | lower layers |  N . .      . ? .  |  . O .  |  . O .      . . ?  |
   | codepoints   |  N . .      . . ?  |  . . Y  |  N . .      . . ?  |
   | reordering   |  . . Y      . . Y  |  . . Y  |  . O .      . . ?  |
   | ctrl pkts    |  . . Y      . . Y  |  . O .  |  . O .      . . ?  |
   |              |                    |         |                    |
   |              |                    |  Note 1 |                    |
   +--------------+--------------------+---------+--------------------+

             Note 1: Only feasible if classic ECN is obsolete.

    Table 1: Comparison of the Merits of Three Alternative Identifiers

   The schemes are scored based on both their capabilities now
   ('initial') and in the long term ('eventual').  The 'ECN' scheme
   shares the 'eventual' scores of the 'ECT(1) + CE' scheme.  The scores
   are one of 'N, O, Y', meaning 'Poor', 'Ordinary', 'Good'
   respectively.  The same scores are aligned vertically to aid the eye.
   A score of "?" in one of the positions means that this approach might
   optimisitically become this good, given sufficient effort.  The table
   summarises the text and is not meant to be understandable without
   having read the text.

Appendix C.  Potential Competing Uses for the ECT(1) Codepoint

   The ECT(1) codepoint of the ECN field has already been assigned once
   for the ECN nonce [RFC3540], which has now been categorized as
   historic [RFC8311].  ECN is probably the only remaining field in the
   Internet Protocol that is common to IPv4 and IPv6 and still has
   potential to work end-to-end, with tunnels and with lower layers.
   Therefore, ECT(1) should not be reassigned to a different
   experimental use (L4S) without carefully assessing competing
   potential uses.  These fall into the following categories:

C.1.  Integrity of Congestion Feedback

   Receiving hosts can fool a sender into downloading faster by
   suppressing feedback of ECN marks (or of losses if retransmissions
   are not necessary or available otherwise).

   The historic ECN nonce protocol [RFC3540] proposed that a TCP sender
   could set either of ECT(0) or ECT(1) in each packet of a flow and

De Schepper & Briscoe  Expires September 12, 2019              [Page 38]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   remember the sequence it had set.  If any packet was lost or
   congestion marked, the receiver would miss that bit of the sequence.
   An ECN Nonce receiver had to feed back the least significant bit of
   the sum, so it could not suppress feedback of a loss or mark without
   a 50-50 chance of guessing the sum incorrectly.

   It is highly unlikely that ECT(1) will be needed for integrity
   protection in future.  The ECN Nonce RFC [RFC3540] as been
   reclassified as historic, partly because other ways have been
   developed to protect TCP feedback integrity [RFC8311] that do not
   consume a codepoint in the IP header.  For instance:

   o  the sender can test the integrity of the receiver's feedback by
      occasionally setting the IP-ECN field to a value normally only set
      by the network.  Then it can test whether the receiver's feedback
      faithfully reports what it expects (see para 2 of Section 20.2 of
      [RFC3168].  This works for loss and it will work for the accurate
      ECN feedback [RFC7560] intended for L4S;

   o  A network can enforce a congestion response to its ECN markings
      (or packet losses) by auditing congestion exposure (ConEx)
      [RFC7713].  Whether the receiver or a downstream network is
      suppressing congestion feedback or the sender is unresponsive to
      the feedback, or both, ConEx audit can neutralise any advantage
      that any of these three parties would otherwise gain.

   o  The TCP authentication option (TCP-AO [RFC5925]) can be used to
      detect any tampering with TCP congestion feedback (whether
      malicious or accidental).  TCP's congestion feedback fields are
      immutable end-to-end, so they are amenable to TCP-AO protection,
      which covers the main TCP header and TCP options by default.
      However, TCP-AO is often too brittle to use on many end-to-end
      paths, where middleboxes can make verification fail in their
      attempts to improve performance or security, e.g. by
      resegmentation or shifting the sequence space.

C.2.  Notification of Less Severe Congestion than CE

   Various researchers have proposed to use ECT(1) as a less severe
   congestion notification than CE, particularly to enable flows to fill
   available capacity more quickly after an idle period, when another
   flow departs or when a flow starts, e.g.  VCP [VCP], Queue View (QV)
   [QV].

   Before assigning ECT(1) as an identifer for L4S, we must carefully
   consider whether it might be better to hold ECT(1) in reserve for
   future standardisation of rapid flow acceleration, which is an
   important and enduring problem [RFC6077].

De Schepper & Briscoe  Expires September 12, 2019              [Page 39]
Internet-Draft     ECN Semantics for Low Queuing Delay        March 2019

   Pre-Congestion Notification (PCN) is another scheme that assigns
   alternative semantics to the ECN field.  It uses ECT(1) to signify a
   less severe level of pre-congestion notification than CE [RFC6660].
   However, the ECN field only takes on the PCN semantics if packets
   carry a Diffserv codepoint defined to indicate PCN marking within a
   controlled environment.  PCN is required to be applied solely to the
   outer header of a tunnel across the controlled region in order not to
   interfere with any end-to-end use of the ECN field.  Therefore a PCN
   region on the path would not interfere with any of the L4S service
   identifiers proposed in Appendix B.

Authors' Addresses

   Koen De Schepper
   Nokia Bell Labs
   Antwerp
   Belgium

   Email: koen.de_schepper@nokia.com
   URI:   https://www.bell-labs.com/usr/koen.de_schepper

   Bob Briscoe (editor)
   CableLabs
   UK

   Email: ietf@bobbriscoe.net
   URI:   http://bobbriscoe.net/

De Schepper & Briscoe  Expires September 12, 2019              [Page 40]