Explicit Congestion Notification (ECN) Protocol for Very Low Queuing Delay (L4S)
draft-ietf-tsvwg-ecn-l4s-id-17
The information below is for an old version of the document.
| Document | Type | Active Internet-Draft (tsvwg WG) | |
|---|---|---|---|
| Authors | Koen De Schepper , Bob Briscoe | ||
| Last updated | 2021-05-21 | ||
| Replaces | draft-briscoe-tsvwg-ecn-l4s-id | ||
| Stream | Internet Engineering Task Force (IETF) | ||
| Formats | plain text xml htmlized pdfized bibtex | ||
| Stream | WG state | WG Document | |
| Associated WG milestone |
|
||
| Document shepherd | Wesley Eddy | ||
| IESG | IESG state | I-D Exists | |
| Consensus boilerplate | Unknown | ||
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | Wesley Eddy <wes@mti-systems.com> |
draft-ietf-tsvwg-ecn-l4s-id-17
Transport Services (tsv) K. De Schepper
Internet-Draft Nokia Bell Labs
Intended status: Experimental B. Briscoe, Ed.
Expires: November 22, 2021 Independent
May 21, 2021
Explicit Congestion Notification (ECN) Protocol for Very Low Queuing
Delay (L4S)
draft-ietf-tsvwg-ecn-l4s-id-17
Abstract
This specification defines the protocol to be used for a new network
service called low latency, low loss and scalable throughput (L4S).
L4S uses an Explicit Congestion Notification (ECN) scheme at the IP
layer that is similar to the original (or 'Classic') ECN approach,
except as specified within. L4S uses 'scalable' congestion control,
which induces much more frequent control signals from the network and
it responds to them with much more fine-grained adjustments, so that
very low (typically sub-millisecond on average) and consistently low
queuing delay becomes possible for L4S traffic without compromising
link utilization. Thus even capacity-seeking (TCP-like) traffic can
have high bandwidth and very low delay at the same time, even during
periods of high traffic load.
The L4S identifier defined in this document distinguishes L4S from
'Classic' (e.g. TCP-Reno-friendly) traffic. It gives an incremental
migration path so that suitably modified network bottlenecks can
distinguish and isolate existing traffic that still follows the
Classic behaviour, to prevent it degrading the low queuing delay and
low loss of L4S traffic. This specification defines the rules that
L4S transports and network elements need to follow with the intention
that L4S flows neither harm each other's performance nor that of
Classic traffic. Examples of new active queue management (AQM)
marking algorithms and examples of new transports (whether TCP-like
or real-time) are specified separately.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
De Schepper & Briscoe Expires November 22, 2021 [Page 1]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on November 22, 2021.
Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Latency, Loss and Scaling Problems . . . . . . . . . . . 4
1.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 7
1.3. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2. Choice of L4S Packet Identifier: Requirements . . . . . . . . 9
3. L4S Packet Identification . . . . . . . . . . . . . . . . . . 10
4. Transport Layer Behaviour (the 'Prague Requirements') . . . . 11
4.1. Codepoint Setting . . . . . . . . . . . . . . . . . . . . 11
4.2. Prerequisite Transport Feedback . . . . . . . . . . . . . 11
4.3. Prerequisite Congestion Response . . . . . . . . . . . . 12
4.4. Filtering or Smoothing of ECN Feedback . . . . . . . . . 15
5. Network Node Behaviour . . . . . . . . . . . . . . . . . . . 15
5.1. Classification and Re-Marking Behaviour . . . . . . . . . 15
5.2. The Strength of L4S CE Marking Relative to Drop . . . . . 17
5.3. Exception for L4S Packet Identification by Network Nodes
with Transport-Layer Awareness . . . . . . . . . . . . . 18
5.4. Interaction of the L4S Identifier with other Identifiers 18
5.4.1. DualQ Examples of Other Identifiers Complementing L4S
Identifiers . . . . . . . . . . . . . . . . . . . . . 18
5.4.1.1. Inclusion of Additional Traffic with L4S . . . . 18
5.4.1.2. Exclusion of Traffic From L4S Treatment . . . . . 20
5.4.1.3. Generalized Combination of L4S and Other
Identifiers . . . . . . . . . . . . . . . . . . . 21
5.4.2. Per-Flow Queuing Examples of Other Identifiers
De Schepper & Briscoe Expires November 22, 2021 [Page 2]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Complementing L4S Identifiers . . . . . . . . . . . . 22
5.5. Limiting Packet Bursts from Links Supporting L4S AQMs . . 22
6. Behaviour of Tunnels and Encapsulations . . . . . . . . . . . 23
6.1. No Change to ECN Tunnels and Encapsulations in General . 23
6.2. VPN Behaviour to Avoid Limitations of Anti-Replay . . . . 24
7. L4S Experiments . . . . . . . . . . . . . . . . . . . . . . . 25
7.1. Open Questions . . . . . . . . . . . . . . . . . . . . . 25
7.2. Open Issues . . . . . . . . . . . . . . . . . . . . . . . 26
7.3. Future Potential . . . . . . . . . . . . . . . . . . . . 27
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27
9. Security Considerations . . . . . . . . . . . . . . . . . . . 28
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 28
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 29
11.1. Normative References . . . . . . . . . . . . . . . . . . 29
11.2. Informative References . . . . . . . . . . . . . . . . . 30
Appendix A. The 'Prague L4S Requirements' . . . . . . . . . . . 38
A.1. Requirements for Scalable Transport Protocols . . . . . . 39
A.1.1. Use of L4S Packet Identifier . . . . . . . . . . . . 39
A.1.2. Accurate ECN Feedback . . . . . . . . . . . . . . . . 39
A.1.3. Capable of Replacement by Classic Congestion Control 39
A.1.4. Fall back to Classic Congestion Control on Packet
Loss . . . . . . . . . . . . . . . . . . . . . . . . 40
A.1.5. Coexistence with Classic Congestion Control at
Classic ECN bottlenecks . . . . . . . . . . . . . . . 41
A.1.6. Reduce RTT dependence . . . . . . . . . . . . . . . . 44
A.1.7. Scaling down to fractional congestion windows . . . . 44
A.1.8. Measuring Reordering Tolerance in Time Units . . . . 46
A.2. Scalable Transport Protocol Optimizations . . . . . . . . 48
A.2.1. Setting ECT in Control Packets and Retransmissions . 48
A.2.2. Faster than Additive Increase . . . . . . . . . . . . 49
A.2.3. Faster Convergence at Flow Start . . . . . . . . . . 49
Appendix B. Compromises in the Choice of L4S Identifier . . . . 50
Appendix C. Potential Competing Uses for the ECT(1) Codepoint . 55
C.1. Integrity of Congestion Feedback . . . . . . . . . . . . 55
C.2. Notification of Less Severe Congestion than CE . . . . . 56
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 56
1. Introduction
This specification defines the protocol to be used for a new network
service called low latency, low loss and scalable throughput (L4S).
L4S uses an Explicit Congestion Notification (ECN) scheme at the IP
layer that is similar to the original (or 'Classic') Explicit
Congestion Notification (ECN [RFC3168]). RFC 3168 required an ECN
mark to be equivalent to a drop, both when applied in the network and
when responded to by a transport. Unlike Classic ECN marking, the
network applies L4S marking more immediately and more aggressively
than drop, and the transport response to each mark is reduced and
De Schepper & Briscoe Expires November 22, 2021 [Page 3]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
smoothed relative to that for drop. The two changes counterbalance
each other so that the throughput of an L4S flow will be roughly the
same as a comparable non-L4S flow under the same conditions.
Nonetheless, the much more frequent control signals and the finer
responses to them result in very low queuing delay without
compromising link utilization, and this low delay can be maintained
during high load. Very low queuing delay means that queuing due to
the e2e congestion control is less than 1 millisecond (ms) on average
and less than about 2 ms at the 99th percentile.
L4S relies on 'scalable' congestion controls for these delay
properties and for preserving low delay as flow rate scales, hence
the name. The congestion control used in Data Center TCP (DCTCP) is
an example of a scalable congestion control, but DCTCP is applicable
solely to controlled environments like data centres [RFC8257],
because it is too aggressive to co-exist with existing TCP-Reno-
friendly traffic. The DualQ Coupled AQM, which is defined in a
complementary experimental specification
[I-D.ietf-tsvwg-aqm-dualq-coupled], is an AQM framework that enables
scalable congestion controls derived from DCTCP to co-exist with
existing traffic, each getting roughly the same flow rate when they
compete under similar conditions. Note that a scalable congestion
control is still not safe to deploy on the Internet unless it
satisfies the requirements listed in Section 4.
L4S is not only for elastic (TCP-like) traffic - there are scalable
congestion controls for real-time media, such as the L4S variant of
the SCReAM [RFC8298] real-time media congestion avoidance technique
(RMCAT). The factor that distinguishes L4S from Classic traffic is
its behaviour in response to congestion. The transport wire
protocol, e.g. TCP, QUIC, SCTP, DCCP, RTP/RTCP, is orthogonal (and
therefore not suitable for distinguishing L4S from Classic packets).
The L4S identifier defined in this document is the key piece that
distinguishes L4S from 'Classic' (e.g. Reno-friendly) traffic. It
gives an incremental migration path so that suitably modified network
bottlenecks can distinguish and isolate existing Classic traffic from
L4S traffic to prevent the former from degrading the very low delay
and loss of the new scalable transports, without harming Classic
performance at these bottlenecks. Initial implementation of the
separate parts of the system has been motivated by the performance
benefits.
1.1. Latency, Loss and Scaling Problems
Latency is becoming the critical performance factor for many (most?)
applications on the public Internet, e.g. interactive Web, Web
services, voice, conversational video, interactive video, interactive
De Schepper & Briscoe Expires November 22, 2021 [Page 4]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
remote presence, instant messaging, online gaming, remote desktop,
cloud-based applications, and video-assisted remote control of
machinery and industrial processes. In the 'developed' world,
further increases in access network bit-rate offer diminishing
returns, whereas latency is still a multi-faceted problem. In the
last decade or so, much has been done to reduce propagation time by
placing caches or servers closer to users. However, queuing remains
a major intermittent component of latency.
The Diffserv architecture provides Expedited Forwarding [RFC3246], so
that low latency traffic can jump the queue of other traffic. If
growth in high-throughput latency-sensitive applications continues,
periods with solely latency-sensitive traffic will become
increasingly common on links where traffic aggregation is low. For
instance, on the access links dedicated to individual sites (homes,
small enterprises or mobile devices). These links also tend to
become the path bottleneck under load. During these periods, if all
the traffic were marked for the same treatment at these bottlenecks,
Diffserv would make no difference. Instead, it becomes imperative to
remove the underlying causes of any unnecessary delay.
The bufferbloat project has shown that excessively-large buffering
('bufferbloat') has been introducing significantly more delay than
the underlying propagation time. These delays appear only
intermittently--only when a capacity-seeking (e.g. TCP) flow is long
enough for the queue to fill the buffer, making every packet in other
flows sharing the buffer sit through the queue.
Active queue management (AQM) was originally developed to solve this
problem (and others). Unlike Diffserv, which gives low latency to
some traffic at the expense of others, AQM controls latency for _all_
traffic in a class. In general, AQM methods introduce an increasing
level of discard from the buffer the longer the queue persists above
a shallow threshold. This gives sufficient signals to capacity-
seeking (aka. greedy) flows to keep the buffer empty for its intended
purpose: absorbing bursts. However, RED [RFC2309] and other
algorithms from the 1990s were sensitive to their configuration and
hard to set correctly. So, this form of AQM was not widely deployed.
More recent state-of-the-art AQM methods, e.g. FQ-CoDel [RFC8290],
PIE [RFC8033], Adaptive RED [ARED01], are easier to configure,
because they define the queuing threshold in time not bytes, so it is
invariant for different link rates. However, no matter how good the
AQM, the sawtoothing sending window of a Classic congestion control
will either cause queuing delay to vary or cause the link to be
under-utilized. Even with a perfectly tuned AQM, the additional
queuing delay will be of the same order as the underlying speed-of-
light delay across the network.
De Schepper & Briscoe Expires November 22, 2021 [Page 5]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
If a sender's own behaviour is introducing queuing delay variation,
no AQM in the network can 'un-vary' the delay without significantly
compromising link utilization. Even flow-queuing (e.g. [RFC8290]),
which isolates one flow from another, cannot isolate a flow from the
delay variations it inflicts on itself. Therefore those applications
that need to seek out high bandwidth but also need low latency will
have to migrate to scalable congestion control.
Altering host behaviour is not enough on its own though. Even if
hosts adopt low latency behaviour (scalable congestion controls),
they need to be isolated from the behaviour of existing Classic
congestion controls that induce large queue variations. L4S enables
that migration by providing latency isolation in the network and
distinguishing the two types of packets that need to be isolated: L4S
and Classic. L4S isolation can be achieved with a queue per flow
(e.g. [RFC8290]) but a DualQ [I-D.ietf-tsvwg-aqm-dualq-coupled] is
sufficient, and actually gives better tail latency. Both approaches
are addressed in this document.
The DualQ solution was developed to make very low latency available
without requiring per-flow queues at every bottleneck. This was
because FQ has well-known downsides - not least the need to inspect
transport layer headers in the network, which makes it incompatible
with privacy approaches such as IPSec VPN tunnels, and incompatible
with link layer queue management, where transport layer headers can
be hidden, e.g. 5G.
Latency is not the only concern addressed by L4S: It was known when
TCP congestion avoidance was first developed that it would not scale
to high bandwidth-delay products (footnote 6 of Jacobson and Karels
[TCP-CA]). Given regular broadband bit-rates over WAN distances are
already [RFC3649] beyond the scaling range of Reno congestion
control, 'less unscalable' Cubic [RFC8312] and
Compound [I-D.sridharan-tcpm-ctcp] variants of TCP have been
successfully deployed. However, these are now approaching their
scaling limits. Unfortunately, fully scalable congestion controls
such as DCTCP [RFC8257] outcompete Classic ECN congestion controls
sharing the same queue, which is why they have been confined to
private data centres or research testbeds.
It turns out that these scalable congestion control algorithms that
solve the latency problem can also solve the scalability problem of
Classic congestion controls. The finer sawteeth in the congestion
window have low amplitude, so they cause very little queuing delay
variation and the average time to recover from one congestion signal
to the next (the average duration of each sawtooth) remains
invariant, which maintains constant tight control as flow-rate
scales. A background paper [DCttH15] gives the full explanation of
De Schepper & Briscoe Expires November 22, 2021 [Page 6]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
why the design solves both the latency and the scaling problems, both
in plain English and in more precise mathematical form. The
explanation is summarised without the maths in the L4S architecture
document [I-D.ietf-tsvwg-l4s-arch].
1.2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in
[RFC2119]. In this document, these words will appear with that
interpretation only when in ALL CAPS. Lower case uses of these words
are not to be interpreted as carrying RFC-2119 significance.
Classic Congestion Control: A congestion control behaviour that can
co-exist with standard Reno [RFC5681] without causing
significantly negative impact on its flow rate [RFC5033]. With
Classic congestion controls, as flow rate scales, the number of
round trips between congestion signals (losses or ECN marks) rises
with the flow rate. So it takes longer and longer to recover
after each congestion event. Therefore control of queuing and
utilization becomes very slack, and the slightest disturbance
prevents a high rate from being attained [RFC3649].
For instance, with 1500 byte packets and an end-to-end round trip
time (RTT) of 36 ms, over the years, as Reno flow rate scales from
2 to 100 Mb/s the number of round trips taken to recover from a
congestion event rises proportionately, from 4 round trips to 200.
Cubic [RFC8312] was developed to be less unscalable, but it is
approaching its scaling limit; with the same RTT of 36ms, at
100Mb/s it takes about 106 round trips to recover, and at 800 Mb/s
its recovery time triples to over 340 round trips, or still more
than 12 seconds (Reno would take 57 seconds). Cubic only becomes
significantly better than Reno at high delay and rate
combinations, for example at 90 ms RTT and 800 Mb/s a Reno flow
takes 4000 RTTs or 6 minutes to recover, whereas Cubic 'only'
needs 188 RTTs, which is still 17 seconds (double its recovery
time at 100Mb/s).
Scalable Congestion Control: A congestion control where the average
time from one congestion signal to the next (the recovery time)
remains invariant as the flow rate scales, all other factors being
equal. This maintains the same degree of control over queueing
and utilization whatever the flow rate, as well as ensuring that
high throughput is robust to disturbances. For instance, DCTCP
averages 2 congestion signals per round-trip whatever the flow
rate, as do other recently developed scalable congestion controls,
e.g. Relentless TCP [Mathis09], TCP Prague
De Schepper & Briscoe Expires November 22, 2021 [Page 7]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
[I-D.briscoe-iccrg-prague-congestion-control], [PragueLinux] and
the L4S variant of SCREAM for real-time media [RFC8298]). See
Section 4.3 for more explanation.
Classic service: The Classic service is intended for all the
congestion control behaviours that co-exist with Reno [RFC5681]
(e.g. Reno itself, Cubic [RFC8312], Compound
[I-D.sridharan-tcpm-ctcp], TFRC [RFC5348]). The term 'Classic
queue' means a queue providing the Classic service.
Low-Latency, Low-Loss Scalable throughput (L4S) service: The 'L4S'
service is intended for traffic from scalable congestion control
algorithms, such as TCP Prague
[I-D.briscoe-iccrg-prague-congestion-control]. The L4S service is
for more general traffic than just TCP Prague--it allows the set
of congestion controls with similar scaling properties to Prague
to evolve, such as the examples listed above (Relentless, SCReAM).
The term 'L4S queue' means a queue providing the L4S service.
The terms Classic or L4S can also qualify other nouns, such as
'queue', 'codepoint', 'identifier', 'classification', 'packet',
'flow'. For example: an L4S packet means a packet with an L4S
identifier sent from an L4S congestion control.
Both Classic and L4S services can cope with a proportion of
unresponsive or less-responsive traffic as well, as long as it
does not build a queue (e.g. DNS, VoIP, game sync datagrams, etc).
Reno-friendly: The subset of Classic traffic that is friendly to the
standard Reno congestion control defined for TCP in [RFC5681].
Reno-friendly is used in place of 'TCP-friendly', given the latter
has become imprecise, because the TCP protocol is now used with so
many different congestion control behaviours, and Reno is used in
non-TCP transports such as QUIC.
Classic ECN: The original Explicit Congestion Notification (ECN)
protocol [RFC3168], which requires ECN signals to be treated the
same as drops, both when generated in the network and when
responded to by the sender. The names used for the four
codepoints of the 2-bit IP-ECN field are as defined in [RFC3168]:
Not ECT, ECT(0), ECT(1) and CE, where ECT stands for ECN-Capable
Transport and CE stands for Congestion Experienced. A packet
marked with the CE codepoint is termed 'ECN-marked' or sometimes
just 'marked' where the context makes ECN obvious.
De Schepper & Briscoe Expires November 22, 2021 [Page 8]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
1.3. Scope
The new L4S identifier defined in this specification is applicable
for IPv4 and IPv6 packets (as for Classic ECN [RFC3168]). It is
applicable for the unicast, multicast and anycast forwarding modes.
The L4S identifier is an orthogonal packet classification to the
Differentiated Services Code Point (DSCP) [RFC2474]. Section 5.4
explains what this means in practice.
This document is intended for experimental status, so it does not
update any standards track RFCs. Therefore it depends on [RFC8311],
which is a standards track specification that:
o updates the ECN proposed standard [RFC3168] to allow experimental
track RFCs to relax the requirement that an ECN mark must be
equivalent to a drop (when the network applies markings and/or
when the sender responds to them). For instance, in the ABE
experiment [RFC8511] this permits a sender to respond less to ECN
marks than to drops;
o changes the status of the experimental ECN nonce [RFC3540] to
historic;
o makes consequent updates to the following additional proposed
standard RFCs to reflect the above two bullets:
* ECN for RTP [RFC6679];
* the congestion control specifications of various DCCP
congestion control identifier (CCID) profiles [RFC4341],
[RFC4342], [RFC5622].
This document is about identifiers that are used for interoperation
between hosts and networks. So the audience is broad, covering
developers of host transports and network AQMs, as well as covering
how operators might wish to combine various identifiers, which would
require flexibility from equipment developers.
2. Choice of L4S Packet Identifier: Requirements
This subsection briefly records the process that led to the chosen
L4S identifier.
The identifier for packets using the Low Latency, Low Loss, Scalable
throughput (L4S) service needs to meet the following requirements:
De Schepper & Briscoe Expires November 22, 2021 [Page 9]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
o it SHOULD survive end-to-end between source and destination end-
points: across the boundary between host and network, between
interconnected networks, and through middleboxes;
o it SHOULD be visible at the IP layer;
o it SHOULD be common to IPv4 and IPv6 and transport-agnostic;
o it SHOULD be incrementally deployable;
o it SHOULD enable an AQM to classify packets encapsulated by outer
IP or lower-layer headers;
o it SHOULD consume minimal extra codepoints;
o it SHOULD be consistent on all the packets of a transport layer
flow, so that some packets of a flow are not served by a different
queue to others.
Whether the identifier would be recoverable if the experiment failed
is a factor that could be taken into account. However, this has not
been made a requirement, because that would favour schemes that would
be easier to fail, rather than those more likely to succeed.
It is recognised that any choice of identifier is unlikely to satisfy
all these requirements, particularly given the limited space left in
the IP header. Therefore a compromise will always be necessary,
which is why all the above requirements are expressed with the word
'SHOULD' not 'MUST'.
After extensive assessment of alternative schemes, "ECT(1) and CE
codepoints" was chosen as the best compromise. Therefore this scheme
is defined in detail in the following sections, while Appendix B
records its pros and cons against the above requirements.
3. L4S Packet Identification
The L4S treatment is an experimental track alternative packet marking
treatment [RFC4774] to the Classic ECN treatment in [RFC3168], which
has been updated by [RFC8311] to allow experiments such as the one
defined in the present specification. Like Classic ECN, L4S ECN
identifies both network and host behaviour: it identifies the marking
treatment that network nodes are expected to apply to L4S packets,
and it identifies packets that have been sent from hosts that are
expected to comply with a broad type of sending behaviour.
For a packet to receive L4S treatment as it is forwarded, the sender
sets the ECN field in the IP header to the ECT(1) codepoint. See
De Schepper & Briscoe Expires November 22, 2021 [Page 10]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Section 4 for full transport layer behaviour requirements, including
feedback and congestion response.
A network node that implements the L4S service always classifies
arriving ECT(1) packets for L4S treatment and by default classifies
CE packets for L4S treatment unless the heuristics described in
Section 5.3 are employed. See Section 5 for full network element
behaviour requirements, including classification, ECN-marking and
interaction of the L4S identifier with other identifiers and per-hop
behaviours.
4. Transport Layer Behaviour (the 'Prague Requirements')
4.1. Codepoint Setting
A sender that wishes a packet to receive L4S treatment as it is
forwarded, MUST set the ECN field in the IP header (v4 or v6) to the
ECT(1) codepoint.
4.2. Prerequisite Transport Feedback
For a transport protocol to provide scalable congestion control
(Section 4.3) it MUST provide feedback of the extent of CE marking on
the forward path. When ECN was added to TCP [RFC3168], the feedback
method reported no more than one CE mark per round trip. Some
transport protocols derived from TCP mimic this behaviour while
others report the accurate extent of ECN marking. This means that
some transport protocols will need to be updated as a prerequisite
for scalable congestion control. The position for a few well-known
transport protocols is given below.
TCP: Support for the accurate ECN feedback requirements [RFC7560]
(such as that provided by AccECN [I-D.ietf-tcpm-accurate-ecn]) by
both ends is a prerequisite for scalable congestion control in
TCP. Therefore, the presence of ECT(1) in the IP headers even in
one direction of a TCP connection will imply that both ends must
be supporting accurate ECN feedback. However, the converse does
not apply. So even if both ends support AccECN, either of the two
ends can choose not to use a scalable congestion control, whatever
the other end's choice.
SCTP: A suitable ECN feedback mechanism for SCTP could add a chunk
to report the number of received CE marks
(e.g. [I-D.stewart-tsvwg-sctpecn]), and update the ECN feedback
protocol sketched out in Appendix A of the standards track
specification of SCTP [RFC4960].
De Schepper & Briscoe Expires November 22, 2021 [Page 11]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
RTP over UDP: A prerequisite for scalable congestion control is for
both (all) ends of one media-level hop to signal ECN support
[RFC6679] and use the new generic RTCP feedback format of
[I-D.ietf-avtcore-cc-feedback-message]. The presence of ECT(1)
implies that both (all) ends of that media-level hop support ECN.
However, the converse does not apply. So each end of a media-
level hop can independently choose not to use a scalable
congestion control, even if both ends support ECN.
QUIC: Support for sufficiently fine-grained ECN feedback is provided
by the v1 IETF QUIC transport [I-D.ietf-quic-transport].
DCCP: The ACK vector in DCCP [RFC4340] is already sufficient to
report the extent of CE marking as needed by a scalable congestion
control.
4.3. Prerequisite Congestion Response
As a condition for a host to send packets with the L4S identifier
(ECT(1)), it SHOULD implement a congestion control behaviour that
ensures that, in steady state, the average duration between induced
ECN marks does not increase as flow rate scales up, all other factors
being equal. This is termed a scalable congestion control. This
invariant duration ensures that, as flow rate scales, the average
period with no feedback information about capacity does not become
excessive. It also ensures that queue variations remain small,
without having to sacrifice utilization.
With a congestion control that sawtooths to probe capacity, this
duration is called the recovery time, because each time the sawtooth
yields, on average it take this time to recover to its previous high
point. A scalable congestion control does not have to sawtooth, but
it has to coexist with scalable congestion controls that do.
For instance, for DCTCP [RFC8257], TCP Prague
[I-D.briscoe-iccrg-prague-congestion-control], [PragueLinux] and the
L4S variant of SCReAM [RFC8298], the average recovery time is always
half a round trip (or half a reference round trip), whatever the flow
rate.
As with all transport behaviours, a detailed specification (probably
an experimental RFC) is expected for each congestion control,
following the guidelines for specifying new congestion control
algorithms in [RFC5033]. In addition it is expected to document
these L4S-specific matters, specifically the timescale over which the
proportionality is averaged, and control of burstiness. The recovery
time requirement above is worded as a 'SHOULD' rather than a 'MUST'
to allow reasonable flexibility for such implementations.
De Schepper & Briscoe Expires November 22, 2021 [Page 12]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
The condition 'all other factors being equal', allows the recovery
time to be different for different round trip times, as long as it
does not increase with flow rate for any particular RTT.
Saying that the recovery time remains roughly invariant is equivalent
to saying that the number of ECN CE marks per round trip remains
invariant as flow rate scales, all other factors being equal. For
instance, an average recovery time of half of 1 RTT is equivalent to
2 ECN marks per round trip. For those familiar with steady-state
congestion response functions, it is also equivalent to say that the
congestion window is inversely proportional to the proportion of
bytes in packets marked with the CE codepoint (see section 2 of
[PI2]).
In order to coexist safely with other Internet traffic, a scalable
congestion control MUST NOT tag its packets with the ECT(1) codepoint
unless it complies with the following bulleted requirements:
o A scalable congestion control MUST be capable of being replaced by
a Classic congestion control (by application and/or by
administrative control). If a Classic congestion control is
activated, it will not tag its packets with the ECT(1) codepoint
(see Appendix A.1.3 for rationale).
o As well as responding to ECN markings, a scalable congestion
control MUST react to packet loss in a way that will coexist
safely with Classic congestion controls such as standard Reno
[RFC5681], as required by [RFC5033] (see Appendix A.1.4 for
rationale).
o In uncontrolled environments, monitoring MUST be implemented to
support detection of problems with an ECN-capable AQM at the path
bottleneck that appears not to support L4S and might be in a
shared queue. Such monitoring SHOULD be applied to live traffic
that is using Scalable congestion control. Alternatively,
monitoring need not be applied to live traffic, if monitoring has
been arranged to cover the paths that live traffic takes through
uncontrolled environments.
The detection function SHOULD be capable of making the congestion
control adapt its ECN-marking response to coexist safely with
Classic congestion controls such as standard Reno [RFC5681], as
required by [RFC5033]. Alternatively, if adaptation is not
implemented and problems with such an AQM are detected, the
scalable congestion control MUST be replaced by a Classic
congestion control.
De Schepper & Briscoe Expires November 22, 2021 [Page 13]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Note that a scalable congestion control is not expected to change
to setting ECT(0) while it transiently adapts to coexist with
Classic congestion controls.
See Appendix A.1.5 and [I-D.ietf-tsvwg-l4sops] for rationale.
o In the range between the minimum likely RTT and typical RTTs
expected in the intended deployment scenario, a scalable
congestion control MUST converge towards a rate that is as
independent of RTT as is possible without compromising stability
or efficiency (see Appendix A.1.6 for rationale).
o A scalable congestion control SHOULD remain responsive to
congestion when typical RTTs over the public Internet are
significantly smaller because they are no longer inflated by
queuing delay. It would be preferable for the minimum window of a
scalable congestion control to be lower than 1 segment rather than
use the timeout approach described for TCP in S.6.1.2 of [RFC3168]
(or an equivalent for other transports). However, a lower minimum
is not set as a formal requirement for L4S experiments (see
Appendix A.1.7 for rationale).
o A scalable congestion control's loss detection SHOULD be resilient
to reordering over an adaptive time interval that scales with
throughput and adapts to reordering (as in [RFC8985]), as opposed
to counting only in fixed units of packets (as in the 3 DupACK
rule of [RFC5681] and [RFC6675], which is not scalable). As data
rates increase (e.g., due to new and/or improved technology),
congestion controls that detect loss by counting in units of
packets become more likely to incorrectly treat reordering events
as congestion-caused loss events (see Appendix A.1.8 for further
rationale). This requirement does not apply to congestion
controls that are solely used in controlled environments where the
network introduces hardly any reordering.
o A scalable congestion control is expected to limit the queue
caused by bursts of packets. It would not seem necessary to set
the limit any lower than 10% of the minimum RTT expected in a
typical deployment (e.g. additional queuing of roughly 250 us for
the public Internet). This would be converted to a number of
packets under the worst-case assumption that the bottleneck link
capacity equals the current flow rate. No normative requirement
to limit bursts is given here and, until there is more industry
experience from the L4S experiment, it is not even known whether
one is needed - it seems to be in an L4S sender's self-interest to
limit bursts.
De Schepper & Briscoe Expires November 22, 2021 [Page 14]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Each sender in a session can use a scalable congestion control
independently of the congestion control used by the receiver(s) when
they send data. Therefore there might be ECT(1) packets in one
direction and ECT(0) or Not-ECT in the other.
Later (Section 5.4.1.1) this document discusses the conditions for
mixing other "'Safe' Unresponsive Traffic" (e.g. DNS, LDAP, NTP,
voice, game sync packets) with L4S traffic. To be clear, although
such traffic can share the same queue as L4S traffic, it is not
appropriate for the sender to tag it as ECT(1), except in the
(unlikely) case that it satisfies the above conditions.
4.4. Filtering or Smoothing of ECN Feedback
Section 5.2 below specifies that an L4S AQM is expected to signal L4S
ECN without filtering or smoothing. This contrasts with a Classic
AQM, which filters out variations in the queue before signalling ECN
marking or drop. In the L4S architecture [I-D.ietf-tsvwg-l4s-arch],
responsibility for smoothing out these variations shifts to the
sender's congestion control.
This shift of responsibility has the advantage that each sender can
smooth variations over a timescale proportionate to its own RTT.
Whereas, in the Classic approach, the network doesn't know the RTTs
of all the flows, so it has to smooth out variations for a worst-case
RTT to ensure stability. For all the typical flows with shorter RTT
than the worst-case, this makes congestion control unnecessarily
sluggish.
This also gives an L4S sender the choice not to smooth, depending on
its context (start-up, congestion avoidance, etc). Therefore, this
document places no requirement on an L4S congestion control to smooth
out variations in any particular way. Implementers are encouraged to
openly publish the approach they take to smoothing, and the results
and experience they gain during the L4S experiment.
5. Network Node Behaviour
5.1. Classification and Re-Marking Behaviour
A network node that implements the L4S service:
o MUST classify arriving ECT(1) packets for L4S treatment, unless
overridden by another classifier (e.g., see Section 5.4.1.2);
o MUST classify arriving CE packets for L4S treatment as well,
unless overridden by a another classifier or unless the exception
referred to next applies;
De Schepper & Briscoe Expires November 22, 2021 [Page 15]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
CE packets might have originated as ECT(1) or ECT(0), but the
above rule to classify them as if they originated as ECT(1) is the
safe choice (see Appendix B for rationale). The exception is
where some flow-aware in-network mechanism happens to be available
for distinguishing CE packets that originated as ECT(0), as
described in Section 5.3, but there is no implication that such a
mechanism is necessary.
An L4S AQM treatment follows similar codepoint transition rules to
those in RFC 3168. Specifically, the ECT(1) codepoint MUST NOT be
changed to any other codepoint than CE, and CE MUST NOT be changed to
any other codepoint. An ECT(1) packet is classified as ECN-capable
and, if congestion increases, an L4S AQM algorithm will increasingly
mark the ECN field as CE, otherwise forwarding packets unchanged as
ECT(1). Necessary conditions for an L4S marking treatment are
defined in Section 5.2.
Under persistent overload an L4S marking treatment MUST begin
applying drop to L4S traffic until the overload episode has subsided,
as recommended for all AQM methods in [RFC7567] (Section 4.2.1),
which follows the similar advice in RFC 3168 (Section 7). During
overload, it MUST apply the same drop probability to L4S traffic as
it would to Classic traffic.
Where an L4S AQM is transport-aware, this requirement could be
satisfied by using drop in only the most overloaded individual per-
flow AQMs. In a DualQ with flow-aware queue protection (e.g.
[I-D.briscoe-docsis-q-protection]), this could be achieved by
redirecting packets in those flows contributing most to the overload
out of the L4S queue so that they are subjected to drop in the
Classic queue.
For backward compatibility in uncontrolled environments, a network
node that implements the L4S treatment MUST also implement an AQM
treatment for the Classic service as defined in Section 1.2. This
Classic AQM treatment need not mark ECT(0) packets, but if it does,
see Section 5.2 for the strengths of the markings relative to drop.
It MUST classify arriving ECT(0) and Not-ECT packets for treatment by
this Classic AQM (for the DualQ Coupled AQM, see the extensive
discussion on classification in Sections 2.3 and 2.5.1.1 of
[I-D.ietf-tsvwg-aqm-dualq-coupled]).
In case unforeseen problems arise with the L4S experiment, it MUST be
possible to configure an L4S implementation to disable the L4S
treatment. Once disabled, all packets of all ECN codepoints will
receive Classic treatment and ECT(1) packets MUST be treated as if
they were {ToDo: Not-ECT / ECT(0) ?}.
De Schepper & Briscoe Expires November 22, 2021 [Page 16]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
5.2. The Strength of L4S CE Marking Relative to Drop
The relative strengths of L4S CE and drop are irrelevant where AQMs
are implemented in separate queues per-application-flow, which are
then explicitly scheduled (e.g. with an FQ scheduler as in
[RFC8290]). Nonetheless, the relationship between them needs to be
defined for the coupling between L4S and Classic congestion signals
in a DualQ Coupled AQM [I-D.ietf-tsvwg-aqm-dualq-coupled], as below.
Unless an AQM node schedules application flows explicitly, the
likelihood that the AQM drops a Not-ECT Classic packet (p_C) MUST be
roughly proportional to the square of the likelihood that it would
have marked it if it had been an L4S packet (p_L). That is
p_C ~= (p_L / k)^2
The constant of proportionality (k) does not have to be standardised
for interoperability, but a value of 2 is RECOMMENDED. The term
'likelihood' is used above to allow for marking and dropping to be
either probabilistic or deterministic.
This formula ensures that Scalable and Classic flows will converge to
roughly equal congestion windows, for the worst case of Reno
congestion control. This is because the congestion windows of
Scalable and Classic congestion controls are inversely proportional
to p_L and sqrt(p_C) respectively. So squaring p_C in the above
formula counterbalances the square root that characterizes Reno-
friendly flows.
Note that, contrary to RFC 3168, an AQM implementing the L4S and
Classic treatments does not mark an ECT(1) packet under the same
conditions that it would have dropped a Not-ECT packet, as allowed by
[RFC8311], which updates RFC 3168. However, if it marks ECT(0)
packets, it does so under the same conditions that it would have
dropped a Not-ECT packet [RFC3168].
Also, L4S CE marking needs to be interpreted as an unsmoothed signal,
in contrast to the Classic approach in which AQMs filter out
variations before signalling congestion. An L4S AQM SHOULD NOT
smooth or filter out variations in the queue before signalling
congestion. In the L4S architecture [I-D.ietf-tsvwg-l4s-arch], the
sender, not the network, is responsible for smoothing out variations.
This requirement is worded as 'SHOULD NOT' rather than 'MUST NOT' to
allow for the case where the signals from a Classic smoothed AQM are
coupled with those from an unsmoothed L4S AQM. Nonetheless, the
spirit of the requirement is for all systems to expect that L4S ECN
De Schepper & Briscoe Expires November 22, 2021 [Page 17]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
signalling is unsmoothed and unfiltered, which is important for
interoperability.
5.3. Exception for L4S Packet Identification by Network Nodes with
Transport-Layer Awareness
To implement the L4S treatment, a network node does not need to
identify transport-layer flows. Nonetheless, if an implementer is
willing to identify transport-layer flows at a network node, and if
the most recent ECT packet in the same flow was ECT(0), the node MAY
classify CE packets for Classic ECN [RFC3168] treatment. In all
other cases, a network node MUST classify all CE packets for L4S
treatment. Examples of such other cases are: i) if no ECT packets
have yet been identified in a flow; ii) if it is not desirable for a
network node to identify transport-layer flows; or iii) if the most
recent ECT packet in a flow was ECT(1).
If an implementer uses flow-awareness to classify CE packets, to
determine whether the flow is using ECT(0) or ECT(1) it only uses the
most recent ECT packet of a flow (this advice will need to be
verified as part of L4S experiments). This is because a sender might
switch from sending ECT(1) (L4S) packets to sending ECT(0) (Classic
ECN) packets, or back again, in the middle of a transport-layer flow
(e.g. it might manually switch its congestion control module mid-
connection, or it might be deliberately attempting to confuse the
network).
5.4. Interaction of the L4S Identifier with other Identifiers
The examples in this section concern how additional identifiers might
complement the L4S identifier to classify packets between class-based
queues. Firstly Section 5.4.1 considers two queues, L4S and Classic,
as in the Coupled DualQ AQM [I-D.ietf-tsvwg-aqm-dualq-coupled],
either alone (Section 5.4.1.1) or within a larger queuing hierarchy
(Section 5.4.1.2). Then Section 5.4.2 considers schemes that might
combine per-flow 5-tuples with other identifiers.
5.4.1. DualQ Examples of Other Identifiers Complementing L4S
Identifiers
5.4.1.1. Inclusion of Additional Traffic with L4S
In a typical case for the public Internet a network element that
implements L4S in a shared queue might want to classify some low-rate
but unresponsive traffic (e.g. DNS, LDAP, NTP, voice, game sync
packets) into the low latency queue to mix with L4S traffic.
De Schepper & Briscoe Expires November 22, 2021 [Page 18]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
In this case it would not be appropriate to call the queue an L4S
queue, because it is shared by L4S and non-L4S traffic. Instead it
will be called the low latency or L queue. The L queue then offers
two different treatments:
o The L4S treatment, which is a combination of the L4S AQM treatment
and a priority scheduling treatment;
o The low latency treatment, which is solely the priority scheduling
treatment, without ECN-marking by the AQM.
To identify packets for just the scheduling treatment, it would be
inappropriate to use the L4S ECT(1) identifier, because such traffic
is unresponsive to ECN marking. Examples of relevant non-ECN
identifiers are:
o address ranges of specific applications or hosts configured to be,
or known to be, safe, e.g. hard-coded IoT devices sending low
intensity traffic;
o certain low data-volume applications or protocols (e.g. ARP, DNS);
o specific Diffserv codepoints that indicate traffic with limited
burstiness such as the EF (Expedited Forwarding [RFC3246]), Voice-
Admit [RFC5865] or proposed NQB (Non-Queue-Building
[I-D.ietf-tsvwg-nqb]) service classes or equivalent local-use
DSCPs (see [I-D.briscoe-tsvwg-l4s-diffserv]).
In summary, a network element that implements L4S in a shared queue
MAY classify additional types of packets into the L queue based on
identifiers other than the ECN field, but the types SHOULD be 'safe'
to mix with L4S traffic, where 'safe' is explained in
Section 5.4.1.1.1.
A packet that carries one of these non-ECN identifiers to classify it
into the L queue would not be subject to the L4S ECN marking
treatment, unless it also carried an ECT(1) or CE codepoint. The
specification of an L4S AQM MUST define the behaviour for packets
with unexpected combinations of codepoints, e.g. a non-ECN-based
classifier for the L queue, but ECT(0) in the ECN field (for examples
see section 2.5.1.1 of [I-D.ietf-tsvwg-aqm-dualq-coupled]).
For clarity, non-ECN identifiers, such as the examples itemized
above, might be used by some network operators who believe they
identify non-L4S traffic that would be safe to mix with L4S traffic.
They are not alternative ways for a host to indicate that it is
sending L4S packets. Only the ECT(1) ECN codepoint indicates to a
network element that a host is sending L4S packets (and CE indicates
De Schepper & Briscoe Expires November 22, 2021 [Page 19]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
that it could have originated as ECT(1)). Specifically ECT(1)
indicates that the host claims its behaviour satisfies the
prerequisite transport requirements in Section 4.
To include additional traffic with L4S, a network element only reads
identifiers such as those itemized above. It MUST NOT alter these
non-ECN identifiers, so that they survive for any potential use later
on the network path.
5.4.1.1.1. 'Safe' Unresponsive Traffic
The above section requires unresponsive traffic to be 'safe' to mix
with L4S traffic. Ideally this means that the sender never sends any
sequence of packets at a rate that exceeds the available capacity of
the bottleneck link. However, typically an unresponsive transport
does not even know the bottleneck capacity of the path, let alone its
available capacity. Nonetheless, an application can be considered
safe enough if it paces packets out (not necessarily completely
regularly) such that its maximum instantaneous rate from packet to
packet stays well below a typical broadband access rate.
This is a vague but useful definition, because many low latency
applications of interest, such as DNS, voice, game sync packets, RPC,
ACKs, keep-alives, could match this description.
5.4.1.2. Exclusion of Traffic From L4S Treatment
To extend the above example, an operator might want to exclude some
traffic from the L4S treatment for a policy reason, e.g. security
(traffic from malicious sources) or commercial (e.g. initially the
operator may wish to confine the benefits of L4S to business
customers).
In this exclusion case, the operator MUST classify on the relevant
locally-used identifiers (e.g. source addresses) before classifying
the non-matching traffic on the end-to-end L4S ECN identifier.
The operator MUST NOT alter the end-to-end L4S ECN identifier from
L4S to Classic, because an operator decision to exclude certain
traffic from L4S treatment is local-only. The end-to-end L4S
identifier then survives for other operators to use, or indeed, they
can apply their own policy, independently based on their own choice
of locally-used identifiers. This approach also allows any operator
to remove its locally-applied exclusions in future, e.g. if it wishes
to widen the benefit of the L4S treatment to all its customers.
De Schepper & Briscoe Expires November 22, 2021 [Page 20]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
An operator that excludes traffic carrying the L4S identifier from
L4S treatment MUST NOT treat such traffic as if it carries the ECT(0)
codepoint, which could confuse the sender.
5.4.1.3. Generalized Combination of L4S and Other Identifiers
L4S concerns low latency, which it can provide for all traffic
without differentiation and without _necessarily_ affecting bandwidth
allocation. Diffserv provides for differentiation of both bandwidth
and low latency, but its control of latency depends on its control of
bandwidth. The two can be combined if a network operator wants to
control bandwidth allocation but it also wants to provide low latency
- for any amount of traffic within one of these allocations of
bandwidth (rather than only providing low latency by limiting
bandwidth) [I-D.briscoe-tsvwg-l4s-diffserv].
The DualQ examples so far have been framed in the context of
providing the default Best Efforts Per-Hop Behaviour (PHB) using two
queues - a Low Latency (L) queue and a Classic (C) Queue. This
single DualQ structure is expected to be the most common and useful
arrangement. But, more generally, an operator might choose to
control bandwidth allocation through a hierarchy of Diffserv PHBs at
a node, and to offer one (or more) of these PHBs with a low latency
and a Classic variant.
In the first case, if we assume that a network element provides no
PHBs except the DualQ, if a packet carries ECT(1) or CE, the network
element would classify it for the L4S treatment irrespective of its
DSCP. And, if a packet carried (say) the EF DSCP, the network
element could classify it into the L queue irrespective of its ECN
codepoint. However, where the DualQ is in a hierarchy of other PHBs,
the classifier would classify some traffic into other PHBs based on
DSCP before classifying between the low latency and Classic queues
(based on ECT(1), CE and perhaps also the EF DSCP or other
identifiers as in the above example).
[I-D.briscoe-tsvwg-l4s-diffserv] gives a number of examples of such
arrangements to address various requirements.
[I-D.briscoe-tsvwg-l4s-diffserv] describes how an operator might use
L4S to offer low latency for all L4S traffic as well as using
Diffserv for bandwidth differentiation. It identifies two main types
of approach, which can be combined: the operator might split certain
Diffserv PHBs between L4S and a corresponding Classic service. Or it
might split the L4S and/or the Classic service into multiple Diffserv
PHBs. In either of these cases, a packet would have to be classified
on its Diffserv and ECN codepoints.
De Schepper & Briscoe Expires November 22, 2021 [Page 21]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
In summary, there are numerous ways in which the L4S ECN identifier
(ECT(1) and CE) could be combined with other identifiers to achieve
particular objectives. The following categorization articulates
those that are valid, but it is not necessarily exhaustive. Those
tagged 'Recommended-standard-use' could be set by the sending host or
a network. Those tagged 'Local-use' would only be set by a network:
1. Identifiers Complementing the L4S Identifier
A. Including More Traffic in the L Queue
(Could use Recommended-standard-use or Local-use identifiers)
B. Excluding Certain Traffic from the L Queue
(Local-use only)
2. Identifiers to place L4S classification in a PHB Hierarchy
(Could use Recommended-standard-use or Local-use identifiers)
A. PHBs Before L4S ECN Classification
B. PHBs After L4S ECN Classification
5.4.2. Per-Flow Queuing Examples of Other Identifiers Complementing L4S
Identifiers
At a node with per-flow queueing (e.g. FQ-CoDel [RFC8290]), the L4S
identifier could complement the Layer-4 flow ID as a further level of
flow granularity (i.e. Not-ECT and ECT(0) queued separately from
ECT(1) and CE packets). "Risk of reordering Classic CE packets" in
Appendix B discusses the resulting ambiguity if packets originally
marked ECT(0) are marked CE by an upstream AQM before they arrive at
a node that classifies CE as L4S. It argues that the risk of
reordering is vanishingly small and the consequence of such a low
level of reordering is minimal.
Alternatively, it could be assumed that it is not in a flow's own
interest to mix Classic and L4S identifiers. Then the AQM could use
the ECN field to switch itself between a Classic and an L4S AQM
behaviour within one per-flow queue. For instance, for ECN-capable
packets, the AQM might consist of a simple marking threshold and an
L4S ECN identifier might simply select a shallower threshold than a
Classic ECN identifier would.
5.5. Limiting Packet Bursts from Links Supporting L4S AQMs
As well as senders needing to limit packet bursts (Section 4.3),
links need to limit the degree of burstiness they introduce. In both
cases (senders and links) this is a tradeoff, because batch-handling
De Schepper & Briscoe Expires November 22, 2021 [Page 22]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
of packets is done for good reason, e.g. processing efficiency or to
make efficient use of medium acquisition delay. Some take the
attitude that there is no point reducing burst delay at the sender
below that introduced by links (or vice versa). However, delay
reduction proceeds by cutting down 'the longest pole in the tent',
which turns the spotlight on the next longest, and so on.
This document does not set any quantified requirements for links to
limit burst delay, primarily because link technologies are outside
the remit of L4S specifications. Nonetheless, it would not make
sense to implement an L4S AQM that feeds into a particular link
technology without also reviewing opportunities to reduce any form of
burst delay introduced by that link technology. This would at least
limit the bursts that the link would otherwise introduce into the
onward traffic, which would cause jumpy feedback to the sender as
well as potential extra queuing delay downstream. This document does
not presume to even give guidance on an appropriate target for such
burst delay until there is more industry experience of L4S. However,
as suggested in Section 4.3 it would not seem necessary to limit
bursts lower than roughly 10% of the minimum base RTT expected in the
typical deployment scenario (e.g. 250 us burst duration for links
within the public Internet).
6. Behaviour of Tunnels and Encapsulations
6.1. No Change to ECN Tunnels and Encapsulations in General
The L4S identifier is expected to work through and within any tunnel
without modification, as long as the tunnel propagates the ECN field
in any of the ways that have been defined since the first variant in
the year 2001 [RFC3168]. L4S will also work with (but does not rely
on) any of the more recent updates to ECN propagation in [RFC4301],
[RFC6040] or [I-D.ietf-tsvwg-rfc6040update-shim]. However, it is
likely that some tunnels still do not implement ECN propagation at
all. In these cases, L4S will work through such tunnels, but within
them the outer header of L4S traffic will appear as Classic.
AQMs are typically implemented where an IP-layer buffer feeds into a
lower layer, so they are agnostic to link layer encapsulations.
Where a bottleneck link is not IP-aware, the L4S identifier is still
expected to work within any lower layer encapsulation without
modification, as long it propagates the ECN field as defined for the
link technology, for example for MPLS [RFC5129] or TRILL
[I-D.ietf-trill-ecn-support]. In some of these cases, e.g. layer-3
Ethernet switches, the AQM accesses the IP layer header within the
outer encapsulation, so again the L4S identifier is expected to work
without modification. Nonetheless, the programme to define ECN for
De Schepper & Briscoe Expires November 22, 2021 [Page 23]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
other lower layers is still in progress
[I-D.ietf-tsvwg-ecn-encap-guidelines].
6.2. VPN Behaviour to Avoid Limitations of Anti-Replay
If a mix of L4S and Classic packets is sent into the same security
association (SA) of a virtual private network (VPN), and if the VPN
egress is employing the optional anti-replay feature, it could
inappropriately discard Classic packets (or discard the records in
Classic packets) by mistaking their greater queuing delay for a
replay attack (see [Heist21] for the potential performance impact).
This known problem is common to both IPsec [RFC4301] and DTLS
[RFC6347] VPNs, given they use similar anti-replay window mechanisms.
The specifications of IPsec AH [RFC4302] and ESP [RFC4303] suggest
that an implementer scales the size of the anti-replay window with
interface speed, and the current draft of DTLS 1.3 says "The receiver
SHOULD pick a window large enough to handle any plausible reordering,
which depends on the data rate." However, in practice, the size of
the VPN's anti-replay window is not always scaled appropriately.
To avoid this limitation, a VPN ingress participating in the L4S
experiment SHOULD map packets onto two parallel SAs indexed by the
lowest significant bit of the IP-ECN field. The mapping of traffic
onto these two parallel SAs is local to the VPN ingress and not
negotiated with the VPN egress. This recommendation applies to both
transport and tunnel mode SAs.
The above recommendation to use two parallel SAs applies to IPsec
VPNs used in the L4S experiment whether using AH or ESP, and it
applies to DTLS VPNs used in the L4S experiment whether encapsulated
in UDP or in other protocols, for instance DTLS over DCCP [RFC5328],
CAPWAP [RFC5415], SCTP [RFC6083] or SRTP [RFC5764]. These RFCs are
not updated by the present document, which would only become
appropriate if L4S progressed from the experimental to the standards
track. Longer term, if Classic became used rarely, two parallel SAs
would become unnecessary.
Where L4S is used over a VPN, and the ingress VPN's classifier cannot
be modified as above, one of the alternatives listed in order of
preference below might be possible instead:
o If the VPN can be configured to classify packets into different
SAs based on DSCP, apply a different DSCP to Classic and L4S
packets (these DSCPs only need to survive as far as the VPN
ingress);
o If the anti-replay window configured at the VPN egress is
insufficient, increase it; otherwise no further action is needed;
De Schepper & Briscoe Expires November 22, 2021 [Page 24]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
o If none of the above are possible and it is necessary to use L4S,
either of the following might be appropriate as a last resort:
* disable anti-replay protection at the VPN egress, after
considering the security implications (optional anti-replay is
mandatory in both IPsec and DTLS);
* configure the tunnel ingress not to propagate ECN to the outer,
which would lose the benefits of L4S and Classic ECN over the
VPN.
7. L4S Experiments
This section describes open questions that L4S Experiments ought to
focus on. This section also documents outstanding open issues that
will need to be investigated as part of L4S experimentation, given
they could not be fully resolved during the WG phase. It also lists
metrics that will need to be monitored during experiments
(summarizing text elsewhere in L4S documents) and finally lists some
potential future directions that researchers might wish to
investigate.
In addition to this section, [I-D.ietf-tsvwg-aqm-dualq-coupled] sets
operational and management requirements for experiments with DualQ
Coupled AQMs; and General operational and management requirements for
experiments with L4S congestion controls are given in Section 4 and
Section 5 above, e.g. co-existence and scaling requirements,
incremental deployment arrangements.
The specification of each scalable congestion control will need to
include protocol-specific requirements for configuration and
monitoring performance during experiments. Appendix A of [RFC5706]
provides a helpful checklist.
7.1. Open Questions
L4S experiments would be expected to answer the following questions:
o Have all the parts of L4S been deployed, and if so, what
proportion of paths support it?
o Does use of L4S over the Internet result in significantly improved
user experience?
o Has L4S enabled novel interactive applications?
o Did use of L4S over the Internet result in improvements to the
following metrics:
De Schepper & Briscoe Expires November 22, 2021 [Page 25]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
o
* queue delay (mean and 99th percentile) under various loads;
* utilization;
* starvation / fairness;
* scaling range of flow rates and RTTs?
o How much does burstiness in the Internet affect L4S performance,
and how much limitation of bustiness was needed and/or was
realized - both at senders and at links, especially radio links?
o Was per-flow queue protection typically (un)necessary?
* How well did overload protection or queue protection work?
o How well did L4S flows coexist with Classic flows when sharing a
bottleneck?
o
* How frequently did problems arise?
* What caused any coexistence problems, and were any problems due
to single-queue Classic ECN AQMs (this assumes single-queue
Classic ECN AQMs can be distinguished from FQ ones)?
o How prevalent were problems with the L4S service due to tunnels /
encapsulations that do not support ECN decapsulation?
o How easy was it to implement a fully compliant L4S congestion
control, over various different transport protocols (TCP. QUIC,
RMCAT, etc)?
Monitoring for harm to other traffic, specifically bandwidth
starvation or excess queuing delay, will need to be conducted
alongside all early L4S experiments. It is hard, if not impossible,
for an individual flow to measure its impact on other traffic. So
such monitoring will need to be conducted using bespoke monitoring
across flows and/or across classes of traffic.
7.2. Open Issues
o What is the best way forward to deal with L4S over single-queue
Classic ECN AQM bottlenecks, given current problems with
misdetecting L4S AQMs as Classic ECN AQMs?
De Schepper & Briscoe Expires November 22, 2021 [Page 26]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
o Fixing the poor Interaction between current L4S congestion
controls and CoDel with only Classic ECN support during flow
startup.
7.3. Future Potential
Researchers might find that L4S opens up the following interesting
areas for investigation:
o Potential for faster convergence time and tracking of available
capacity;
o Potential for improvements to particular link technologies, and
cross-layer interactions with them;
o Potential for using virtual queues, e.g. to further reduce latency
jitter, or to leave headroom for capacity variation in radio
networks;
o Development and specification of reverse path congestion control
using L4S building bocks (e.g. AccECN, QUIC);
o Once queuing delay is cut down, what becomes the 'second longest
pole in the tent' (other than the speed of light)?
o Novel alternatives to the existing set of L4S AQMs;
o Novel applications enabled by L4S.
8. IANA Considerations
The 01 codepoint of the ECN Field of the IP header is specified by
the present Experimental RFC. The process for an experimental RFC to
assign this codepoint in the IP header (v4 and v6) is documented in
Proposed Standard [RFC8311], which updates the Proposed Standard
[RFC3168].
When the present document is published as an RFC, IANA is asked to
update the 01 entry in the registry, "ECN Field (Bits 6-7)" to the
following (see https://www.iana.org/assignments/dscp-registry/dscp-
registry.xhtml#ecn-field ):
De Schepper & Briscoe Expires November 22, 2021 [Page 27]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
+--------+-----------------------------+----------------------------+
| Binary | Keyword | References |
+--------+-----------------------------+----------------------------+
| 01 | ECT(1) (ECN-Capable | [RFC8311] |
| | Transport(1))[1] | [RFC Errata 5399] |
| | | [RFCXXXX] |
+--------+-----------------------------+----------------------------+
[XXXX is the number that the RFC Editor assigns to the present
document (this sentence to be removed by the RFC Editor)].
9. Security Considerations
Approaches to assure the integrity of signals using the new
identifier are introduced in Appendix C.1. See the security
considerations in the L4S architecture [I-D.ietf-tsvwg-l4s-arch] for
further discussion of mis-use of the identifier, as well as extensive
discussion of policing rate and latency in regard to L4S.
The anti-replay window mechanism in VPNs is known to incorrectly
discard higher delay packets (e.g. Classic) when carried alongside
low delay packets (e.g. L4S) in the same VPN. Section 6.2
recommends that VPNs used in L4S experiments are configured to
separate Classic and L4S traffic into separate security associations
(SAs) to prevent this problem. This is modelled on the multi-SA
approach used for Diffserv over IPsec, which is intended to avoid the
same problem (see section 4.1 of [RFC4301]).
If a user taking part in the L4S experiment sets up a VPN without
being aware of the above advice, and if the user allows anyone to
send traffic into their VPN, they would open up a DoS vulnerability
in which an attacker could induce the VPN's anti-replay mechanism to
discard enough of the user's Classic (C) traffic (if they are
receiving any) to cause a significant rate reduction. While the user
is actively downloading C traffic, the attacker sends C traffic into
the VPN to fill the remainder of the bottleneck link, then sends
intermittent L4S packets to maximize the chance of exceeding the
VPN's replay window. The user can prevent this attack by following
the recommendations in Section 6.2.
The recommendation to detect loss in time units prevents the ACK-
splitting attacks described in [Savage-TCP].
10. Acknowledgements
Thanks to Richard Scheffenegger, John Leslie, David Taeht, Jonathan
Morton, Gorry Fairhurst, Michael Welzl, Mikael Abrahamsson and Andrew
McGregor for the discussions that led to this specification. Ing-jyh
De Schepper & Briscoe Expires November 22, 2021 [Page 28]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
(Inton) Tsang was a contributor to the early drafts of this document.
And thanks to Mikael Abrahamsson, Lloyd Wood, Nicolas Kuhn, Greg
White, Tom Henderson, David Black, Gorry Fairhurst, Brian Carpenter,
Jake Holland, Rod Grimes, Richard Scheffenegger, Sebastian Moeller,
Neal Cardwell, Praveen Balasubramanian, Reza Marandian Hagh, Stuart
Cheshire and Vidhi Goel for providing help and reviewing this draft
and thanks to Ingemar Johansson for reviewing and providing
substantial text. Thanks to Sebastian Moeller for identifying the
interaction with VPN anti-replay and to Jonathan Morton for
identifying the attack based on this. Particular thanks to tsvwg
chairs Gorry Fairhurst, David Black and Wes Eddy for patiently
helping this and the other L4S drafts through the IETF process.
Appendix A listing the Prague L4S Requirements is based on text
authored by Marcelo Bagnulo Braun that was originally an appendix to
[I-D.ietf-tsvwg-l4s-arch]. That text was in turn based on the
collective output of the attendees listed in the minutes of a 'bar
BoF' on DCTCP Evolution during IETF-94 [TCPPrague].
The authors' contributions were part-funded by the European Community
under its Seventh Framework Programme through the Reducing Internet
Transport Latency (RITE) project (ICT-317700). Bob Briscoe was also
funded partly by the Research Council of Norway through the TimeIn
project, partly by CableLabs and partly by the Comcast Innovation
Fund. The views expressed here are solely those of the authors.
11. References
11.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
of Explicit Congestion Notification (ECN) to IP",
RFC 3168, DOI 10.17487/RFC3168, September 2001,
<https://www.rfc-editor.org/info/rfc3168>.
[RFC4774] Floyd, S., "Specifying Alternate Semantics for the
Explicit Congestion Notification (ECN) Field", BCP 124,
RFC 4774, DOI 10.17487/RFC4774, November 2006,
<https://www.rfc-editor.org/info/rfc4774>.
[RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P.,
and K. Carlberg, "Explicit Congestion Notification (ECN)
for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August
2012, <https://www.rfc-editor.org/info/rfc6679>.
De Schepper & Briscoe Expires November 22, 2021 [Page 29]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
11.2. Informative References
[A2DTCP] Zhang, T., Wang, J., Huang, J., Huang, Y., Chen, J., and
Y. Pan, "Adaptive-Acceleration Data Center TCP", IEEE
Transactions on Computers 64(6):1522-1533, June 2015,
<http://ieeexplore.ieee.org/xpl/
articleDetails.jsp?arnumber=6871352>.
[Ahmed19] Ahmed, A., "Extending TCP for Low Round Trip Delay",
Masters Thesis, Uni Oslo , August 2019,
<https://www.duo.uio.no/handle/10852/70966>.
[Alizadeh-stability]
Alizadeh, M., Javanmard, A., and B. Prabhakar, "Analysis
of DCTCP: Stability, Convergence, and Fairness", ACM
SIGMETRICS 2011 , June 2011.
[ARED01] Floyd, S., Gummadi, R., and S. Shenker, "Adaptive RED: An
Algorithm for Increasing the Robustness of RED's Active
Queue Management", ACIRI Technical Report , August 2001,
<http://www.icir.org/floyd/red.html>.
[DCttH15] De Schepper, K., Bondarenko, O., Briscoe, B., and I.
Tsang, "'Data Centre to the Home': Ultra-Low Latency for
All", RITE Project Technical Report , 2015,
<http://riteproject.eu/publications/>.
[ecn-fallback]
Briscoe, B. and A. Ahmed, "TCP Prague Fall-back on
Detection of a Classic ECN AQM", bobbriscoe.net Technical
Report TR-BB-2019-002, April 2020,
<https://arxiv.org/abs/1911.00710>.
[Heist21] Heist, P., "Dropped Packets for Tunnels with Replay
Protection Enabled", github README, May 2021,
<https://github.com/heistp/l4s-tests/#dropped-packets-for-
tunnels-with-replay-protection-enabled>.
[I-D.briscoe-docsis-q-protection]
Briscoe, B. and G. White, "Queue Protection to Preserve
Low Latency", draft-briscoe-docsis-q-protection-00 (work
in progress), July 2019.
[I-D.briscoe-iccrg-prague-congestion-control]
Schepper, K. D., Tilmans, O., and B. Briscoe, "Prague
Congestion Control", draft-briscoe-iccrg-prague-
congestion-control-00 (work in progress), March 2021.
De Schepper & Briscoe Expires November 22, 2021 [Page 30]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
[I-D.briscoe-tsvwg-l4s-diffserv]
Briscoe, B., "Interactions between Low Latency, Low Loss,
Scalable Throughput (L4S) and Differentiated Services",
draft-briscoe-tsvwg-l4s-diffserv-02 (work in progress),
November 2018.
[I-D.ietf-avtcore-cc-feedback-message]
Sarker, Z., Perkins, C., Singh, V., and M. A. Ramalho,
"RTP Control Protocol (RTCP) Feedback for Congestion
Control", draft-ietf-avtcore-cc-feedback-message-09 (work
in progress), November 2020.
[I-D.ietf-quic-transport]
Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed
and Secure Transport", draft-ietf-quic-transport-34 (work
in progress), January 2021.
[I-D.ietf-tcpm-accurate-ecn]
Briscoe, B., Kuehlewind, M., and R. Scheffenegger, "More
Accurate ECN Feedback in TCP", draft-ietf-tcpm-accurate-
ecn-14 (work in progress), February 2021.
[I-D.ietf-tcpm-generalized-ecn]
Bagnulo, M. and B. Briscoe, "ECN++: Adding Explicit
Congestion Notification (ECN) to TCP Control Packets",
draft-ietf-tcpm-generalized-ecn-07 (work in progress),
February 2021.
[I-D.ietf-trill-ecn-support]
Eastlake, D. E. and B. Briscoe, "TRILL (TRansparent
Interconnection of Lots of Links): ECN (Explicit
Congestion Notification) Support", draft-ietf-trill-ecn-
support-07 (work in progress), February 2018.
[I-D.ietf-tsvwg-aqm-dualq-coupled]
Schepper, K. D., Briscoe, B., and G. White, "DualQ Coupled
AQMs for Low Latency, Low Loss and Scalable Throughput
(L4S)", draft-ietf-tsvwg-aqm-dualq-coupled-14 (work in
progress), March 2021.
[I-D.ietf-tsvwg-ecn-encap-guidelines]
Briscoe, B. and J. Kaippallimalil, "Guidelines for Adding
Congestion Notification to Protocols that Encapsulate IP",
draft-ietf-tsvwg-ecn-encap-guidelines-15 (work in
progress), March 2021.
De Schepper & Briscoe Expires November 22, 2021 [Page 31]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
[I-D.ietf-tsvwg-l4s-arch]
Briscoe, B., Schepper, K. D., Bagnulo, M., and G. White,
"Low Latency, Low Loss, Scalable Throughput (L4S) Internet
Service: Architecture", draft-ietf-tsvwg-l4s-arch-08 (work
in progress), November 2020.
[I-D.ietf-tsvwg-l4sops]
White, G., "Operational Guidance for Deployment of L4S in
the Internet", draft-ietf-tsvwg-l4sops-00 (work in
progress), May 2021.
[I-D.ietf-tsvwg-nqb]
White, G. and T. Fossati, "A Non-Queue-Building Per-Hop
Behavior (NQB PHB) for Differentiated Services", draft-
ietf-tsvwg-nqb-05 (work in progress), March 2021.
[I-D.ietf-tsvwg-rfc6040update-shim]
Briscoe, B., "Propagating Explicit Congestion Notification
Across IP Tunnel Headers Separated by a Shim", draft-ietf-
tsvwg-rfc6040update-shim-13 (work in progress), March
2021.
[I-D.sridharan-tcpm-ctcp]
Sridharan, M., Tan, K., Bansal, D., and D. Thaler,
"Compound TCP: A New TCP Congestion Control for High-Speed
and Long Distance Networks", draft-sridharan-tcpm-ctcp-02
(work in progress), November 2008.
[I-D.stewart-tsvwg-sctpecn]
Stewart, R. R., Tuexen, M., and X. Dong, "ECN for Stream
Control Transmission Protocol (SCTP)", draft-stewart-
tsvwg-sctpecn-05 (work in progress), January 2014.
[LinuxPacedChirping]
Misund, J. and B. Briscoe, "Paced Chirping - Rethinking
TCP start-up", Proc. Linux Netdev 0x13 , March 2019,
<https://www.netdevconf.org/0x13/session.html?talk-chirp>.
[Mathis09]
Mathis, M., "Relentless Congestion Control", PFLDNeT'09 ,
May 2009, <http://www.hpcc.jp/pfldnet2009/
Program_files/1569198525.pdf>.
[Paced-Chirping]
Misund, J., "Rapid Acceleration in TCP Prague", Masters
Thesis , May 2018,
<https://riteproject.files.wordpress.com/2018/07/
misundjoakimmastersthesissubmitted180515.pdf>.
De Schepper & Briscoe Expires November 22, 2021 [Page 32]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
[PI2] De Schepper, K., Bondarenko, O., Tsang, I., and B.
Briscoe, "PI^2 : A Linearized AQM for both Classic and
Scalable TCP", Proc. ACM CoNEXT 2016 pp.105-119, December
2016,
<http://dl.acm.org/citation.cfm?doid=2999572.2999578>.
[PragueLinux]
Briscoe, B., De Schepper, K., Albisser, O., Misund, J.,
Tilmans, O., Kuehlewind, M., and A. Ahmed, "Implementing
the `TCP Prague' Requirements for Low Latency Low Loss
Scalable Throughput (L4S)", Proc. Linux Netdev 0x13 ,
March 2019, <https://www.netdevconf.org/0x13/
session.html?talk-tcp-prague-l4s>.
[QV] Briscoe, B. and P. Hurtig, "Up to Speed with Queue View",
RITE Technical Report D2.3; Appendix C.2, August 2015,
<https://riteproject.files.wordpress.com/2015/12/rite-
deliverable-2-3.pdf>.
[RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering,
S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G.,
Partridge, C., Peterson, L., Ramakrishnan, K., Shenker,
S., Wroclawski, J., and L. Zhang, "Recommendations on
Queue Management and Congestion Avoidance in the
Internet", RFC 2309, DOI 10.17487/RFC2309, April 1998,
<https://www.rfc-editor.org/info/rfc2309>.
[RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black,
"Definition of the Differentiated Services Field (DS
Field) in the IPv4 and IPv6 Headers", RFC 2474,
DOI 10.17487/RFC2474, December 1998,
<https://www.rfc-editor.org/info/rfc2474>.
[RFC3246] Davie, B., Charny, A., Bennet, J., Benson, K., Le Boudec,
J., Courtney, W., Davari, S., Firoiu, V., and D.
Stiliadis, "An Expedited Forwarding PHB (Per-Hop
Behavior)", RFC 3246, DOI 10.17487/RFC3246, March 2002,
<https://www.rfc-editor.org/info/rfc3246>.
[RFC3540] Spring, N., Wetherall, D., and D. Ely, "Robust Explicit
Congestion Notification (ECN) Signaling with Nonces",
RFC 3540, DOI 10.17487/RFC3540, June 2003,
<https://www.rfc-editor.org/info/rfc3540>.
[RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows",
RFC 3649, DOI 10.17487/RFC3649, December 2003,
<https://www.rfc-editor.org/info/rfc3649>.
De Schepper & Briscoe Expires November 22, 2021 [Page 33]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
[RFC4301] Kent, S. and K. Seo, "Security Architecture for the
Internet Protocol", RFC 4301, DOI 10.17487/RFC4301,
December 2005, <https://www.rfc-editor.org/info/rfc4301>.
[RFC4302] Kent, S., "IP Authentication Header", RFC 4302,
DOI 10.17487/RFC4302, December 2005,
<https://www.rfc-editor.org/info/rfc4302>.
[RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)",
RFC 4303, DOI 10.17487/RFC4303, December 2005,
<https://www.rfc-editor.org/info/rfc4303>.
[RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram
Congestion Control Protocol (DCCP)", RFC 4340,
DOI 10.17487/RFC4340, March 2006,
<https://www.rfc-editor.org/info/rfc4340>.
[RFC4341] Floyd, S. and E. Kohler, "Profile for Datagram Congestion
Control Protocol (DCCP) Congestion Control ID 2: TCP-like
Congestion Control", RFC 4341, DOI 10.17487/RFC4341, March
2006, <https://www.rfc-editor.org/info/rfc4341>.
[RFC4342] Floyd, S., Kohler, E., and J. Padhye, "Profile for
Datagram Congestion Control Protocol (DCCP) Congestion
Control ID 3: TCP-Friendly Rate Control (TFRC)", RFC 4342,
DOI 10.17487/RFC4342, March 2006,
<https://www.rfc-editor.org/info/rfc4342>.
[RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol",
RFC 4960, DOI 10.17487/RFC4960, September 2007,
<https://www.rfc-editor.org/info/rfc4960>.
[RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion
Control Algorithms", BCP 133, RFC 5033,
DOI 10.17487/RFC5033, August 2007,
<https://www.rfc-editor.org/info/rfc5033>.
[RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion
Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January
2008, <https://www.rfc-editor.org/info/rfc5129>.
[RFC5328] Adolf, A. and P. MacAvock, "A Uniform Resource Name (URN)
Namespace for the Digital Video Broadcasting Project
(DVB)", RFC 5328, DOI 10.17487/RFC5328, September 2008,
<https://www.rfc-editor.org/info/rfc5328>.
De Schepper & Briscoe Expires November 22, 2021 [Page 34]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
[RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP
Friendly Rate Control (TFRC): Protocol Specification",
RFC 5348, DOI 10.17487/RFC5348, September 2008,
<https://www.rfc-editor.org/info/rfc5348>.
[RFC5415] Calhoun, P., Ed., Montemurro, M., Ed., and D. Stanley,
Ed., "Control And Provisioning of Wireless Access Points
(CAPWAP) Protocol Specification", RFC 5415,
DOI 10.17487/RFC5415, March 2009,
<https://www.rfc-editor.org/info/rfc5415>.
[RFC5562] Kuzmanovic, A., Mondal, A., Floyd, S., and K.
Ramakrishnan, "Adding Explicit Congestion Notification
(ECN) Capability to TCP's SYN/ACK Packets", RFC 5562,
DOI 10.17487/RFC5562, June 2009,
<https://www.rfc-editor.org/info/rfc5562>.
[RFC5622] Floyd, S. and E. Kohler, "Profile for Datagram Congestion
Control Protocol (DCCP) Congestion ID 4: TCP-Friendly Rate
Control for Small Packets (TFRC-SP)", RFC 5622,
DOI 10.17487/RFC5622, August 2009,
<https://www.rfc-editor.org/info/rfc5622>.
[RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion
Control", RFC 5681, DOI 10.17487/RFC5681, September 2009,
<https://www.rfc-editor.org/info/rfc5681>.
[RFC5706] Harrington, D., "Guidelines for Considering Operations and
Management of New Protocols and Protocol Extensions",
RFC 5706, DOI 10.17487/RFC5706, November 2009,
<https://www.rfc-editor.org/info/rfc5706>.
[RFC5764] McGrew, D. and E. Rescorla, "Datagram Transport Layer
Security (DTLS) Extension to Establish Keys for the Secure
Real-time Transport Protocol (SRTP)", RFC 5764,
DOI 10.17487/RFC5764, May 2010,
<https://www.rfc-editor.org/info/rfc5764>.
[RFC5865] Baker, F., Polk, J., and M. Dolly, "A Differentiated
Services Code Point (DSCP) for Capacity-Admitted Traffic",
RFC 5865, DOI 10.17487/RFC5865, May 2010,
<https://www.rfc-editor.org/info/rfc5865>.
[RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP
Authentication Option", RFC 5925, DOI 10.17487/RFC5925,
June 2010, <https://www.rfc-editor.org/info/rfc5925>.
De Schepper & Briscoe Expires November 22, 2021 [Page 35]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
[RFC6040] Briscoe, B., "Tunnelling of Explicit Congestion
Notification", RFC 6040, DOI 10.17487/RFC6040, November
2010, <https://www.rfc-editor.org/info/rfc6040>.
[RFC6077] Papadimitriou, D., Ed., Welzl, M., Scharf, M., and B.
Briscoe, "Open Research Issues in Internet Congestion
Control", RFC 6077, DOI 10.17487/RFC6077, February 2011,
<https://www.rfc-editor.org/info/rfc6077>.
[RFC6083] Tuexen, M., Seggelmann, R., and E. Rescorla, "Datagram
Transport Layer Security (DTLS) for Stream Control
Transmission Protocol (SCTP)", RFC 6083,
DOI 10.17487/RFC6083, January 2011,
<https://www.rfc-editor.org/info/rfc6083>.
[RFC6347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer
Security Version 1.2", RFC 6347, DOI 10.17487/RFC6347,
January 2012, <https://www.rfc-editor.org/info/rfc6347>.
[RFC6660] Briscoe, B., Moncaster, T., and M. Menth, "Encoding Three
Pre-Congestion Notification (PCN) States in the IP Header
Using a Single Diffserv Codepoint (DSCP)", RFC 6660,
DOI 10.17487/RFC6660, July 2012,
<https://www.rfc-editor.org/info/rfc6660>.
[RFC6675] Blanton, E., Allman, M., Wang, L., Jarvinen, I., Kojo, M.,
and Y. Nishida, "A Conservative Loss Recovery Algorithm
Based on Selective Acknowledgment (SACK) for TCP",
RFC 6675, DOI 10.17487/RFC6675, August 2012,
<https://www.rfc-editor.org/info/rfc6675>.
[RFC7560] Kuehlewind, M., Ed., Scheffenegger, R., and B. Briscoe,
"Problem Statement and Requirements for Increased Accuracy
in Explicit Congestion Notification (ECN) Feedback",
RFC 7560, DOI 10.17487/RFC7560, August 2015,
<https://www.rfc-editor.org/info/rfc7560>.
[RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF
Recommendations Regarding Active Queue Management",
BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015,
<https://www.rfc-editor.org/info/rfc7567>.
[RFC7713] Mathis, M. and B. Briscoe, "Congestion Exposure (ConEx)
Concepts, Abstract Mechanism, and Requirements", RFC 7713,
DOI 10.17487/RFC7713, December 2015,
<https://www.rfc-editor.org/info/rfc7713>.
De Schepper & Briscoe Expires November 22, 2021 [Page 36]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
[RFC8033] Pan, R., Natarajan, P., Baker, F., and G. White,
"Proportional Integral Controller Enhanced (PIE): A
Lightweight Control Scheme to Address the Bufferbloat
Problem", RFC 8033, DOI 10.17487/RFC8033, February 2017,
<https://www.rfc-editor.org/info/rfc8033>.
[RFC8257] Bensley, S., Thaler, D., Balasubramanian, P., Eggert, L.,
and G. Judd, "Data Center TCP (DCTCP): TCP Congestion
Control for Data Centers", RFC 8257, DOI 10.17487/RFC8257,
October 2017, <https://www.rfc-editor.org/info/rfc8257>.
[RFC8290] Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys,
J., and E. Dumazet, "The Flow Queue CoDel Packet Scheduler
and Active Queue Management Algorithm", RFC 8290,
DOI 10.17487/RFC8290, January 2018,
<https://www.rfc-editor.org/info/rfc8290>.
[RFC8298] Johansson, I. and Z. Sarker, "Self-Clocked Rate Adaptation
for Multimedia", RFC 8298, DOI 10.17487/RFC8298, December
2017, <https://www.rfc-editor.org/info/rfc8298>.
[RFC8311] Black, D., "Relaxing Restrictions on Explicit Congestion
Notification (ECN) Experimentation", RFC 8311,
DOI 10.17487/RFC8311, January 2018,
<https://www.rfc-editor.org/info/rfc8311>.
[RFC8312] Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and
R. Scheffenegger, "CUBIC for Fast Long-Distance Networks",
RFC 8312, DOI 10.17487/RFC8312, February 2018,
<https://www.rfc-editor.org/info/rfc8312>.
[RFC8511] Khademi, N., Welzl, M., Armitage, G., and G. Fairhurst,
"TCP Alternative Backoff with ECN (ABE)", RFC 8511,
DOI 10.17487/RFC8511, December 2018,
<https://www.rfc-editor.org/info/rfc8511>.
[RFC8985] Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "The
RACK-TLP Loss Detection Algorithm for TCP", RFC 8985,
DOI 10.17487/RFC8985, February 2021,
<https://www.rfc-editor.org/info/rfc8985>.
[Savage-TCP]
Savage, S., Cardwell, N., Wetherall, D., and T. Anderson,
"TCP Congestion Control with a Misbehaving Receiver", ACM
SIGCOMM Computer Communication Review 29(5):71--78,
October 1999.
De Schepper & Briscoe Expires November 22, 2021 [Page 37]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
[sub-mss-prob]
Briscoe, B. and K. De Schepper, "Scaling TCP's Congestion
Window for Small Round Trip Times", BT Technical Report
TR-TUB8-2015-002, May 2015,
<https://arxiv.org/abs/1904.07598>.
[TCP-CA] Jacobson, V. and M. Karels, "Congestion Avoidance and
Control", Laurence Berkeley Labs Technical Report ,
November 1988, <http://ee.lbl.gov/papers/congavoid.pdf>.
[TCPPrague]
Briscoe, B., "Notes: DCTCP evolution 'bar BoF': Tue 21 Jul
2015, 17:40, Prague", tcpprague mailing list archive ,
July 2015, <https://www.ietf.org/mail-
archive/web/tcpprague/current/msg00001.html>.
[VCP] Xia, Y., Subramanian, L., Stoica, I., and S. Kalyanaraman,
"One more bit is enough", Proc. SIGCOMM'05, ACM CCR
35(4)37--48, 2005,
<http://doi.acm.org/10.1145/1080091.1080098>.
Appendix A. The 'Prague L4S Requirements'
This appendix is informative, not normative. It gives a list of
modifications to current scalable congestion controls so that they
can be deployed over the public Internet and coexist safely with
existing traffic. The list complements the normative requirements in
Section 4 that a sender has to comply with before it can set the L4S
identifier in packets it sends into the Internet. As well as
necessary safety improvements (requirements) this appendix also
includes preferable performance improvements (optimizations).
These recommendations have become know as the Prague L4S
Requirements, because they were originally identified at an ad hoc
meeting during IETF-94 in Prague [TCPPrague]. They were originally
called the 'TCP Prague Requirements', but they are not solely
applicable to TCP, so the name and wording has been generalized for
all transport protocols, and the name 'TCP Prague' is now used for a
specific implementation of the requirements.
At the time of writing, DCTCP [RFC8257] is the most widely used
scalable transport protocol. In its current form, DCTCP is specified
to be deployable only in controlled environments. Deploying it in
the public Internet would lead to a number of issues, both from the
safety and the performance perspective. The modifications and
additional mechanisms listed in this section will be necessary for
its deployment over the global Internet. Where an example is needed,
DCTCP is used as a base, but it is likely that most of these
De Schepper & Briscoe Expires November 22, 2021 [Page 38]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
requirements equally apply to other scalable congestion controls,
covering adaptive real-time media, etc., not just capacity-seeking
behaviours.
A.1. Requirements for Scalable Transport Protocols
A.1.1. Use of L4S Packet Identifier
Description: A scalable congestion control needs to distinguish the
packets it sends from those sent by Classic congestion controls (see
the precise normative requirement wording in Section 4.1).
Motivation: It needs to be possible for a network node to classify
L4S packets without flow state into a queue that applies an L4S ECN
marking behaviour and isolates L4S packets from the queuing delay of
Classic packets.
A.1.2. Accurate ECN Feedback
Description: The transport protocol for a scalable congestion control
needs to provide timely, accurate feedback about the extent of ECN
marking experienced by all packets (see the precise normative
requirement wording in Section 4.2).
Motivation: Classic congestion controls only need feedback about the
existence of a congestion episode within a round trip, not precisely
how many packets were marked with ECN or dropped. Therefore, in
2001, when ECN feedback was added to TCP [RFC3168], it could not
inform the sender of more than one ECN mark per RTT. Since then,
requirements for more accurate ECN feedback in TCP have been defined
in [RFC7560] and [I-D.ietf-tcpm-accurate-ecn] specifies a change to
the TCP protocol to satisfy these requirements. Most other transport
protocols already satisfy this requirement (see Section 4.2).
A.1.3. Capable of Replacement by Classic Congestion Control
Description: It needs to be possible to replace the implementation of
a scalable congestion control with a Classic control (see the precise
normative requirement wording in Section 4.3).
Motivation: L4S is an experimental protocol, therefore it seems
prudent to be able to disable it at source in case of insurmountable
problems, perhaps due to some unexpected interaction on a particular
sender; over a particular path or network; with a particular receiver
or even ultimately an insurmountable problem with the experiment as a
whole.
De Schepper & Briscoe Expires November 22, 2021 [Page 39]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
A.1.4. Fall back to Classic Congestion Control on Packet Loss
Description: As well as responding to ECN markings in a scalable way,
a scalable congestion control needs to react to packet loss in a way
that will coexist safely with a Reno congestion control [RFC5681]
(see the precise normative requirement wording in Section 4.3).
Motivation: Part of the safety conditions for deploying a scalable
congestion control on the public Internet is to make sure that it
behaves properly when it builds a queue at a network bottleneck that
has not been upgraded to support L4S. Packet loss can have many
causes, but it usually has to be conservatively assumed that it is a
sign of congestion. Therefore, on detecting packet loss, a scalable
congestion control will need to fall back to Classic congestion
control behaviour. If it does not comply with this requirement it
could starve Classic traffic.
A scalable congestion control can be used for different types of
transport, e.g. for real-time media or for reliable transport like
TCP. Therefore, the particular Classic congestion control behaviour
to fall back on will need to be dependent on the specific congestion
control implementation. In the particular case of DCTCP, the DCTCP
specification [RFC8257] states that "It is RECOMMENDED that an
implementation deal with loss episodes in the same way as
conventional TCP." For safe deployment of a scalable congestion
control in the public Internet, the above requirement would need to
be defined as a "MUST".
Even though a bottleneck is L4S capable, it might still become
overloaded and have to drop packets. In this case, the sender may
receive a high proportion of packets marked with the CE bit set and
also experience loss. Current DCTCP implementations each react
differently to this situation. At least one implementation reacts
only to the drop signal (e.g. by halving the CWND) and at least
another DCTCP implementation reacts to both signals (e.g. by halving
the CWND due to the drop and also further reducing the CWND based on
the proportion of marked packet). A third approach for the public
Internet has been proposed that adjusts the loss response to result
in a halving when combined with the ECN response. We believe that
further experimentation is needed to understand what is the best
behaviour for the public Internet, which may or not be one of these
existing approaches.
De Schepper & Briscoe Expires November 22, 2021 [Page 40]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
A.1.5. Coexistence with Classic Congestion Control at Classic ECN
bottlenecks
Description: Monitoring has to be in place so that a non-L4S but ECN-
capable AQM can be detected at path bottlenecks. This is in case
such an AQM has been implemented in a shared queue, in which case any
long-running scalable flow would predominate over any simultaneous
long-running Classic flow sharing the queue. The requirement is
written so that such a problem could either be resolved in real-time,
or via administrative intervention (see the precise normative
requirement wording in Section 4.3).
Motivation: Similarly to the requirement in Appendix A.1.4, this
requirement is a safety condition to ensure an L4S congestion control
coexists well with Classic flows when it builds a queue at a shared
network bottleneck that has not been upgraded to support L4S.
Nonetheless, if necessary, it is considered reasonable to resolve
such problems over management timescales (possibly involving human
intervention) because:
o although a Classic flow can considerably reduce its throughput in
the face of a competing scalable flow, it still makes progress and
does not starve;
o implementations of a Classic ECN AQM in a queue that is intended
to be shared are believed to be rare;
o detection of such AQMs is not always clear-cut; so focused out-of-
band testing (or even contacting the relevant network operator)
would improve certainty.
Therefore, the relevant normative requirement (Section 4.3) is
divided into three stages: monitoring, detection and action:
Monitoring: Monitoring involves collection of the measurement data
to be analysed. Monitoring is expressed as a 'MUST' for
uncontrolled environments, although the placement of the
monitoring function is left open. Whether monitoring has to be
applied in real-time is expressed as a 'SHOULD'. This allows for
the possibility that the operator of an L4S sender (e.g. a CDN)
might prefer to test out-of-band for signs of Classic ECN AQMs,
perhaps to avoid continually consuming resources to monitor live
traffic.
Detection: Detection involves analysis of the monitored data to
detect the likelihood of a Classic ECN AQM. The requirements
recommend that detection occurs live in real-time. However,
De Schepper & Briscoe Expires November 22, 2021 [Page 41]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
detection is allowed to be deferred (e.g. it might involve further
testing targeted at candidate AQMs);
Action: This involves the act of switching the sender to a Classic
congestion control. This might occur in real-time within the
congestion control for the subsequent duration of a flow, or it
might involve administrative action to switch to Classic
congestion control for a specific interface or for a certain set
of destination addresses.
Instead of the sender taking action itself, the operator of the
sender (e.g. a CDN) might prefer to ask the network operator to
modify the Classic AQM's treatment of L4S packets; or to ensure
L4S packets bypass the AQM; or to upgrade the AQM to support L4S.
Once L4S flows no longer shared the Classic ECN AQM they would
obviously no longer detect it, and the requirement to act on it
would no longer apply.
The whole set of normative requirements concerning Classic ECN AQMs
does not apply in controlled environments, such as private networks
or data centre networks. CDN servers placed within an access ISP's
network can be considered as a single controlled environment, but any
onward networks served by the access network, including all the
attached customer networks, would be unlikely to fall under the same
degree of coordinated control. Monitoring is expressed as a 'MUST'
for these uncontrolled segments of paths (e.g. beyond the access ISP
in a home network), because there is a possibility that there might
be a shared queue Classic ECN AQM in that segment. Nonetheless, the
intent is to only require occasional monitoring of these uncontrolled
regions, and not to burden CDN operators if monitoring never uncovers
any potential problems, given it is anyway in the CDN's own interests
not to degrade the service of its own customers.
More detailed discussion of all the above options and alternatives
can be found in [I-D.ietf-tsvwg-l4sops].
Having said all the above, the approach recommended in the
requirements is to monitor, detect and act in real-time on live
traffic. A passive monitoring algorithm to detect a Classic ECN AQM
at the bottleneck and fall back to Classic congestion control is
described in an extensive technical report [ecn-fallback], which also
provides a link to Linux source code, and a large online
visualization of its evaluation results. Very briefly, the algorithm
primarily monitors RTT variation using the same algorithm that
maintains the mean deviation of TCP's smoothed RTT, but it smooths
over a duration of the order of a Classic sawtooth. The outcome is
also conditioned on other metrics such as the presence of CE marking
and congestion avoidance phase having stabilized. The report also
De Schepper & Briscoe Expires November 22, 2021 [Page 42]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
identifies further work to improve the approach, for instance
improvements with low capacity links and combining the measurements
with a cache of what had been learned about a path in previous
connections. The report also suggests alternative approaches.
Although using passive measurements within live traffic (as above)
can detect a Classic ECN AQM, it is much harder (perhaps impossible)
to determine whether or not the AQM is in a shared queue.
Nonetheless, this is much easier using active test traffic out-of-
band, because two flows can be used. Section 4 of the same report
[ecn-fallback] describes a simple technique to detect a Classic ECN
AQM and determine whether it is in a shared queue, summarized here.
An L4S-enabled test server could be set up so that, when a test
client accesses it, it serves a script that gets the client to open
two parallel long-running flows. It could serve one with a Classic
congestion control (C, that sets ECT(0)) and one with a scaleable CC
(L, that sets ECT(1)).If neither flow induces any ECN marks, it can
be presumed the path does not contain a Classic ECN AQM. If either
flow induces some ECN marks, the server could measure the relative
flow rates and round trip times of the two flows. Table 1 shows the
AQM that can be inferred for various cases.
+--------+-------+------------------------+
| Rate | RTT | Inferred AQM |
+--------+-------+------------------------+
| L > C | L = C | Classic ECN AQM (FIFO) |
| L = C | L = C | Classic ECN AQM (FQ) |
| L = C | L < C | FQ-L4S AQM |
| L ~= C | L < C | Coupled DualQ AQM |
+--------+-------+------------------------+
Table 1: Out-of-band testing with two parallel flows. L:=L4S,
C:=Classic.
Finally, we motivate the recommendation in Section 4.3 that a
scalable congestion control is not expected to change to setting
ECT(0) while it adapts its behaviour to coexist with Classic flows.
This is because the sender needs to continue to check whether it made
the right decision - and switch back if it was wrong, or if a
different link becomes the bottleneck:
o If, as recommended, the sender changes only its behaviour but not
its codepoint to Classic, its codepoint will still be compatible
with either an L4S or a Classic AQM. If the bottleneck does
actually support both, it will still classify ECT(1) into the same
L4S queue, where the sender can measure that switching to Classic
behaviour was wrong, so that it can switch back.
De Schepper & Briscoe Expires November 22, 2021 [Page 43]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
o In contrast, if the sender changes both its behaviour and its
codepoint to Classic, even if the bottleneck supports both, it
will classify ECT(0) into the Classic queue, reinforcing the
sender's incorrect decision so that it never switches back.
o Also, not changing codepoint avoids the risk of being flipped to a
different path by a load balancer or multipath routing that hashes
on the whole of the ex-ToS byte (unfortunately still a common
pathology).
Note that if a flow is configured to _only_ use a Classic congestion
control, it is then entirely appropriate not to use ECT(1).
A.1.6. Reduce RTT dependence
Description: A scalable congestion control needs to reduce RTT bias
as much as possible at least over the low to typical range of RTTs
that will interact in the intended deployment scenario (see the
precise normative requirement wording in Section 4.3).
Motivation: The throughput of Classic congestion controls is known to
be inversely proportional to RTT, so one would expect flows over very
low RTT paths to nearly starve flows over larger RTTs. However,
Classic congestion controls have never allowed a very low RTT path to
exist because they induce a large queue. For instance, consider two
paths with base RTT 1ms and 100ms. If a Classic congestion control
induces a 100ms queue, it turns these RTTs into 101ms and 200ms
leading to a throughput ratio of about 2:1. Whereas if a scalable
congestion control induces only a 1ms queue, the ratio is 2:101,
leading to a throughput ratio of about 50:1.
Therefore, with very small queues, long RTT flows will essentially
starve, unless scalable congestion controls comply with this
requirement.
The RTT bias in current Classic congestion controls works
satisfactorily when the RTT is higher than typical, and L4S does not
change that. So, there is no additional requirement for high RTT L4S
flows to remove RTT bias - they can but they don't have to.
A.1.7. Scaling down to fractional congestion windows
Description: A scalable congestion control needs to remain responsive
to congestion when typical RTTs over the public Internet are
significantly smaller because they are no longer inflated by queuing
delay (see the precise normative requirement wording in Section 4.3).
De Schepper & Briscoe Expires November 22, 2021 [Page 44]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Motivation: As currently specified, the minimum congestion window of
ECN-capable TCP (and its derivatives) is expected to be 2 sender
maximum segment sizes (SMSS), or 1 SMSS after a retransmission
timeout. Once the congestion window reaches this minimum, if there
is further ECN-marking, TCP is meant to wait for a retransmission
timeout before sending another segment (see section 6.1.2 of
[RFC3168]). In practice, most known window-based congestion control
algorithms become unresponsive to congestion signals at this point.
No matter how much drop or ECN marking, the congestion window no
longer reduces. Instead, the sender's lack of any further congestion
response forces the queue to grow, overriding any AQM and increasing
queuing delay (making the window large enough to become responsive
again).
Most congestion controls for other transport protocols have a similar
minimum, albeit when measured in bytes for those that use smaller
packets.
L4S mechanisms significantly reduce queueing delay so, over the same
path, the RTT becomes lower. Then this problem becomes surprisingly
common [sub-mss-prob]. This is because, for the same link capacity,
smaller RTT implies a smaller window. For instance, consider a
residential setting with an upstream broadband Internet access of 8
Mb/s, assuming a max segment size of 1500 B. Two upstream flows will
each have the minimum window of 2 SMSS if the RTT is 6ms or less,
which is quite common when accessing a nearby data centre. So, any
more than two such parallel TCP flows will become unresponsive and
increase queuing delay.
Unless scalable congestion controls address this requirement from the
start, they will frequently become unresponsive, negating the low
latency benefit of L4S, for themselves and for others.
That would seem to imply that scalable congestion controllers ought
to be required to be able work with a congestion window less than 1
SMSS. For instance, if an ECN-capable TCP gets an ECN-mark when it
is already sitting at a window of 1 SMSS, RFC 3168 requires it to
defer sending for a retransmission timeout. A less drastic but more
complex mechanism can maintain a congestion window less than 1 SMSS
(significantly less if necessary), as described in [Ahmed19]. Other
approaches are likely to be feasible.
However, the requirement in Section 4.3 is worded as a "SHOULD"
because the existence of a minimum window is not all bad. When
competing with an unresponsive flow, a minimum window naturally
protects the flow from starvation by at least keeping some data
flowing.
De Schepper & Briscoe Expires November 22, 2021 [Page 45]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
By stating the requirement to go lower than 1 SMSS as a "SHOULD",
while the requirement in RFC 3168 still stands as well, we shall be
able to watch the choices of minimum window evolve in different
scalable congestion controllers.
A.1.8. Measuring Reordering Tolerance in Time Units
Description: When detecting loss, a scalable congestion control needs
to be tolerant to reordering over an adaptive time interval, which
scales with throughput, rather than counting only in fixed units of
packets, which does not scale (see the precise normative requirement
wording in Section 4.3).
Motivation: A primary purpose of L4S is scalable throughput (it's in
the name). Scalability in all dimensions is, of course, also a goal
of all IETF technology. The inverse linear congestion response in
Section 4.3 is necessary, but not sufficient, to solve the congestion
control scalability problem identified in [RFC3649]. As well as
maintaining frequent ECN signals as rate scales, it is also important
to ensure that a potentially false perception of loss does not limit
throughput scaling.
End-systems cannot know whether a missing packet is due to loss or
reordering, except in hindsight - if it appears later. So they can
only deem that there has been a loss if a gap in the sequence space
has not been filled, either after a certain number of subsequent
packets has arrived (e.g. the 3 DupACK rule of standard TCP
congestion control [RFC5681]) or after a certain amount of time
(e.g. the RACK approach [RFC8985]).
As we attempt to scale packet rate over the years:
o Even if only _some_ sending hosts still deem that loss has
occurred by counting reordered packets, _all_ networks will have
to keep reducing the time over which they keep packets in order.
If some link technologies keep the time within which reordering
occurs roughly unchanged, then loss over these links, as perceived
by these hosts, will appear to continually rise over the years.
o In contrast, if all senders detect loss in units of time, the time
over which the network has to keep packets in order stays roughly
invariant.
Therefore hosts have an incentive to detect loss in time units (so as
not to fool themselves too often into detecting losses when there are
none). And for hosts that are changing their congestion control
implementation to L4S, there is no downside to including time-based
loss detection code in the change (loss recovery implemented in
De Schepper & Briscoe Expires November 22, 2021 [Page 46]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
hardware is an exception, covered later). Therefore requiring L4S
hosts to detect loss in time-based units would not be a burden.
If this requirement is not placed on L4S hosts, even though it would
be no burden on them to do so, all networks will face unnecessary
uncertainty over whether some L4S hosts might be detecting loss by
counting packets. Then _all_ link technologies will have to
unnecessarily keep reducing the time within which reordering occurs.
That is not a problem for some link technologies, but it becomes
increasingly challenging for other link technologies to continue to
scale, particularly those relying on channel bonding for scaling,
such as LTE, 5G and DOCSIS.
Given Internet paths traverse many link technologies, any scaling
limit for these more challenging access link technologies would
become a scaling limit for the Internet as a whole.
It might be asked how it helps to place this loss detection
requirement only on L4S hosts, because networks will still face
uncertainty over whether non-L4S flows are detecting loss by counting
DupACKs. The answer is that those link technologies for which it is
challenging to keep squeezing the reordering time will only need to
do so for non-L4S traffic (which they can do because the L4S
identifier is visible at the IP layer). Therefore, they can focus
their processing and memory resources into scaling non-L4S (Classic)
traffic. Then, the higher the proportion of L4S traffic, the less of
a scaling challenge they will have.
To summarize, there is no reason for L4S hosts not to be part of the
solution instead of part of the problem.
Requirement ("MUST") or recommendation ("SHOULD")? As explained
above, this is a subtle interoperability issue between hosts and
networks, which seems to need a "MUST". Unless networks can be
certain that all L4S hosts follow the time-based approach, they still
have to cater for the worst case - continually squeeze reordering
into a smaller and smaller duration - just for hosts that might be
using the counting approach. However, it was decided to express this
as a recommendation, using "SHOULD". The main justification was that
networks can still be fairly certain that L4S hosts will follow this
recommendation, because following it offers only gain and no pain.
Details:
The speed of loss recovery is much more significant for short flows
than long, therefore a good compromise is to adapt the reordering
window; from a small fraction of the RTT at the start of a flow, to a
De Schepper & Briscoe Expires November 22, 2021 [Page 47]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
larger fraction of the RTT for flows that continue for many round
trips.
This is broadly the approach adopted by TCP RACK (Recent
ACKnowledgements) [RFC8985]. However, RACK starts with the 3 DupACK
approach, because the RTT estimate is not necessarily stable. As
long as the initial window is paced, such initial use of 3 DupACK
counting would amount to time-based loss detection and therefore
would satisfy the time-based loss detection recommendation of
Section 4.3. This is because pacing of the initial window would
ensure that 3 DupACKs early in the connection would be spread over a
small fraction of the round trip.
As mentioned above, hardware implementations of loss recovery using
DupACK counting exist (e.g. some implementations of RoCEv2 for RDMA).
For low latency, these implementations can change their congestion
control to implement L4S, because the congestion control (as distinct
from loss recovery) is implemented in software. But they cannot
easily satisfy this loss recovery requirement. However, it is
believed they do not need to, because such implementations are
believed to solely exist in controlled environments, where the
network technology keeps reordering extremely low anyway. This is
why controlled environments with hardly any reordering are excluded
from the scope of the normative recommendation in Section 4.3.
Detecting loss in time units also prevents the ACK-splitting attacks
described in [Savage-TCP].
A.2. Scalable Transport Protocol Optimizations
A.2.1. Setting ECT in Control Packets and Retransmissions
Description: This item concerns TCP and its derivatives (e.g. SCTP)
as well as RTP/RTCP [RFC6679]. The original specification of ECN for
TCP precluded the use of ECN on control packets and retransmissions.
To improve performance, scalable transport protocols ought to enable
ECN at the IP layer in TCP control packets (SYN, SYN-ACK, pure ACKs,
etc.) and in retransmitted packets. The same is true for derivatives
of TCP, e.g. SCTP. Similarly [RFC6679] precludes the use of ECT on
RTCP datagrams, in case the path changes after it has been checked
for ECN traversal.
Motivation (TCP): RFC 3168 prohibits the use of ECN on these types of
TCP packet, based on a number of arguments. This means these packets
are not protected from congestion loss by ECN, which considerably
harms performance, particularly for short flows.
[I-D.ietf-tcpm-generalized-ecn] proposes experimental use of ECN on
all types of TCP packet as long as AccECN feedback
De Schepper & Briscoe Expires November 22, 2021 [Page 48]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
[I-D.ietf-tcpm-accurate-ecn] is available (which itself satisfies the
accurate feedback requirement in Section 4.2 for using a scalable
congestion control).
Motivation (RTCP): L4S experiments in general will need to observe
the rule in [RFC6679] that precludes ECT on RTCP datagrams.
Nonetheless, as ECN usage becomes more widespread, it would be useful
to conduct specific experiments with ECN-capable RTCP to gather data
on whether such caution is necessary.
A.2.2. Faster than Additive Increase
Description: It would improve performance if scalable congestion
controls did not limit their congestion window increase to the
standard additive increase of 1 SMSS per round trip [RFC5681] during
congestion avoidance. The same is true for derivatives of TCP
congestion control, including similar approaches used for real-time
media.
Motivation: As currently defined [RFC8257], DCTCP uses the
traditional Reno additive increase in congestion avoidance phase.
When the available capacity suddenly increases (e.g. when another
flow finishes, or if radio capacity increases) it can take very many
round trips to take advantage of the new capacity. TCP Cubic was
designed to solve this problem, but as flow rates have continued to
increase, the delay accelerating into available capacity has become
prohibitive. See, for instance, the examples in Section 1.2. Even
when out of its Reno-compatibility mode, every 8x scaling of Cubic's
flow rate leads to 2x more acceleration delay.
In the steady state, DCTCP induces about 2 ECN marks per round trip,
so it is possible to quickly detect when these signals have
disappeared and seek available capacity more rapidly, while
minimizing the impact on other flows (Classic and scalable)
[LinuxPacedChirping]. Alternatively, approaches such as Adaptive
Acceleration (A2DTCP [A2DTCP]) have been proposed to address this
problem in data centres, which might be deployable over the public
Internet.
A.2.3. Faster Convergence at Flow Start
Description: It would improve performance if scalable congestion
controls converged (reached their steady-state share of the capacity)
faster than Classic congestion controls or at least no slower. This
affects the flow start behaviour of any L4S congestion control
derived from a Classic transport that uses TCP slow start, including
those for real-time media.
De Schepper & Briscoe Expires November 22, 2021 [Page 49]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Motivation: As an example, a new DCTCP flow takes longer than a
Classic congestion control to obtain its share of the capacity of the
bottleneck when there are already ongoing flows using the bottleneck
capacity. In a data centre environment DCTCP takes about a factor of
1.5 to 2 longer to converge due to the much higher typical level of
ECN marking that DCTCP background traffic induces, which causes new
flows to exit slow start early [Alizadeh-stability]. In testing for
use over the public Internet the convergence time of DCTCP relative
to a regular loss-based TCP slow start is even less favourable
[Paced-Chirping] due to the shallow ECN marking threshold needed for
L4S. It is exacerbated by the typically greater mismatch between the
link rate of the sending host and typical Internet access
bottlenecks. This problem is detrimental in general, but would
particularly harm the performance of short flows relative to Classic
congestion controls.
Appendix B. Compromises in the Choice of L4S Identifier
This appendix is informative, not normative. As explained in
Section 2, there is insufficient space in the IP header (v4 or v6) to
fully accommodate every requirement. So the choice of L4S identifier
involves tradeoffs. This appendix records the pros and cons of the
choice that was made.
Non-normative recap of the chosen codepoint scheme:
Packets with ECT(1) and conditionally packets with CE signify L4S
semantics as an alternative to the semantics of Classic ECN
[RFC3168], specifically:
* The ECT(1) codepoint signifies that the packet was sent by an
L4S-capable sender.
* Given shortage of codepoints, both L4S and Classic ECN sides of
an AQM have to use the same CE codepoint to indicate that a
packet has experienced congestion. If a packet that had
already been marked CE in an upstream buffer arrived at a
subsequent AQM, this AQM would then have to guess whether to
classify CE packets as L4S or Classic ECN. Choosing the L4S
treatment is a safer choice, because then a few Classic packets
might arrive early, rather than a few L4S packets arriving
late.
* Additional information might be available if the classifier
were transport-aware. Then it could classify a CE packet for
Classic ECN treatment if the most recent ECT packet in the same
flow had been marked ECT(0). However, the L4S service ought
not to need tranport-layer awareness.
De Schepper & Briscoe Expires November 22, 2021 [Page 50]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Cons:
Consumes the last ECN codepoint: The L4S service could potentially
supersede the service provided by Classic ECN, therefore using
ECT(1) to identify L4S packets could ultimately mean that the
ECT(0) codepoint was 'wasted' purely to distinguish one form of
ECN from its successor.
ECN hard in some lower layers: It is not always possible to support
the equivalent of an IP-ECN field in an AQM acting in a buffer
below the IP layer [I-D.ietf-tsvwg-ecn-encap-guidelines]. Then,
depending on the lower layer scheme, the L4S service might have to
drop rather than mark frames even though they might encapsulate an
ECN-capable packet.
Risk of reordering Classic CE packets within a flow: Classifying all
CE packets into the L4S queue risks any CE packets that were
originally ECT(0) being incorrectly classified as L4S. If there
were delay in the Classic queue, these incorrectly classified CE
packets would arrive early, which is a form of reordering.
Reordering within a microflow can cause TCP senders (and senders
of similar transports) to retransmit spuriously. However, the
risk of spurious retransmissions would be extremely low for the
following reasons:
1. It is quite unusual to experience queuing at more than one
bottleneck on the same path (the available capacities have to
be identical).
2. In only a subset of these unusual cases would the first
bottleneck support Classic ECN marking while the second
supported L4S ECN marking, which would be the only scenario
where some ECT(0) packets could be CE marked by an AQM
supporting Classic ECN then the remainder experienced further
delay through the Classic side of a subsequent L4S DualQ AQM.
3. Even then, when a few packets are delivered early, it takes
very unusual conditions to cause a spurious retransmission, in
contrast to when some packets are delivered late. The first
bottleneck has to apply CE-marks to at least N contiguous
packets and the second bottleneck has to inject an
uninterrupted sequence of at least N of these packets between
two packets earlier in the stream (where N is the reordering
window that the transport protocol allows before it considers
a packet is lost).
For example consider N=3, and consider the sequence of
packets 100, 101, 102, 103,... and imagine that packets
De Schepper & Briscoe Expires November 22, 2021 [Page 51]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
150,151,152 from later in the flow are injected as follows:
100, 150, 151, 101, 152, 102, 103... If this were late
reordering, even one packet arriving out of sequence would
trigger a spurious retransmission, but there is no spurious
retransmission here with early reordering, because packet
101 moves the cumulative ACK counter forward before 3
packets have arrived out of order. Later, when packets
148, 149, 153... arrive, even though there is a 3-packet
hole, there will be no problem, because the packets to fill
the hole are already in the receive buffer.
4. Even with the current TCP recommendation of N=3 [RFC5681]
spurious retransmissions will be unlikely for all the above
reasons. As RACK [RFC8985] is becoming widely deployed, it
tends to adapt its reordering window to a larger value of N,
which will make the chance of a contiguous sequence of N early
arrivals vanishingly small.
5. Even a run of 2 CE marks within a Classic ECN flow is
unlikely, given FQ-CoDel is the only known widely deployed AQM
that supports Classic ECN marking and it takes great care to
separate out flows and to space any markings evenly along each
flow.
It is extremely unlikely that the above set of 5 eventualities
that are each unusual in themselves would all happen
simultaneously. But, even if they did, the consequences would
hardly be dire: the odd spurious fast retransmission. Whenever
the traffic source (a Classic congestion control) mistakes the
reordering of a string of CE marks for a loss, one might think
that it will reduce its congestion window as well as emitting a
spurious retransmission. However, it would have already reduced
its congestion window when the CE markings arrived early. If it
is using ABE [RFC8511], it might reduce cwnd a little more for a
loss than for a CE mark. But it will revert that reduction once
it detects that the retransmission was spurious.
In conclusion, the impact of early reordering on spurious
retransmissions due to CE being ambiguous will generally be
vanishingly small.
Pre-existing VPNs do not separate SAs by ECN: If delay is only
reduced for a subset of the flows within a VPN, the anti-replay
feature of VPNs is known to potentially mistake the difference in
delay for a replay attack. In IPsec [RFC4301], it is recommended
to classify traffic on its DSCP in order to split flows with
different delay into separate security associations (SA), and
therefore separate anti-replay windows. Section 6.2 recommends a
De Schepper & Briscoe Expires November 22, 2021 [Page 52]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
similar approach for the L4S experiment, by using two parallel
security associations (SAs) indexed by the least significant bit
of the IP-ECN field. However, classification on the ECN field is
unlikely to be supported by existing VPN software. Section 6.2
suggests alternative work-rounds, but end-users using L4S over a
VPN will need to be able to recognize the symptoms of this
problem, in order to seek out these work-rounds.
Hard to distinguish Classic ECN AQM: With this scheme, when a source
receives ECN feedback, it is not explicitly clear which type of
AQM generated the CE markings. This is not a problem for Classic
ECN sources that send ECT(0) packets, because an L4S AQM will
recognize the ECT(0) packets as Classic and apply the appropriate
Classic ECN marking behaviour.
However, in the absence of explicit disambiguation of the CE
markings, an L4S source needs to use heuristic techniques to work
out which type of congestion response to apply (see
Appendix A.1.5). Otherwise, if long-running Classic flow(s) are
sharing a Classic ECN AQM bottleneck with long-running L4S
flow(s), which then apply an L4S response to Classic CE signals,
the L4S flows would outcompete the Classic flow(s). Experiments
have shown that L4S flows can take about 20 times more capacity
share than equivalent Classic flows. Nonetheless, as link
capacity reduces (e.g. to 4 Mb/s), the inequality reduces. So
Classic flows always make progress and are not starved.
When L4S was first proposed (in 2015, 14 years after [RFC3168] was
published), it was believed that Classic ECN AQMs had failed to be
deployed, because research measurements had found little or no
evidence of CE marking. In subsequent years Classic ECN was
included in per-flow-queuing (FQ) deployments, however an FQ
scheduler stops an L4S flow outcompeting Classic, because it
enforces equality between flow rates. It is not known whether
there have been any non-FQ deployments of Classic ECN AQMs in the
subsequent years, or whether there will be in future.
An algorithm for detecting a Classic ECN AQM as soon as a flow
stabilizes after start-up has been proposed [ecn-fallback] (see
Appendix A.1.5 for a brief summary). Testbed evaluations of v2 of
the algorithm have shown detection is reasonably good for Classic
ECN AQMs, in a wide range of circumstances. However, although it
can correctly detect an L4S ECN AQM in many circumstances, its is
often incorrect at low link capacities and/or high RTTs. Although
this is the safe way round, there is a danger that it will
discourage use of the algorithm.
De Schepper & Briscoe Expires November 22, 2021 [Page 53]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Non-L4S service for control packets: Solely for the case of TCP, the
Classic ECN RFCs [RFC3168] and [RFC5562] require a sender to clear
the ECN field to Not-ECT on retransmissions and on certain control
packets specifically pure ACKs, window probes and SYNs. When L4S
packets are classified by the ECN field, these TCP control packets
would not be classified into an L4S queue, and could therefore be
delayed relative to the other packets in the flow. This would not
cause reordering (because retransmissions are already out of
order, and these control packets typically carry no data).
However, it would make critical TCP control packets more
vulnerable to loss and delay. To address this problem,
[I-D.ietf-tcpm-generalized-ecn] proposes an experiment in which
all TCP control packets and retransmissions are ECN-capable as
long as appropriate ECN feedback is available in each case.
Pros:
Should work e2e: The ECN field generally propagates end-to-end
across the Internet without being wiped or mangled, at least over
fixed networks. Unlike the DSCP, the setting of the ECN field is
at least meant to be forwarded unchanged by networks that do not
support ECN.
Should work in tunnels: The L4S identifiers work across and within
any tunnel that propagates the ECN field in any of the variant
ways it has been defined since ECN-tunneling was first specified
in the year 2001 [RFC3168]. However, it is likely that some
tunnels still do not implement ECN propagation at all.
Should work for many link technologies: At most, but not all, path
bottlenecks there is IP-awareness, so that L4S AQMs can be located
where the IP-ECN field can be manipulated. Bottlenecks at lower
layer nodes without IP-awareness either have to use drop to signal
congestion or a specific congestion notification facility has to
be defined for that link technology, including propagation to and
from IP-ECN. The programme to define these is progressing and in
each case so far the scheme already defined for ECN inherently
supports L4S as well (see Section 6.1).
Could migrate to one codepoint: If all Classic ECN senders
eventually evolve to use the L4S service, the ECT(0) codepoint
could be reused for some future purpose, but only once use of
ECT(0) packets had reduced to zero, or near-zero, which might
never happen.
L4 not required: Being based on the ECN field, this scheme does not
need the network to access transport layer flow identifiers.
Nonetheless, it does not preclude solutions that do.
De Schepper & Briscoe Expires November 22, 2021 [Page 54]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Appendix C. Potential Competing Uses for the ECT(1) Codepoint
The ECT(1) codepoint of the ECN field has already been assigned once
for the ECN nonce [RFC3540], which has now been categorized as
historic [RFC8311]. ECN is probably the only remaining field in the
Internet Protocol that is common to IPv4 and IPv6 and still has
potential to work end-to-end, with tunnels and with lower layers.
Therefore, ECT(1) should not be reassigned to a different
experimental use (L4S) without carefully assessing competing
potential uses. These fall into the following categories:
C.1. Integrity of Congestion Feedback
Receiving hosts can fool a sender into downloading faster by
suppressing feedback of ECN marks (or of losses if retransmissions
are not necessary or available otherwise).
The historic ECN nonce protocol [RFC3540] proposed that a TCP sender
could set either of ECT(0) or ECT(1) in each packet of a flow and
remember the sequence it had set. If any packet was lost or
congestion marked, the receiver would miss that bit of the sequence.
An ECN Nonce receiver had to feed back the least significant bit of
the sum, so it could not suppress feedback of a loss or mark without
a 50-50 chance of guessing the sum incorrectly.
It is highly unlikely that ECT(1) will be needed for integrity
protection in future. The ECN Nonce RFC [RFC3540] as been
reclassified as historic, partly because other ways have been
developed to protect feedback integrity of TCP and other transports
[RFC8311] that do not consume a codepoint in the IP header. For
instance:
o the sender can test the integrity of the receiver's feedback by
occasionally setting the IP-ECN field to a value normally only set
by the network. Then it can test whether the receiver's feedback
faithfully reports what it expects (see para 2 of Section 20.2 of
[RFC3168]. This works for loss and it will work for the accurate
ECN feedback [RFC7560] intended for L4S.
o A network can enforce a congestion response to its ECN markings
(or packet losses) by auditing congestion exposure (ConEx)
[RFC7713]. Whether the receiver or a downstream network is
suppressing congestion feedback or the sender is unresponsive to
the feedback, or both, ConEx audit can neutralise any advantage
that any of these three parties would otherwise gain.
o The TCP authentication option (TCP-AO [RFC5925]) can be used to
detect any tampering with TCP congestion feedback (whether
De Schepper & Briscoe Expires November 22, 2021 [Page 55]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
malicious or accidental). TCP's congestion feedback fields are
immutable end-to-end, so they are amenable to TCP-AO protection,
which covers the main TCP header and TCP options by default.
However, TCP-AO is often too brittle to use on many end-to-end
paths, where middleboxes can make verification fail in their
attempts to improve performance or security, e.g. by
resegmentation or shifting the sequence space.
C.2. Notification of Less Severe Congestion than CE
Various researchers have proposed to use ECT(1) as a less severe
congestion notification than CE, particularly to enable flows to fill
available capacity more quickly after an idle period, when another
flow departs or when a flow starts, e.g. VCP [VCP], Queue View (QV)
[QV].
Before assigning ECT(1) as an identifier for L4S, we must carefully
consider whether it might be better to hold ECT(1) in reserve for
future standardisation of rapid flow acceleration, which is an
important and enduring problem [RFC6077].
Pre-Congestion Notification (PCN) is another scheme that assigns
alternative semantics to the ECN field. It uses ECT(1) to signify a
less severe level of pre-congestion notification than CE [RFC6660].
However, the ECN field only takes on the PCN semantics if packets
carry a Diffserv codepoint defined to indicate PCN marking within a
controlled environment. PCN is required to be applied solely to the
outer header of a tunnel across the controlled region in order not to
interfere with any end-to-end use of the ECN field. Therefore a PCN
region on the path would not interfere with the L4S service
identifier defined in Section 3.
Authors' Addresses
Koen De Schepper
Nokia Bell Labs
Antwerp
Belgium
Email: koen.de_schepper@nokia.com
URI: https://www.bell-labs.com/usr/koen.de_schepper
De Schepper & Briscoe Expires November 22, 2021 [Page 56]
Internet-Draft L4S ECN Protocol for Very Low Queuing Delay May 2021
Bob Briscoe (editor)
Independent
UK
Email: ietf@bobbriscoe.net
URI: http://bobbriscoe.net/
De Schepper & Briscoe Expires November 22, 2021 [Page 57]