[Search] [txt|pdfized|bibtex] [Tracker] [Email] [Diff1] [Diff2] [Nits]
Versions: 00 01 02 03                                                   
Network Working Group                                       I. Johansson
Internet-Draft                                               Ericsson AB
Intended status: Informational                              May 30, 2017
Expires: December 1, 2017

                          ECN support in QUIC


   This memo outlines the ECN (Explicit Congestion Notification) support
   in QUIC.  The draft specifies the ECN negotiation and the ECN echo
   and in addition, different aspects of fallback in case of ECN failure
   as well as OS specific issues with ECN and monitoring for ECN
   capability.  The intention is that most of the material ends up
   updating other new or existing QUIC protocol specifications, thus it
   may be possible that this draft does not warrant a working group

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   document are to be interpreted as described in RFC 2119 [RFC2119].

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on December 1, 2017.

Copyright Notice

   Copyright (c) 2017 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

Johansson               Expires December 1, 2017                [Page 1]

Internet-Draft             ECN support in QUIC                  May 2017

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

   This document may contain material from IETF Documents or IETF
   Contributions published or made publicly available before November
   10, 2008.  The person(s) controlling the copyright in some of this
   material may not have granted the IETF Trust the right to allow
   modifications of such material outside the IETF Standards Process.
   Without obtaining an adequate license from the person(s) controlling
   the copyright in such materials, this document may not be modified
   outside the IETF Standards Process, and derivative works of it may
   not be created outside the IETF Standards Process, except to format
   it for publication as an RFC or to translate it into languages other
   than English.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Elements of ECN support . . . . . . . . . . . . . . . . . . .   3
     2.1.  ECN negotiation . . . . . . . . . . . . . . . . . . . . .   3
       2.1.1.  Challenge/Response  . . . . . . . . . . . . . . . . .   4
       2.1.2.  Determine degree of ECN support . . . . . . . . . . .   5
     2.2.  ECN bits in the IP header, semantics  . . . . . . . . . .   6
     2.3.  ECN echo  . . . . . . . . . . . . . . . . . . . . . . . .   6
     2.4.  Fallback in case of ECN fault . . . . . . . . . . . . . .   7
     2.5.  OS socket specifics, access to the ECN bits . . . . . . .   7
     2.6.  Monitoring  . . . . . . . . . . . . . . . . . . . . . . .   8
   3.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   8
   4.  Open questions  . . . . . . . . . . . . . . . . . . . . . . .   8
   5.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
   6.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   9
   7.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   9
     7.1.  Normative References  . . . . . . . . . . . . . . . . . .   9
     7.2.  Informative References  . . . . . . . . . . . . . . . . .   9
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  11

1.  Introduction

   ECN support in transport protocols is a fundamental feature that
   should be included in the QUIC specification as a mandatory element.
   ECN has the key benefit that it allows for non-destructive congestion

Johansson               Expires December 1, 2017                [Page 2]

Internet-Draft             ECN support in QUIC                  May 2017

   notification by network node, i.e packets are marked instead
   discarded.  This is particularly beneficial for realtime applications
   with requirements on latency, ECN also has the benefit that it
   provides with a congestion signal that is unambiguous.  The benefits
   with ECN is described in more detail in [I-D.ietf-aqm-ecn-benefits].
   The ECN support should be implemented to support both present and
   future ECN, the latter is outlined in
   [I-D.ietf-tsvwg-ecn-experimentation], of particular interest is the
   ability to discriminate between classic ECN and L4S ECN by means of
   differentiation between the use of the ECT(0) and ECT(1) code points.
   This draft does however not delve into the details of the congestion
   control implementation.

2.  Elements of ECN support

   This draft covers the following aspects of ECN support:

   o  ECN negotiation

   o  ECN echo

   o  ECN bits in the IP header, semantics

   o  Fallback in case of ECN fault

   o  OS socket specifics, access to the ECN bits

   o  Monitoring

2.1.  ECN negotiation

   ECN support in QUIC needs to be negotiated.  The reasons is that
   network elements may not support ECN and may either clear the ECN
   bits or simply discard packets that have the ECN bits set.  In
   addition, a QUIC implementation may not have access to the ECN bits
   in the IP header due to OS dependent restrictions, investigations
   (Piers O'Hanlon) have indicated that this is in certain cases an
   asymmetric property, for instance while it is possible to set the ECN
   bits it is not possible to read them.

   It is also required that the ECN negotiation does not interfere with
   the connection setup, in other words a failed ECN negotiation should
   not cause an extra roundtrip for the connection setup.

   The suggested method in this draft is to send an ECN negotiation
   frame when connection setup is completed.  Both peers MUST transmit
   the ECN negotiation frame.  The ECN negotiation frame is shown below.

Johansson               Expires December 1, 2017                [Page 3]

Internet-Draft             ECN support in QUIC                  May 2017

      0                   1
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
     |  Type         |C|R|W|U U U|E E|

                      Figure 1: ECN negotation frame

   The 2nd byte contains the flags:

   o  C: Challenge bit, indicates that the transmitted ECN negotiation
      frame is a challenge, if bit is not set then it is a response.

   o  R: Possible to read ECN bits in IP header

   o  W: Possible to write ECN bits in IP header

   o  EE : Echo of ECN bits

   o  U: Unused

   The ECN negotiation has two steps.

   o  Challenge/response

   o  Determine degree of ECN support

2.1.1.  Challenge/Response

   A peer transmits the ECN negotiation frame with the R,W and EE bits
   in the 2nd byte set to '0' and the C bit set to '1'.  This frame is
   echoed back with the flags set according to the degree of ECN support
   and with the ECN bits in the IP header of the received ECN
   negotiation frame copied to the EE field, the C bit is '0'.  As both
   peers MUST transmit an ECN negotiation frame there will be a total of
   4 ECN negotiation frames transmitted, two challenges and two

   An ECN negotiation frame should be transmitted in a unique packet,
   this to avoid that possible loss of ECN negotiation packets cause
   loss of other frames than the ECN negotiation frame.

   The IP header for the ECN negotiation frame should set the ECN bits
   to CE '11'.  When the corresponding response is received then an EE
   pattern of '11' indicates that ECN is likely supported in the
   network.  This does not give a full guarantee that ECN is supported
   in the network.  Monitoring of the ECN field in the ACK-frame serves
   to give further indication of ECN support once ECN is turned on.

Johansson               Expires December 1, 2017                [Page 4]

Internet-Draft             ECN support in QUIC                  May 2017

   An ECN negotiation is declared successful when an ECN negotiation
   response is received that indicates ECN support.  A peer is not
   allowed to set ECT on outgoing data packets until a successful ECN
   negotiation is done.  In other words it is only the ECN negotiation
   frame that is allowed to set the ECN bits in the IP header until ECN
   negotiation is concluded and successful.

   A lack of an ECN negotiation response may indicate that the ECN
   challenge frame or the ECN response frame was lost or that a node in
   the network deliberately discards ECN-CE marked packets.  The peer
   should transmit an additional ECN challenge within an RTO interval in
   case a negotiation response is not received, a maximum of
   retransmissions are attempted.

   A failed challenge/response phase indicates that ECN should not be
   used in the connection.  [NOTE, a special case is where one peer does
   not receive an ECN negotiation response but still receives ECT and CE
   marked packets from the other peer.  It is T.B.D how this should be

2.1.2.  Determine degree of ECN support

   If the ECN challenge/response is successful, the degree of ECN
   capability depends on how the R, W and EE bits are set.

   o  R='1' and EE= '11': It is possible to set the ECN bits in outgoing

   o  R='0' or EE <> '11': ECN support is not certain as it is either
      not possible for remote peer to read the ECN bits or that the ECN
      bits are altered.

   o  W='1' : It is meaningful to send ECN feedback

   o  W='0' : It is not meaningful to send ECN feedback as the remote
      peer cannot set (write) the ECN bits in the IP header.

   The mode mechanism in [RFC6679] can serve as in input to a solution
   for the support of ECN in the case that OS ECN support is asymmetric.
   It is however unclear how a QUIC implementation can determine
   asymmetric ECN support in the underlying OS.  For instance the method
   to send ECN marked packets to the local host to determine OS support
   does not reveal if the OS ECN support is asymmetric.

Johansson               Expires December 1, 2017                [Page 5]

Internet-Draft             ECN support in QUIC                  May 2017

2.2.  ECN bits in the IP header, semantics

   The ECN bits in the IP header should be set according to the
   recommendations in [I-D.ietf-tsvwg-ecn-experimentation].  This means
   that the meaning of ECT(0) and ECT(1) differ.

2.3.  ECN echo

   The ECN echo should go into the ACK frame [I-D.ietf-quic-transport],
   this is beneficial as the ECN information can then use some of the
   already existing data in the ACK frame for improved efficiency.

   The proposed alternative use one byte to encode how many bits that
   encode each of the ECT/CE fields.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     |              First Ack Block Length (8/16/32/48)            ...
     |  [Gap 1 (8)]  |       [Ack Block 1 Length (8/16/32/48)]     ...
     |  [Gap 2 (8)]  |       [Ack Block 2 Length (8/16/32/48)]     ...
     |  [Gap N (8)]  |       [Ack Block N Length (8/16/32/48)]     ...
     |R|R|E1 |E2 |CE | # ECT(0) bytes (0/16/32/48)                 ...
     |  # ECT(1) bytes (0/16/32/48)                                ...
     |  # ECN-CE bytes (0/16/32/48)                                ...

                Figure 2: ECN field in ACK frame ACK block

   The E1,E2 and CE fields indicate the length of each encoding for the
   number of ECT(0), ECT(1) and ECN-CE marked bytes.  This is encoded

   o  00: 0 bits

   o  01: 16 bits

   o  10: 32 bits

   o  11: 48bits

Johansson               Expires December 1, 2017                [Page 6]

Internet-Draft             ECN support in QUIC                  May 2017

   R indicates reserved bits.

   The proposed encoding enables flexible encoding of the ECN
   information, with a minimal 1 octet overhead for the cases where ECN
   is not supported by the connection.

2.4.  Fallback in case of ECN fault

   ECN can be subject to issues in network equipment, such as remarking
   to Not-ECN, remarking from ECT(0) to ECT(1) and vice versa or
   constant remarking to ECN-CE.  Furthermore ECT marked packets may be
   discarded in the network.  While these problems seem to be rare, see
   for instance [McQuistin-Perkins]and [APPLE-ECN], it is still
   necessary to safeguard against such problems.

   A peer should disable ECN for its outgoing packets if ECN fault is
   detected, it is however still possible for the other peer to use ECN.

   TODO add more information as regards to how to detect network ECN
   faults.  [ECN-fallback](expired) gives a few examples for fault
   detection.  Examples on how to detect ECN faults include for instance
   the method to set ECT and CE for outgoing packets according to a
   given pattern.

   Fallback in case of ECN faults is not an issue only for QUIC, it is
   here suggested that mechanisms for this is described in a non QUIC
   related draft, for instance in TSVWG.

2.5.  OS socket specifics, access to the ECN bits

   ECN support in QUIC comes with the additional challenge that it is
   necessary to somehow access the ECN bits in the IP headers.  In TCP
   this is provided without major concerns as TCP is generally
   implemented in OS kernel space.  QUIC can however be implemented both
   in user space or kernel space and is layered on top of UDP, which
   means that access to the ECN bits is not a given, instead various
   tricks are needed.

   The text below is copy-pasted from [OHanlon].

   "To set ECN on Linux, BSD and OSX one can use IP_TOS socket option,
   with the setsockopt() call, to set the relevant ECN bits of the TOS
   byte.  On Windows one can use a similar technique though firstly one
   has to enable TOS byte setting by enabling a particular Registry key
   ( DisableUserTOSSetting=0 (see https://msdn.microsoft.com/en-
   us/library/windows/desktop/dd874008%28v=vs.85%29.aspx One could also
   probably use the libpcap write functionality."

Johansson               Expires December 1, 2017                [Page 7]

Internet-Draft             ECN support in QUIC                  May 2017

   "To obtain the ECN bits from a packet one needs a mechanism to
   retrieve the ECN bits from each packet.  On Linux, one needs to
   firstly set the IP_RECVTOS socket option on the receiving socket, and
   use the recvmsg() call to receive a packet, and then retrieve the TOS
   byte from the associated csmg structure returned by the recvmsg()
   call.  This still works with linux-4.2.3.  On OSX/BSD there are no
   suitable socket options to retrieve the ECN/TOS bits and one cannot
   use raw sockets as they do not function for UDP/TCP sockets (they do
   work with ICMP), so one has to use alternatives such the bpf
   interface, or a REDIRECT socket.  Whilst on Windows it seems that the
   only way to retrieve the ECN bits is via a raw socket, or custom NDIS
   driver, though it's possible there's an API I'm missing."

   TODO: Write a more detailed description on how to implement ECN
   support in QUIC for different OS stacks.

2.6.  Monitoring

   A QUIC implementation should monitor the ECN functionality in order
   to provide input to e.g. service providers to improve ECN support in
   the networks.  Items of interest are:

   o  Black holes, ECT or CE marked packets are discarded.

   o  Faulty remarking, e.g.  ECT(0) is remarked to ECT(1) or Not-ECT.

   o  Continuous CE marking, possible indication of faulty on/off ECN
      marking, but can also be an effect of severe congestion.

   o  Degree of L4S support.  L4S should generally give low queue
      latency.  Estimation of one way queue delay for L4S enabled QUIC
      connections can be used to determine if there are congested nodes
      along the path that are not L4S capable.

3.  IANA Considerations


4.  Open questions

   A list of open questions:

   o  Is it sufficient that one peer sends an ECN negotiation challenge

   o  Should all packets be ECT or should there be special patterns to
      improve fault detection.

Johansson               Expires December 1, 2017                [Page 8]

Internet-Draft             ECN support in QUIC                  May 2017

   o  Write up a more detailed description on how to implement ECN
      support in QUIC for different OS stacks.

   o  Is a completely new ACK frame an alternative ?

   o  Should amount on ECT(0), ECT(1) and CE marked bytes account for
      the IP+UDP headers or is it only the QUIC header + data that
      counts ?

   o  Outline possible connection migration actions

   o  Are there any security implications with the small ECN negotiation
      frame ?

5.  Security Considerations


6.  Acknowledgements

   The following persons have contributed with comments and suggestions
   for improvements: Mirja Kuehlewind, Koen De Schepper, Piers O'Hanlon,
   Michael Welzl, Marcelo Bagnulo Braun, Martin Duke

7.  References

7.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,

7.2.  Informative References

              Apple Inc., "TCP ECN: Experience with Enabling ECN on the
              Internet", <https://www.ietf.org/proceedings/98/slides/

   [Bagnulo]  "Adding Explicit Congestion Notification (ECN) to TCP
              control packets and TCP retransmissions",

Johansson               Expires December 1, 2017                [Page 9]

Internet-Draft             ECN support in QUIC                  May 2017

              "A Mechanism for ECN Path Probing and Fallback",

              Fairhurst, G. and M. Welzl, "The Benefits of using
              Explicit Congestion Notification (ECN)", draft-ietf-aqm-
              ecn-benefits-08 (work in progress), November 2015.

              Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed
              and Secure Transport", draft-ietf-quic-transport-03 (work
              in progress), May 2017.

              Black, D., "Explicit Congestion Notification (ECN)
              Experimentation", draft-ietf-tsvwg-ecn-experimentation-02
              (work in progress), April 2017.

              ""Is Explicit Congestion Notification usable with UDP?",
              Proceedings of the ACM Internet Measurement Conference,
              Tokyo, Japan, October 2015. DOI:10.1145/2815675.2815716",

   [OHanlon]  "ECN support in different OS stacks",

   [RFC6679]  Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P.,
              and K. Carlberg, "Explicit Congestion Notification (ECN)
              for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August
              2012, <http://www.rfc-editor.org/info/rfc6679>.

   [RFC6789]  Briscoe, B., Ed., Woundy, R., Ed., and A. Cooper, Ed.,
              "Congestion Exposure (ConEx) Concepts and Use Cases",
              RFC 6789, DOI 10.17487/RFC6789, December 2012,

   [RFC7560]  Kuehlewind, M., Ed., Scheffenegger, R., and B. Briscoe,
              "Problem Statement and Requirements for Increased Accuracy
              in Explicit Congestion Notification (ECN) Feedback",
              RFC 7560, DOI 10.17487/RFC7560, August 2015,

Johansson               Expires December 1, 2017               [Page 10]

Internet-Draft             ECN support in QUIC                  May 2017

Author's Address

   Ingemar Johansson
   Ericsson AB
   Laboratoriegraend 11
   Luleaa  977 53

   Phone: +46 730783289
   Email: ingemar.s.johansson@ericsson.com

Johansson               Expires December 1, 2017               [Page 11]