Delayed Acknowledgments Suppression: Problem Statement, Requirements and Analysis of Potential Solutions
draft-gomez-tcpm-delack-suppr-reqs-00

TCPM Working Group                                              C. Gomez
Internet-Draft                                                       UPC
Intended status: Informational                              J. Crowcroft
Expires: September 8, 2020                       University of Cambridge
                                                           March 7, 2020

Delayed Acknowledgments Suppression: Problem Statement, Requirements and
                    Analysis of Potential Solutions
                 draft-gomez-tcpm-delack-suppr-reqs-00

Abstract

   Delayed Acknowledgments (ACKs) allow reducing protocol overhead in
   many scenarios.  However, in some cases, Delayed ACKs may
   significantly degrade network and device performance in terms of link
   utilization, latency, memory usage and/or energy consumption.  This
   document presents the problem statement regarding Delayed ACKs
   suppression.  The document discusses the scenarios and use cases in
   which Delayed ACKs suppression offers advantages.  Then, requirements
   for a potential solution are derived.  Finally, a number of potential
   solutions are discussed, based on the requirements, and also
   considering pros and cons in each case.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 8, 2020.

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents

Gomez & Crowcroft       Expires September 8, 2020               [Page 1]
Internet-Draft           Delayed ACK suppression              March 2020

   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Conventions used in this document . . . . . . . . . . . . . .   3
   3.  Problem statement: issues due to Delayed ACKs . . . . . . . .   3
     3.1.  Slow start  . . . . . . . . . . . . . . . . . . . . . . .   3
     3.2.  High bit rate environments and short data segments  . . .   4
     3.3.  IoT scenarios . . . . . . . . . . . . . . . . . . . . . .   4
     3.4.  Beyond classic ACK transmission behavior  . . . . . . . .   4
   4.  Requirements for Delayed ACKs suppression . . . . . . . . . .   5
     4.1.  Sender-triggered mechanism  . . . . . . . . . . . . . . .   5
     4.2.  Per-segment granularity . . . . . . . . . . . . . . . . .   5
     4.3.  Header/Message overhead . . . . . . . . . . . . . . . . .   5
     4.4.  Support for enabling generic ACK ratios . . . . . . . . .   5
     4.5.  Middlebox traversal . . . . . . . . . . . . . . . . . . .   6
     4.6.  Safe return to normal Delayed ACKs operation  . . . . . .   6
     4.7.  Impact on existing TCP functionality  . . . . . . . . . .   6
     4.8.  Impact on future TCP development  . . . . . . . . . . . .   6
     4.9.  Avoidance of 'hacks'  . . . . . . . . . . . . . . . . . .   6
   5.  Potential solutions for Delayed ACKs suppression  . . . . . .   6
     5.1.  AckCC . . . . . . . . . . . . . . . . . . . . . . . . . .   7
     5.2.  TLP . . . . . . . . . . . . . . . . . . . . . . . . . . .   7
     5.3.  TCP ACK Pull (AKP) flag . . . . . . . . . . . . . . . . .   7
     5.4.  A new 'ACK Pull' TCP option . . . . . . . . . . . . . . .   8
     5.5.  Reuse of existing TCP header fields . . . . . . . . . . .   8
     5.6.  'Hacks' . . . . . . . . . . . . . . . . . . . . . . . . .   8
   6.  Summary . . . . . . . . . . . . . . . . . . . . . . . . . . .   9
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
   8.  Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .   9
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  10
     9.1.  Normative References  . . . . . . . . . . . . . . . . . .  10
     9.2.  Informative References  . . . . . . . . . . . . . . . . .  10
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  11

1.  Introduction

   Delayed Acknowledgments (ACKs) were specified with the aim to reduce
   protocol overhead [RFC1122].  With Delayed ACKs, a TCP delays sending
   an ACK by up to 500 ms (often 200 ms, with lower values in recent
   implementations such as ~50 ms also reported), and typically sends an

Gomez & Crowcroft       Expires September 8, 2020               [Page 2]
Internet-Draft           Delayed ACK suppression              March 2020

   ACK for at least every second segment received in a stream of full-
   sized segments.  This allows combining several segments into a single
   one (e.g. the application layer response to an application layer data
   message, and the corresponding ACK), and it also saves up to one of
   every two ACKs under many traffic patterns (e.g. bulk transfers).
   The "SHOULD" requirement level for implementing Delayed ACKs in RFC
   1122, along with its expected benefits, has led to a widespread
   deployment of this mechanism.

   However, there exist traffic patterns and scenarios for which Delayed
   ACKs can actually be detrimental to performance.  When a segment
   carrying a message of a size up to one Maximum Segment Size (MSS) is
   transferred, if the message does not elicit an application-layer
   response, and a second data segment is not transferred earlier than
   the Delayed ACK timeout, the ACK is unnecessarily delayed, with a
   number of negative consequences.  Furthermore, there may be reasons
   to allow a sender communicate the ACK ratio to be used in a TCP
   connection, and thus dynamically override (or restore) use of Delayed
   ACKs at the receiver.

   This document presents the problem statement regarding Delayed ACKs
   suppression.  The document discusses the scenarios and use cases in
   which Delayed ACKs suppression offers advantages.  Then, requirements
   for a potential solution are derived.  Finally, a number of potential
   solutions are discussed, based on the requirements, and also
   considering pros and cons in each case.

2.  Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL","SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

3.  Problem statement: issues due to Delayed ACKs

   This section provides scenarios and use cases where performance
   issues arise due to Delayed ACKs.

3.1.  Slow start

   During slow start, the congestion window (cwnd) increases by up to
   Sender Maximum Segment Size (SMSS) upon receipt of an ACK covering
   new data [RFC5681].  However, use of Delayed ACKs reduces the amount
   of ACKs received by the sender, thus reducing the rate of cwnd
   growth, increasing transfer time and reducing throughput, when
   compared with sending an ACK for each incoming data segment.
   Furthermore, Delayed ACKs preclude using sender behaviors intended to
   'more rapidly get up to speed during slow-start without overshoot'

Gomez & Crowcroft       Expires September 8, 2020               [Page 3]
Internet-Draft           Delayed ACK suppression              March 2020

   [I-D.kuehlewind-tcpm-accurate-ecn] (see -03 of the referenced
   document).

3.2.  High bit rate environments and short data segments

   When the Nagle algorithm is used, in some cases the sender may be
   prevented from sending more data while awaiting a delayed ACK.  In
   some high bit rate environment (e.g.  Gigabit Ethernet) use cases,
   such a delay may be very large, and link utilitzation may be
   dramatically reduced, since the Delayed ACK timeout may be several
   orders of magnitude greater than the Round Trip Time (RTT) [RFC8490].

3.3.  IoT scenarios

   Delayed ACKs are also detrimental in Internet of Things (IoT)
   scenarios, where TCP is being increasingly used
   [I-D.ietf-lwig-tcp-constrained-node-networks].  Many IoT devices,
   such as sensors, transfer small messages (e.g. containing sensor
   readings) rather infrequently, therefore if the receiver uses Delayed
   ACKs, the ACK will often be unnecessarily delayed.  The sender cannot
   release the memory resources associated to a transferred data segment
   until the ACK is received and processed.  This may be a problem for
   many IoT devices, which are typically memory-constrained, and may
   even lead to subsequent packet drops if their scarce memory resources
   are blocked while awaiting an ACK.  Moreover, if the IoT device uses
   a radio interface for communication, in some scenarios Delayed ACKs
   will lead to increased energy consumption (e.g. with the radio
   interface of the device staying in receive mode while awaiting the
   ACK).  Since many IoT devices run on small batteries, the device
   lifetime may significantly decrease.  Furthermore, the delay suffered
   by the ACK may interact negatively with layer two mechanisms,
   especially in wireless network technologies where devices remain in
   low-power states for long intervals [RFC8352], potentially leading to
   a further exacerbated delay (by even one or more orders of
   magnitude).

3.4.  Beyond classic ACK transmission behavior

   In some scenarios, it may be desirable to enable ACK transmission
   behaviors beyond the classic ones.  For example, it may be beneficial
   to apply congestion control to ACKs [RFC5690].  For example, reducing
   the amount of ACKs on a congested reverse path may allow alleviating
   congestion on that path, with minimal impact on a relatively
   independent forward path.

Gomez & Crowcroft       Expires September 8, 2020               [Page 4]
Internet-Draft           Delayed ACK suppression              March 2020

4.  Requirements for Delayed ACKs suppression

   This section provides the requirements for a potential solution to
   enable Delayed ACKs suppression.

4.1.  Sender-triggered mechanism

   An assumption is that the sender knows when Delayed ACKs operation
   should be overriden.  For example, the sender may know in advance the
   pattern of the traffic it will generate, or it may know whether an
   application-layer response will be sent by the receiving endpoint
   upon reception of a given message.  Therefore, Delayed ACKs
   suppression has to be sender-triggered.

4.2.  Per-segment granularity

   One approach that cannot be recommended as a general solution for
   Delayed ACKs suppression is (permanently) disabling Delayed ACKs at
   the receiving TCP.  In fact, the latter may interact with a wide
   variety of devices and many of those may still benefit from the
   advantages of Delayed ACKs.  In addition, in some cases, a sender may
   offer a mixed traffic pattern comprising single data segments that
   will lead to unnecessarily delayed ACKs, with other data segments
   upon which Delayed ACKs will act as intended.  Therefore, the
   solution has to be provided at a per-segment granularity.

4.3.  Header/Message overhead

   Since the presented problem is about low performance in various
   scenarios, another requirement for the solution is to minimize
   incurring overhead in terms of header size increase or additional
   packets sent.  For example, in IoT scenarios, every additional
   communicated byte consumes scarce resources (e.g. energy, bandwidth
   and computational resources).

   Another benefit of keeping a low header/message overhead is
   alleviating the processing workload of a receiving TCP.

4.4.  Support for enabling generic ACK ratios

   For many of the scenarios and use cases described in Section 2, an
   ACK ratio of 1 (i.e. a receiver sending one ACK per incoming data
   segment) would solve the mentioned issues.  However, the ability of
   enforcing a generic ACK ratio (including values different from 1 and
   2) allows to enable a wider range of ACK behaviors, which may support
   congestion control for ACKs, sender behaviors not based on ACK-
   clocking, etc.

Gomez & Crowcroft       Expires September 8, 2020               [Page 5]
Internet-Draft           Delayed ACK suppression              March 2020

4.5.  Middlebox traversal

   Deployment of new functionality for TCP faces the risk of packets
   being discarded by existing middleboxes upon detection of unexpected
   or discouraged formats, header field values or even traffic patterns.

   A solution for Delayed ACK suppression should offer relatively good
   middlebox traversal (to the extent possible).

4.6.  Safe return to normal Delayed ACKs operation

   A solution for Delayed ACKs suppression must ensure that normal
   Delayed ACKs operation is in force by default, and also once
   temporary suppression of Delayed ACKs operation needs to end.

4.7.  Impact on existing TCP functionality

   A solution for Delayed ACKs suppression should not reduce the space
   of existing TCP functionality.

4.8.  Impact on future TCP development

   A solution for Delayed ACKs suppression should not pose significant
   risk of preventing future TCP development.  If an available resource
   (e.g. a reserved bit of the TCP header, a new TCP option, etc.) is
   used by a solution, careful analysis must be carried out regarding
   the risks and benefits of using such resource.

4.9.  Avoidance of 'hacks'

   Delayed ACKs suppression might be achieved by using workarounds, such
   as implementation techniques that may produce the desired effect.
   However, such approaches may be suboptimal regarding implementation
   cleanliness, and may entail other performance issues (see section
   5.6).

5.  Potential solutions for Delayed ACKs suppression

   This section enumerates and discusses potential solutions that might
   be considered to enable Delayed ACK suppression.  The list of
   solutions is not necessarily comprehensive.  This section intends to
   illustrate the trade-offs that arise when considering potential
   Delayed ACK suppression solutions.  (Note: the analysis needs to be
   completed for many of the solutions below.)

Gomez & Crowcroft       Expires September 8, 2020               [Page 6]
Internet-Draft           Delayed ACK suppression              March 2020

5.1.  AckCC

   In Acknowledgment Congestion Control (AckCC) [RFC5690], the sender
   tells the receiver the ACK ratio R to use, where the receiver sends
   one ACK per R data packets received.  AckCC defines a 2-byte "TCP ACK
   Congestion Control Permitted Option" for negotiating use of AckCC,
   whereas it defines a 3-byte "ACK ratio TCP option" to communicate the
   ACK Ratio value from the sender to the receiver.

   Middlebox traversal of a new TCP option is often regarded as 'bad'
   (to be confirmed).

5.2.  TLP

   Tail Loss Probe (TLP) [I-D.ietf-tcpm-rack] is intended to avoid RTO-
   expiration-based retransmission when tail loss occurs by inducing
   additional ACKs at the receiver.  This is achieved by sending a probe
   segment after a probe time-out (PTO) when data have been sent but not
   confirmed.  Of course, this means sending a whole new packet to
   trigger ACKs, which adds significant overhead.

   This approach might offer good middlebox traversal (to be confirmed).

5.3.  TCP ACK Pull (AKP) flag

   One solution that has been proposed for Delayed ACKs suppresion is
   called 'TCP ACK Pull' [I-D.gomez-tcpm-ack-pull].

   TCP ACK Pull defines the AKP flag as bit number 6 of the 13th byte of
   the TCP header.  When a TCP sender needs a data segment to be
   acknowledged by the receiving TCP without additional delay, the
   sender sets the AKP flag of the data segment TCP header.  Upon
   reception of a segment with the AKP flag set, a conforming receiving
   TCP behaves accordingly by sending the corresponding ACK without
   additional delay.

   This solution would entail zero header or message overhead.  However,
   it would consume a TCP header bit, leaving only two available TCP
   header reserved bits.  A question is thus whether one TCP header bit
   should be dedicated to this purpose or not.

   Middlebox traversal characteristics of bit 6 of the TCP header need
   to be assessed.

Gomez & Crowcroft       Expires September 8, 2020               [Page 7]
Internet-Draft           Delayed ACK suppression              March 2020

5.4.  A new 'ACK Pull' TCP option

   Another approach relies on defining a new option-kind-only TCP option
   with the same semantics as the AKP flag, which might be called 'ACK
   Pull Option' or 'AKP Option'.

   This solution would consume an available TCP Option Kind number.
   However, most of the 256 numbers in the TCP Option Kind number space
   are currently available.  Therefore, consuming one such number does
   not appear to significantly limit future TCP development.

   The header overhead of the AKP Option is one byte.

   Middlebox traversal of a new TCP option is often regarded as 'bad'
   (to be confirmed).

5.5.  Reuse of existing TCP header fields

   Another approach that might be used to enable Delayed ACKs
   suppression is based on reusing existing TCP header fields.  For
   example, use of the Urgent pointer has been suggested (e.g. by
   reserving 3 of its 16 bits to encode an ACK ratio exponent that may
   be communicated by the sender to the receiver), when URG=0.  A
   problem with this approach is that the semantics of the reused TCP
   header field may become overloaded.  Therefore, in some cases either
   the original intended use of the reused TCP header field may become
   limited, or if it prevails, then Delayed ACKs suppression might not
   always be available for use.

   Middlebox traversal characteristics of this approach might be
   relatively good (to be confirmed).

5.6.  'Hacks'

   One approach that allows eliciting an immediate ACK after sending a
   data segment is sending a subsequent segment carrying a previously
   acknowledged data byte.  However, in addition to the inefficiency of
   sending a byte that has previously been sent, this approach may
   require the transmission of a new packet (even carrying a single byte
   of data payload) just for that purpose, which represents significant
   overhead.  Furthermore, sending a previously sent byte is not a clean
   solution from an implementation perspective.

   Another workaround intended to trigger an immediate ACK from the
   receiving TCP, which is used in the Contiki operating system (a
   popular operating system for constrained devices in IoT scenarios) is
   splitting the data to be sent into two segments of smaller size.  A
   standard compliant TCP receiver will acknowledge the second MSS of

Gomez & Crowcroft       Expires September 8, 2020               [Page 8]
Internet-Draft           Delayed ACK suppression              March 2020

   data.  However, this 'split hack' may not always work since a TCP
   receiver is required to acknowledge every second full-sized segment,
   but not two consecutive small segments.  Furthermore, the overhead of
   sending two IP packets instead of one is another downside of the
   'split hack'.

6.  Summary

   The next table summarizes whether the different solutions presented
   in Section 4 are able to satisfy the requirements stated in
   Section 4.

                +-------+-------+-------+-------+-------+------+------+
                |  Per- | Over- |Generic|Middle-|Impact |Impact|Hack  |
                |segment| head  |ACK rat|box tr.|current|future|Avoid.|
   +------------+-------+-------+-------+-------+-------+------+------+
   |    ACKcc   |  Yes  |  Low  |  Yes  |  Bad? |  No   |  Low | Yes  |
   +------------+-------+-------+-------+-------+-------+------+------+
   |     TLP    |   No  | High  |   No  |  Good |  No   |  No  | Yes  |
   +------------+-------+-------+-------+-------+-------+------+------+
   |  AKP flag  |  Yes  |   No  |  Yes  |   ?   |  No   |Med/Hi| Yes  |
   +------------+-------+-------+-------+-------+-------+------+------+
   | AKP option |  Yes  |  Low  |  Yes  |  Bad? |  No   |  Low | Yes  |
   +------------+-------+-------+-------+-------+-------+------+------+
   |Reuse fields|  Yes  |   No  |  Yes  | Good? |  Yes  |   ?  | Yes  |
   +------------+-------+-------+-------+-------+-------+------+------+
   |   Hacks    |   ?   |Med/Hig|   No  | Good? |  No   |  No  |  No  |
   +------------+-------+-------+-------+-------+-------+------+------+

   Note: all considered potential solutions satisfy the following requirements:
         i) sender-triggered Delayed ACKs suppression, and ii) safe return to normal
         Delayed ACKs operation.

         Figure 1: Summary of potential solutions for Delayed ACKs
                               suppression.

7.  Security Considerations

   TBD

8.  Acknowledgments

   Bob Briscoe, Jonathan Morton, Richard Scheffenegger, Michael Tuexen
   and Jana Iyengar provided useful input for this document.

Gomez & Crowcroft       Expires September 8, 2020               [Page 9]
Internet-Draft           Delayed ACK suppression              March 2020

   Stuart Cheshire, Ted Lemon, Michael Scharf, and Christoph Paasch
   participated in a discussion that was seminal to the TCP ACK Pull
   proposal, which eventually led to this document.

   Carles Gomez has been funded in part by the Spanish Government
   (Ministerio de Ciencia, Innovacion y Universidades) through
   Secretaria d'Universitats i Recerca del Departament d'Empresa i
   Coneixement de la Generalitat de Catalunya 2017 SGR 376.

9.  References

9.1.  Normative References

   [RFC1122]  Braden, R., Ed., "Requirements for Internet Hosts -
              Communication Layers", STD 3, RFC 1122,
              DOI 10.17487/RFC1122, October 1989,
              <https://www.rfc-editor.org/info/rfc1122>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC5681]  Allman, M., Paxson, V., and E. Blanton, "TCP Congestion
              Control", RFC 5681, DOI 10.17487/RFC5681, September 2009,
              <https://www.rfc-editor.org/info/rfc5681>.

   [RFC5690]  Floyd, S., Arcia, A., Ros, D., and J. Iyengar, "Adding
              Acknowledgement Congestion Control to TCP", RFC 5690,
              DOI 10.17487/RFC5690, February 2010,
              <https://www.rfc-editor.org/info/rfc5690>.

9.2.  Informative References

   [I-D.gomez-tcpm-ack-pull]
              Gomez, C. and J. Crowcroft, "TCP ACK Pull", draft-gomez-
              tcpm-ack-pull-01 (work in progress), November 2019.

   [I-D.ietf-lwig-tcp-constrained-node-networks]
              Gomez, C., Crowcroft, J., and M. Scharf, "TCP Usage
              Guidance in the Internet of Things (IoT)", draft-ietf-
              lwig-tcp-constrained-node-networks-09 (work in progress),
              November 2019.

   [I-D.ietf-tcpm-rack]
              Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "RACK:
              a time-based fast loss detection algorithm for TCP",
              draft-ietf-tcpm-rack-07 (work in progress), January 2020.

Gomez & Crowcroft       Expires September 8, 2020              [Page 10]
Internet-Draft           Delayed ACK suppression              March 2020

   [I-D.kuehlewind-tcpm-accurate-ecn]
              Briscoe, B., Kuehlewind, M., and R. Scheffenegger, "More
              Accurate ECN Feedback in TCP", draft-kuehlewind-tcpm-
              accurate-ecn-05 (work in progress), October 2015.

   [RFC8352]  Gomez, C., Kovatsch, M., Tian, H., and Z. Cao, Ed.,
              "Energy-Efficient Features of Internet of Things
              Protocols", RFC 8352, DOI 10.17487/RFC8352, April 2018,
              <https://www.rfc-editor.org/info/rfc8352>.

   [RFC8490]  Bellis, R., Cheshire, S., Dickinson, J., Dickinson, S.,
              Lemon, T., and T. Pusateri, "DNS Stateful Operations",
              RFC 8490, DOI 10.17487/RFC8490, March 2019,
              <https://www.rfc-editor.org/info/rfc8490>.

Authors' Addresses

   Carles Gomez
   UPC
   C/Esteve Terradas, 7
   Castelldefels  08860
   Spain

   Email: carlesgo@entel.upc.edu

   Jon Crowcroft
   University of Cambridge
   JJ Thomson Avenue
   Cambridge, CB3 0FD
   United Kingdom

   Email: jon.crowcroft@cl.cam.ac.uk

Gomez & Crowcroft       Expires September 8, 2020              [Page 11]