Windowless Cumulative ACK Extension for RDMA Retransmission
draft-chen-rdma-windowless-ack-00

Internet Research Task Force                                     D. Chen
Internet-Draft                                                   H. Yang
Intended status: Informational                              China Mobile
Expires: 7 January 2027                                      6 July 2026

      Windowless Cumulative ACK Extension for RDMA Retransmission
                   draft-chen-rdma-windowless-ack-00

Abstract

   Traditional RDMA Reliable Connection (RC) uses selective
   retransmission, while TCP SACK depends on sliding window and RTO
   timer tuning.  Both mechanisms waste network bandwidth when window or
   RTO parameters mismatch real network conditions.  This document
   defines a windowless cumulative selective acknowledgement extension
   (T/C AETH) embedded within the RDMA ETH header, paired with dual-
   trigger ACK reporting logic driven by cumulative receive time
   threshold and cumulative received packet count threshold.

   The receiver generates extended SACK reports without sliding window
   constraints.  The sender accurately identifies lost PSN segments
   through multi-segment confirmation fields carried in T/C AETH, then
   only retransmits missing packets.  This design removes window
   dependency, eliminates redundant retransmissions, and improves
   bandwidth utilization and end-to-end throughput for high-speed RDMA
   workloads in data centers and wide-area networks.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

Chen & Yang              Expires 7 January 2027                 [Page 1]
Internet-Draft            Network Working Group                July 2026

   This Internet-Draft will expire on 7 January 2027.

Copyright Notice

   Copyright (c) 2026 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Mechanism Overview  . . . . . . . . . . . . . . . . . . . . .   4
   3.  Receiver Operation  . . . . . . . . . . . . . . . . . . . . .   5
   4.  Sender Operation  . . . . . . . . . . . . . . . . . . . . . .   6
   5.  T/C AETH Target Fragment Header Format  . . . . . . . . . . .   7
     5.1.  Binary Layout (30 Bytes Total)  . . . . . . . . . . . . .   7
     5.2.  Segment Mapping Visualization . . . . . . . . . . . . . .   8
   6.  Retransmission Decision . . . . . . . . . . . . . . . . . . .   8
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .   8
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   8
   9.  Normative References  . . . . . . . . . . . . . . . . . . . .   9
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   9

1.  Introduction

   Existing reliable transport retransmission schemes have two major
   categories with inherent defects:

      TCP SACK and Go-Back-N: Performance heavily relies on sliding
      window size and RTO timeout configuration.  Small window generates
      excessive reverse-path ACK overhead; oversized window triggers
      massive redundant retransmissions once packet loss occurs.

      Standard RDMA AETH SACK: Only supports a single continuous receive
      segment, cannot compact multiple discontinuous received PSN blocks
      into one ACK frame.

Chen & Yang              Expires 7 January 2027                 [Page 2]
Internet-Draft            Network Working Group                July 2026

   For high-throughput RDMA workloads including distributed AI training,
   distributed storage, and cross-DC data replication, minor mismatch
   between tuning parameters and actual network conditions significantly
   degrades link utilization.  This draft introduces two core
   innovations:

      Dual-trigger ACK reporting: The receiver sends extended SACK once
      either cumulative receive time or cumulative received packet count
      reaches preconfigured threshold; per-packet ACK is eliminated.

      Extended T/C AETH fragment inserted into RDMA ETH header: Carries
      multiple discontinuous receive segments, enabling precise
      selective retransmission without receiver-side sliding window
      maintenance.

   The proposed mechanism fully removes sliding window constraints.  The
   sender transmits data at a fixed preset line rate continuously, while
   the receiver batches loss feedback via extended ACK to minimize
   control-plane overhead on reverse path.

   This specification extends the RDMA ETH header defined in [RFC5040]
   by introducing a new Target Fragment (T/C AETH).  It defines end-to-
   end operation rules for RC QP pairs, applicable to RoCEv1, RoCEv2 and
   InfiniBand RC transport.  This draft does not modify RDMA data
   payload format or existing congestion control algorithms.

   Currently, the application of RDMA for data transmission in wide area
   networks encounters the following issues:

      Go-Back-N Limitation: If a single PSN is lost, the sender
      retransmits all outstanding packets after the last acknowledged
      PSN, resulting in severe bandwidth waste under non-zero packet
      loss rate.

      Sliding Window Overhead: Credit-based window requires frequent ACK
      feedback to advance transmission window.  Small window causes ACK
      flooding on reverse path; large window leads to bulk redundant
      retransmission upon loss.

      RTO Sensitivity: Underestimated RTO induces spurious
      retransmissions; overestimated RTO stalls throughput for long-
      latency cross-DC links.

      Single Segment AETH Restriction: Legacy RDMA AETH only reports one
      continuous receive range, cannot aggregate multiple discontinuous
      received blocks into a single ACK frame.

Chen & Yang              Expires 7 January 2027                 [Page 3]
Internet-Draft            Network Working Group                July 2026

   This draft resolves all above limitations via windowless transmission
   combined with multi-segment batch SACK reporting.

2.  Mechanism Overview

   The complete end-to-end workflow contains two symmetric operation
   planes:

      Receiver Plane (Section 5): Accumulate received PSNs, monitor dual
      time/count thresholds, construct T/C AETH extended ACK and
      transmit to sender when any threshold is hit.

      Sender Plane (Section 6): Transmit packets at fixed preset rate
      without sliding window limitation; parse multi-segment blocks in
      T/C AETH to extract missing PSN ranges, perform selective
      retransmission only for identified lost segments.

   Core technical advantages:

      Windowless sender transmission: No per-flow credit window tracking
      required on sender side.

      Batch ACK reporting: Dual-threshold batching drastically reduces
      reverse-path ACK traffic volume.

      Multi-segment compact SACK: Single ACK frame carries multiple
      discontinuous received PSN ranges to enable precise loss recovery.

   High-Level End-to-End Flow:

      Sender continuously transmits RDMA data packets at negotiated
      fixed preset rate.

      Receiver accumulates received data packets, tracks cumulative
      receive time and cumulative packet count.

      If cumulative time >= T_thresh OR cumulative count >= C_thresh: a.
      Receiver constructs RDMA ACK frame with embedded T/C AETH target
      fragment. b.  Transmit extended ACK (First Message) back to sender
      QP.

      Sender parses T/C AETH segments, calculates PSN gaps between all
      confirmed receive blocks.

      Sender selectively retransmits only PSN ranges identified as lost.

      Fallback timeout retransmission activates if sender receives no
      extended ACK after fixed timeout window expires.

Chen & Yang              Expires 7 January 2027                 [Page 4]
Internet-Draft            Network Working Group                July 2026

3.  Receiver Operation

   For threshold configuration, during RC QP setup handshake:

      Sender advertises its preset packet send rate to receiver via CM
      REQ/REP.

      Receiver computes and configures T_thresh (cumulative time
      threshold) and C_thresh (cumulative count threshold) proportional
      to sender transmission rate.

   Receiver maintains per-QP batch state variables:

      BatchStartTime: Timestamp of the first packet received in current
      batch cycle. - BatchPacketCount: Total valid unique PSNs received
      in current batch.

      SegmentList: Ordered list of continuous received PSN blocks,
      sorted by ascending PSN value.

   For each incoming valid data packet:

      Record PSN if packet is valid and not duplicate.

      Merge new PSN into existing continuous segment in SegmentList, or
      create a new standalone segment block.

      Increment BatchPacketCount counter by one.

      Calculate current cumulative receive time = CurrentTime -
      BatchStartTime.

   After batch receive state tracking, it transmits T/C AETH extended
   ACK when either condition evaluates to true:

      Condition A: Cumulative receive time >= T_thresh

      Condition B: Cumulative receive count >= C_thresh

   After sending extended ACK frame:

      Reset BatchStartTime to current system timestamp.

      Clear BatchPacketCount to zero.

      Flush SegmentList to empty for new batch collection cycle.

Chen & Yang              Expires 7 January 2027                 [Page 5]
Internet-Draft            Network Working Group                July 2026

   To complete the extended ACK transmission, the sender needs to
   construct an extended ACK frame.

      Base frame: Standard RDMA ACK ETH header sequence defined in
      RFC5040.

      Insert new T/C AETH Target Fragment between standard AETH and
      Atomic AETH fields inside ETH header.

      Populate all T/C AETH fields with data extracted from SegmentList,
      detailed format specified in Section 7.

      Encapsulate complete ACK frame and transmit to corresponding
      sender QP.

4.  Sender Operation

   In this draft, we design a windowless continuous transmission.  After
   QP establishment handshake completes:

      Sender transmits data packets at pre-negotiated fixed preset rate
      continuously.

      No sliding window credit check is executed before sending new
      outbound packets.

      Cache all transmitted PSNs within send buffer for potential
      selective retransmission processing.

   Upon receiving T/C AETH extended ACK frame:

      Parse SegmentNum field to read total number of valid receive
      confirmation segments stored inside Target Fragment.

      Extract Last Finished PSN, representing end PSN of first
      continuous fully received block.

      Extract each subsequent SegX Left / SegX Right pair representing
      discontinuous received PSN blocks.

      Compute all PSN gaps between adjacent confirmed segments; these
      gaps are defined as lost packet ranges.

      Trigger selective retransmission for all PSNs inside identified
      gap ranges only.

   The designed multi-segment loss calculation logic is as follows:

Chen & Yang              Expires 7 January 2027                 [Page 6]
Internet-Draft            Network Working Group                July 2026

      Case 1: SegmentNum = 1 (single continuous received block) All PSNs
      smaller than Last Finished PSN are fully received without loss.
      Lost packets = all PSNs greater than Last Finished PSN up to
      latest transmitted PSN.

      Case 2: SegmentNum > 1 (multiple discontinuous receive segments)
      Lost range 1: (Last Finished PSN, Seg1 Left PSN) Lost range N:
      (SegN Right PSN, SegN+1 Left PSN) All PSNs inside above intervals
      shall be retransmitted by sender.

5.  T/C AETH Target Fragment Header Format

   The T/C AETH fragment occupies fixed 30 Bytes inside RDMA ETH header,
   inserted between standard AETH and Atomic AETH subheaders.  All
   fields use big-endian byte order, aligned to 24-bit unless otherwise
   specified.

5.1.  Binary Layout (30 Bytes Total)

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | SegmentNum(24b)| Last Finished PSN (24bit)                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Seg1 Left(24b)| Seg1 Right(24b)                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Seg2 Left(24b)| Seg2 Right(24b)                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Seg3 Left(24b)| Seg3 Right(24b)        Reserved (24 bits)     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   SegmentNum (24 bits): Count of valid receive confirmation segments.
   Valid range: 1 ≤ SegmentNum ≤ 4.  Values exceeding 4 are treated as
   reserved.

   Last Finished PSN (24 bits): Maximum PSN of the first continuous
   receive block (continuous segment without prior packet loss).

   SegX Left (24 bits): Minimum PSN of discontinuous received segment X.

   SegX Right (24 bits): Maximum PSN of discontinuous received segment
   X.

   Reserved (24 bits): Future extension field, set to all zero bits on
   transmission, ignored entirely on receiver processing.

Chen & Yang              Expires 7 January 2027                 [Page 7]
Internet-Draft            Network Working Group                July 2026

5.2.  Segment Mapping Visualization

   T/C AETH fragment encodes receive state in following logical
   sequence: [Continuous Segment: 0 ~ Last Finished PSN] | Lost Gap 1 |
   [Seg1 Left ~ Seg1 Right] | Lost Gap2 | [Seg2 Left ~ Seg2 Right] ...

6.  Retransmission Decision

   Sender strictly retransmits only PSNs within gap ranges calculated
   from T/C AETH segment fields.  No extra packets outside lost
   intervals are retransmitted, eliminating redundant retransmission
   overhead existing in Go-Back-N and oversized sliding window
   implementations.

   For example, sender transmits PSN range 100 ~ 599 at fixed preset
   rate.  Receiver receives continuous block 100~199, loses 200~299,
   receives 300~399, loses 400~499, receives 500~599.

   Corresponding T/C AETH field values:

      SegmentNum = 3

      Seg1 Left=300, Seg1 Right=399

      Seg2 Left=500, Seg2 Right=599

   Sender identifies lost ranges: 200~299, 400~499; retransmits only
   these two PSN intervals.

   To handle potential loss of extended ACK frames on reverse data path:

      Sender maintains per-QP ACK timeout timer configured during QP
      setup.

      If sender transmits PSN range [X, Y] and receives no T/C AETH
      frame after timeout expires: a.  Retransmit the latest transmitted
      sequential PSN block (highest PSNs sent). b.  Restart ACK timeout
      timer immediately after retransmission.

      Fallback timeout mechanism serves only as safety backup; primary
      loss recovery relies on extended T/C AETH SACK reports.

7.  Security Considerations

8.  IANA Considerations

   This document has no requests to IANA.

Chen & Yang              Expires 7 January 2027                 [Page 8]
Internet-Draft            Network Working Group                July 2026

9.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

Authors' Addresses

   Danyang Chen
   China Mobile
   Beijing
   100053
   China
   Email: chendanyang@chinamobile.com

   Hongwei Yang
   China Mobile
   Beijing
   100053
   China
   Email: yanghongwei@chinamobile.com

Chen & Yang              Expires 7 January 2027                 [Page 9]