Network Working Group                                    F. Templin, Ed.
Internet-Draft                              Boeing Research & Technology
Intended status: Informational                          December 7, 2020
Expires: June 10, 2021

                           LTP Fragmentation


   The Licklider Transmission Protocol (LTP) provides a reliable
   datagram convergence layer for the Delay/Disruption Tolerant
   Networking (DTN) Bundle Protocol.  In common practice, LTP is often
   configured over UDP/IP sockets and inherits its maximum segment size
   from the maximum-sized UDP datagram, however when this size exceeds
   the maximum IP packet size for the path a service known as IP
   fragmentation must be employed.  This document discusses LTP
   interactions with IP fragmentation and mitigations for managing the
   amount of IP fragmentation employed.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on June 10, 2021.

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   ( in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect

Templin                   Expires June 10, 2021                 [Page 1]

Internet-Draft              LTP Fragmentation              December 2020

   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   3
   3.  IP Fragmentation Issues . . . . . . . . . . . . . . . . . . .   3
   4.  LTP Fragmentation . . . . . . . . . . . . . . . . . . . . . .   4
   5.  Implementation Status . . . . . . . . . . . . . . . . . . . .   5
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   5
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .   6
   8.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   6
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   6
     9.1.  Normative References  . . . . . . . . . . . . . . . . . .   6
     9.2.  Informative References  . . . . . . . . . . . . . . . . .   7
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .   7

1.  Introduction

   The Licklider Transmission Protocol (LTP) [RFC5326] provides a
   reliable datagram convergence layer for the Delay/Disruption Tolerant
   Networking (DTN) Bundle Protocol (BP) [I-D.ietf-dtn-bpbis].  In
   common practice, LTP is often configured over User Datagram Protocol
   (UDP) / Internet Protocol (IP) [RFC0768][RFC0791] sockets and
   inherits its maximum segment size from the maximum-sized UDP datagram
   (i.e. 2^16 bytes minus header sizes).  However, when the UDP datagram
   size exceeds the maximum IP packet size for the path a service known
   as IP fragmentation must be employed.

   LTP breaks BP bundles into "blocks", then further breaks these blocks
   into "segments".  The segment size is a configurable option and
   represents the largest atomic block of data that LTP will require
   underlying layers to deliver as a single unit.  The segment size is
   therefore also known as the "retransmission unit", since each lost
   segment must be retransmitted in its entirety.

   When LTP presents a segment to the operating system kernel (e.g., via
   a sendmsg() system call), the UDP layer frames the segment in a UDP
   header.  The UDP layer then presents the resulting datagram to the IP
   layer for packet framing and transmission over a networked path.  The
   path is further characterized by the path Maximum Transmission Unit
   (Path-MTU) which is a measure of the smallest link MTU (Link-MTU)
   among all links in the path.

Templin                   Expires June 10, 2021                 [Page 2]

Internet-Draft              LTP Fragmentation              December 2020

   When LTP presents a segment to the kernel that is larger than the
   Path-MTU, the IP layer performs IP fragmentation to break the
   datagram into fragments that are no larger than the Path-MTU.  For
   example, if the LTP segment size is 64000 bytes and the Path-MTU is
   1280 bytes IP fragmentation results in 50+ fragments that are
   transmitted as individual IP packets.  (Note that for IPv4 [RFC0791],
   fragmentation may occur either in the source host or in a router in
   the network path, while for IPv6 [RFC8200] only the source host may
   perform fragmentation.)

   Each IP fragment is subject to the same best-effort delivery service
   offered by the network according to current congestion and/or link
   signal quality conditions; therefore, the IP fragment size becomes
   known as the "loss unit".  Especially when the packet loss rate is
   considerable, however, performance can suffer dramatically when the
   loss unit is significantly smaller than the retransmission unit.  In
   particular, if even a single IP fragment of a fragmented LTP segment
   is lost then the entire LTP segment is deemed lost and must be

   This document discusses LTP interactions with IP fragmentation and
   mitigations for managing the amount of IP fragmentation employed.

2.  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119][RFC8174] when, and only when, they appear in all
   capitals, as shown here.

3.  IP Fragmentation Issues

   IP fragmentation is a fundamental service of the Internet Protocol,
   yet it has long been understood that its use can be problematic in
   some environments.  Beginning as early as 1987, "Fragmentation
   Considered Harmful" [FRAG] outlined multiple issues with the service
   including a performance-crippling condition that can occur at high
   data rates when the loss unit is considerably smaller than the
   retransmission unit during intermittent and/or steady-state loss

   Later investigations also identified the possibility for undetected
   data corruption at high data rates due to a condition known as
   "wraparound" when the 16-bit IP identification field increments to
   overlap with existing fragments still alive in the network and with
   identical ID values [RFC4963][RFC6864].  Although this issue occurs
   only in the IPv4 protocol (and not in IPv6 where the IP ID is 32-bits

Templin                   Expires June 10, 2021                 [Page 3]

Internet-Draft              LTP Fragmentation              December 2020

   in length), the IPv4 concerns along with the fact that IPv6 does not
   permit routers to perform "network fragmentation" have led many to
   discourage its use.

   Even in the modern era, investigators have seen fit to declare "IP
   Fragmentation Considered Fragile" in a publication [RFC8900] that has
   now become an Internet Engineering Task Force (IETF) Best Current
   Practice (BCP) reference.  Indeed, the work cites the Bundle Protocol
   LTP convergence layer as a user of IP fragmentation that depends on
   some of its properties to realize greater performance.  However, the
   BCP summarizes by saying:

      "Rather than deprecating IP fragmentation, this document
      recommends that upper-layer protocols address the problem of
      fragmentation at their layer, reducing their reliance on IP
      fragmentation to the greatest degree possible."

   While the performance implications are considerable and have serious
   implications for real-world applications, our goal in this document
   is neither to condemn nor sanctify IP fragmentation as it pertains to
   the Bundle Protocol LTP convergence layer operating over UDP/IP
   sockets.  Instead, we examine ways in which the benefits of IP
   fragmentation can be realized while avoiding the pitfalls.  We
   therefore next discuss our systematic approach to LTP fragmentation.

4.  LTP Fragmentation

   In common LTP implementations over UDP/IP (e.g., the Interplanetary
   Overlay Network (ION)), performance is greatly dependent on the LTP
   segment size.  This is due to the fact that a larger segment
   presented to UDP/IP as a single unit incurs only a single system call
   and a single data copy from application to kernel space via the
   sendmsg() system call.  Once inside the kernel, the segment incurs
   UDP/IP encapsulation and IP fragmentation which again results in a
   loss unit smaller than the retransmission unit.  However, during
   fragmentation, each fragment is transmitted immediately following the
   previous without delay so that the fragments appear as a "burst" of
   consecutive packets over the network path resulting in high network
   utilization during the burst period.

   In order to avoid retransmission congestion (i.e., especially when
   the loss probability is non-negligible), the natural choice would be
   to set the LTP segment size to a size that is no larger than the
   Path-MTU.  Assuming the minimum IPv4 MTU of 576 bytes, however,
   transmission of 64KB of data using a 576B segment size would require
   over 100 independent sendmsg() system calls and data copies as
   opposed to just one when the largest segment size is used.  This
   greatly reduces the bandwidth advantage offered by IP fragmentation

Templin                   Expires June 10, 2021                 [Page 4]

Internet-Draft              LTP Fragmentation              December 2020

   bursts.  Therefore, a means for providing the best aspects of both
   large segment fragment bursting and small segment retransmission
   efficiency is needed.

   Common operating systems such as linux provide facilities such as the
   sendmmsg() ("send multiple message") system call that allows the LTP
   application to present the kernel with a vector of segments instead
   of just a single segment.  This affords the bursting behavior of IP
   fragmentation coupled with the retransmission efficiency of employing
   small segment sizes.

   This work therefore recommends implementations of LTP to employ a
   large block size, a conservative segment size and a new configuration
   option known as the "Burst-Limit" which determines the number of
   segments that can be presented in a single sendmmsg() system call.
   When the implementation receives an LTP block, it carves Burst-Limit-
   many segments from the block and presents the vector of segments to
   sendmmsg().  The kernel will prepare each segment as an independent
   UDP/IP packet and transmit them into the network as a burst in a
   fashion that parallels IP fragmentation.  The loss unit and
   retransmission unit will be the same, therefore loss of a single
   segment does not result in a retransmission congestion event.

   It should be noted that the Burst-Limit is bounded only by the LTP
   block size and not by the maximum UDP datagram size.  Therefore, each
   burst can in practice convey significantly more data than a single IP
   fragmentation event.  It should also be noted that the segment size
   can still be made larger than the Path-MTU in low-loss environments
   without danger of triggering retransmission storms due to loss of IP
   fragments.  This would result in combined UDP message and IP fragment
   bursting for high network utilization in more robust environments.
   Finally, both the Burst-Limit and UDP message sizes need not be
   static values, and can be tuned to adaptively increase or decrease
   according to time varying network conditions.

5.  Implementation Status

   An implementation is included in the official ION source code
   distribution, beginning with release ion-4.0.1.

6.  IANA Considerations

   This document introduces no IANA considerations.

Templin                   Expires June 10, 2021                 [Page 5]

Internet-Draft              LTP Fragmentation              December 2020

7.  Security Considerations

   Communications networking security is necessary to preserve the
   confidentiality, integrity and availability.

8.  Acknowledgements

   The NASA Space Communications and Networks (SCaN) directorate
   coordinates DTN activities for the International Space Station (ISS)
   and other space exploration initiatives.

   Madhuri Madhava Badgandi, Keith Philpott, Bill Pohlchuck,
   Vijayasarathy Rajagopalan and Eric Yeh are acknowledged for their
   significant contributions.  Tyler Doubrava was the first to mention
   the "sendmmsg()" facility.

9.  References

9.1.  Normative References

   [RFC0768]  Postel, J., "User Datagram Protocol", STD 6, RFC 768,
              DOI 10.17487/RFC0768, August 1980,

   [RFC0791]  Postel, J., "Internet Protocol", STD 5, RFC 791,
              DOI 10.17487/RFC0791, September 1981,

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,

   [RFC5326]  Ramadas, M., Burleigh, S., and S. Farrell, "Licklider
              Transmission Protocol - Specification", RFC 5326,
              DOI 10.17487/RFC5326, September 2008,

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <>.

   [RFC8200]  Deering, S. and R. Hinden, "Internet Protocol, Version 6
              (IPv6) Specification", STD 86, RFC 8200,
              DOI 10.17487/RFC8200, July 2017,

Templin                   Expires June 10, 2021                 [Page 6]

Internet-Draft              LTP Fragmentation              December 2020

9.2.  Informative References

   [FRAG]     Mogul, J. and C. Kent, "Fragmentation Considered Harmful,
              ACM Sigcomm 1987", August 1987.

              Burleigh, S., Fall, K., and E. Birrane, "Bundle Protocol
              Version 7", draft-ietf-dtn-bpbis-29 (work in progress),
              November 2020.

   [RFC4963]  Heffner, J., Mathis, M., and B. Chandler, "IPv4 Reassembly
              Errors at High Data Rates", RFC 4963,
              DOI 10.17487/RFC4963, July 2007,

   [RFC6864]  Touch, J., "Updated Specification of the IPv4 ID Field",
              RFC 6864, DOI 10.17487/RFC6864, February 2013,

   [RFC8900]  Bonica, R., Baker, F., Huston, G., Hinden, R., Troan, O.,
              and F. Gont, "IP Fragmentation Considered Fragile",
              BCP 230, RFC 8900, DOI 10.17487/RFC8900, September 2020,

Author's Address

   Fred L. Templin (editor)
   Boeing Research & Technology
   P.O. Box 3707
   Seattle, WA  98124


Templin                   Expires June 10, 2021                 [Page 7]