IP Parcels
draft-templin-intarea-parcels-32
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft whose latest revision state is "Replaced".
|
|
|---|---|---|---|
| Author | Fred Templin | ||
| Last updated | 2023-01-26 | ||
| Replaced by | draft-templin-intarea-parcels2 | ||
| RFC stream | (None) | ||
| Formats | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-templin-intarea-parcels-32
Network Working Group F. L. Templin, Ed.
Internet-Draft Boeing Research & Technology
Updates: RFC2675 (if approved) 26 January 2023
Intended status: Standards Track
Expires: 30 July 2023
IP Parcels
draft-templin-intarea-parcels-32
Abstract
IP packets (both IPv4 and IPv6) contain a single unit of upper layer
protocol data which becomes the retransmission unit in case of loss.
Upper layer protocols including the Transmission Control Protocol
(TCP) and transports over the User Datagram Protocol (UDP) prepare
data units known as "segments", with each individual IP packet
including only a single segment. This document presents a new
construct known as the "IP Parcel" which permits a single packet to
carry multiple upper layer protocol segments, essentially creating a
"packet-of-packets". IP parcels provide an essential building block
for improved performance, efficiency and integrity while encouraging
larger Maximum Transmission Units (MTUs) in the Internet.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 30 July 2023.
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.
Templin Expires 30 July 2023 [Page 1]
Internet-Draft IP Parcels January 2023
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Background and Motivation . . . . . . . . . . . . . . . . . . 6
4. IP Parcel Formation . . . . . . . . . . . . . . . . . . . . . 7
5. TCP Parcels . . . . . . . . . . . . . . . . . . . . . . . . . 12
6. UDP Parcels . . . . . . . . . . . . . . . . . . . . . . . . . 13
7. Transmission of IP Parcels . . . . . . . . . . . . . . . . . 14
7.1. Individual IP Packets over Non-Parcel Links . . . . . . . 15
7.2. Sub-Parcels over Parcel-capable Links . . . . . . . . . . 17
7.3. Parcels/Sub-Parcels over OMNI Interfaces . . . . . . . . 18
7.4. Final Destination Reassembly . . . . . . . . . . . . . . 20
8. Parcel Path Probing . . . . . . . . . . . . . . . . . . . . . 21
9. Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . 25
10. IP Jumbograms . . . . . . . . . . . . . . . . . . . . . . . . 28
11. Implementation Status . . . . . . . . . . . . . . . . . . . . 29
12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 30
13. Security Considerations . . . . . . . . . . . . . . . . . . . 30
14. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 31
15. References . . . . . . . . . . . . . . . . . . . . . . . . . 31
15.1. Normative References . . . . . . . . . . . . . . . . . . 31
15.2. Informative References . . . . . . . . . . . . . . . . . 32
Appendix A. IP Parcel Futures . . . . . . . . . . . . . . . . . 34
Appendix B. Change Log . . . . . . . . . . . . . . . . . . . . . 35
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 35
1. Introduction
IP packets (both IPv4 [RFC0791] and IPv6 [RFC8200]) contain a single
unit of upper layer protocol data which becomes the retransmission
unit in case of loss. Upper layer protocols such as the Transmission
Control Protocol (TCP) [RFC9293] and transports over the User
Datagram Protocol (UDP) [RFC0768] (including QUIC [RFC9000], LTP
[RFC5326] and others) prepare data units known as "segments", with
each individual IP packet including only a single segment. This
document presents a new construct known as the "IP Parcel" which
permits a single packet to carry multiple upper layer protocol
segments. This essentially creates a "packet-of-packets" with the IP
Templin Expires 30 July 2023 [Page 2]
Internet-Draft IP Parcels January 2023
layer and full {TCP,UDP} headers appearing only once but with
possibly more than one segment included.
Parcels are formed when an upper layer protocol entity identified by
the "5-tuple" (source address, destination address, source port,
destination port, protocol number) prepares a data buffer beginning
with an Integrity Block of up to 256 2-octet Checksums followed by
their corresponding upper layer protocol segments that can be broken
out into smaller sub-parcels and/or individual packets if necessary.
All segments except the final one must be equal in length and no
larger than 65535 octets (minus headers), while the final segment
must not be larger than the others but may be smaller. The upper
layer protocol entity then delivers the buffer, number of segments
and non-final segment size to lower layers which append a {TCP,UDP}
header and an IP header plus extensions that identify this as a
parcel and not an ordinary packet.
Parcels can be forwarded over consecutive parcel-capable links in a
path until they arrive at a router where the next hop is via a link
that does not support parcels, a parcel-capable link with a size
restriction, or an ingress middlebox Overlay Multilink Network (OMNI)
Interface [I-D.templin-intarea-omni] that spans intermediate
Internetworks using adaptation layer encapsulation and fragmentation.
In the first case, the router breaks the parcel into individual IP
packets and forwards them via the next hop link. In the second case,
the router breaks the parcel into smaller sub-parcels and forwards
them via the next hop link. In the final case, the OMNI interface
breaks the parcel into smaller sub-parcels if necessary then applies
adaptation layer encapsulation and fragmentation if necessary.
These OMNI interface sub-parcels may then be recombined into one or
more larger parcels by an egress middlebox OMNI interface which
either delivers them locally or forwards them over additional parcel-
capable links on the path to the final destination. The final
destination can then further re-combine sub-parcels of the same
original parcel so as to present the largest possible data unit to
upper layers. Reordering and even loss or damage of individual
segments within the network is therefore possible, but what matters
is that the parcels delivered to the final destination should be the
largest practical size for best performance and that loss or receipt
of individual segments (and not parcel size) determines the
retransmission unit.
The following sections discuss rationale for creating and shipping IP
parcels as well as the actual protocol constructs and procedures
involved. IP parcels provide an essential building block for
improved performance, efficiency and integrity while encouraging
larger Maximum Transmission Units (MTUs) in the Internet. It is
Templin Expires 30 July 2023 [Page 3]
Internet-Draft IP Parcels January 2023
further expected that the parcel concept will inspire future
innovation in applications, operating systems, network equipment and
data links.
2. Terminology
The Oxford Languages dictionary defines a "parcel" as "a thing or
collection of things wrapped in paper in order to be carried or sent
by mail". Indeed, there are many examples of parcel delivery
services worldwide that provide an essential transit backbone for
efficient business and consumer transactions.
In this same spirit, an "IP parcel" is simply a collection of up to
256 upper layer protocol segments wrapped in an efficient package for
transmission and delivery (i.e., a "packet-of-packets") while a
"singleton IP parcel" is simply a parcel that contains a single
segment. IP parcels are distinguished from ordinary packets through
the special header constructions discussed in this document.
The IP parcel construct is defined for both IPv4 and IPv6. Where the
document refers to "IPv4 header length", it means the total length of
the base IPv4 header plus all included options, i.e., as determined
by consulting the Internet Header Length (IHL) field. Where the
document refers to "IPv6 header length", however, it means only the
length of the base IPv6 header (i.e., 40 octets), while the length of
any extension headers is referred to separately as the "IPv6
extension header length". Finally, the term "IP header plus
extensions" refers generically to an IPv4 header plus all included
options or an IPv6 header plus all included extension headers.
Where the document refers to "{TCP, UDP} header length", it means the
length of either the TCP header plus options (20 or more octets) or
the UDP header (8 octets). It is important to note that only a
single IP header and a single full upper layer header appears in each
parcel regardless of the number of segments included. This
distinction often provides a significant savings in overhead made
possible only by IP parcels.
Where the document refers to checksum calculations, it means the
standard Internet checksum unless otherwise specified. The same as
for TCP [RFC9293], UDP [RFC0768] and IPv4 [RFC0791], the standard
Internet checksum is defined as (sic) "the 16-bit one's complement of
the one's complement sum of all (pseudo-)headers plus data, padded
with zero octets at the end (if necessary) to make a multiple of two
octets". A notional Internet checksum algorithm can be found in
[RFC1071], while practical implementations require special attention
to byte ordering "endianness" to ensure interoperability between
diverse architectures.
Templin Expires 30 July 2023 [Page 4]
Internet-Draft IP Parcels January 2023
The Automatic Extended Route Optimization (AERO)
[I-D.templin-intarea-aero] and Overlay Multilink Network Interface
(OMNI) [I-D.templin-intarea-omni] technologies provide an ideal
architectural framework for transmission of IP parcels. AERO/OMNI
are expected to provide an operational environment for IP parcels
beginning from the earliest deployment phases and extending to
accommodate continuous growth. As more and more parcel-capable links
are deployed (e.g., in data centers, edge networks, space-domain
links and other high data rate services) AERO/OMNI will provide an
essential transit service for true IP parcel Internetworking.
The term "parcel-capable link" refers to any data link medium
(physical or virtual) capable of transiting a {TCP,UDP}/IP packet
that employs the parcel-specific constructions specified in this
document. The source and each router in the path has a "next hop
link" that forwards parcels toward the final destination, while each
router and the final destination has a "previous hop link" that
receives en route parcels. Each next hop link MUST be capable of
forwarding parcels with segment lengths no larger than the minimum of
the link Maximum Transmission Unit (MTU) and 65535, while first
applying parcel subdivision if necessary (see: Section 7).
Currently, only the OMNI link satisfies these properties, but new and
existing link types are encouraged to incorporate parcel support in
their designs.
The term "Maximum Transmission Unit (MTU)" is widely understood in
Internetworking terminology to mean the largest packet size that can
traverse a single link ("link MTU") or an entire path ("path MTU")
without requiring IP layer fragmentation. If the MTU value returned
during parcel path qualification is larger than 65535, it determines
the maximum parcel size with unrestricted segment size that a router
can forward over the path/link without requiring a router to perform
subdivision; otherwise, it determines both the maximum parcel and
segment sizes (see: Section 8).
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119][RFC8174] when, and only when, they appear in all
capitals, as shown here.
Templin Expires 30 July 2023 [Page 5]
Internet-Draft IP Parcels January 2023
3. Background and Motivation
Studies have shown that applications can improve their performance by
sending and receiving larger packets due to reduced numbers of system
calls and interrupts as well as larger atomic data copies between
kernel and user space. Larger packets also result in reduced numbers
of network device interrupts and better network utilization (e.g.,
due to header overhead reduction) in comparison with smaller packets.
A first study [QUIC] involved performance enhancement of the QUIC
protocol [RFC9000] using the linux Generic Segment/Receive Offload
(GSO/GRO) facility. GSO/GRO provides a robust (but non-standard)
service similar in nature to the IP parcel service described here,
and its application has shown significant performance increases due
to the increased transfer unit size between the operating system
kernel and QUIC applications. Unlike IP parcels, however, GSO/GRO
perform fragmentation and reassembly at the transport layer with the
transport protocol segment size limited by the path MTU (typically
1500 octets or smaller in today's Internet).
A second study [I-D.templin-dtn-ltpfrag] showed that GSO/GRO also
improves performance for the Licklider Transmission Protocol (LTP)
[RFC5326] used for the Delay Tolerant Networking (DTN) Bundle
Protocol [RFC9171] for segments larger than the actual path MTU
through the use of OMNI interface encapsulation and fragmentation.
Historically, the NFS protocol also saw significant performance
increases using larger (single-segment) UDP datagrams even when IP
fragmentation is invoked, and LTP still follows this profile today.
Moreover, LTP shows this (single-segment) performance increase
profile extending to the largest possible segment size which suggests
that additional performance gains are possible using (multi-segment)
IP parcels that approach or even exceed 65535 octets.
TCP also benefits from larger packet sizes and efforts have
investigated TCP performance using jumbograms internally with changes
to the linux GSO/GRO facilities [BIG-TCP]. The idea is to use the
Jumbo Payload option internally and to allow GSO/GRO to use buffer
sizes larger than 65535 octets, but with the understanding that links
that support jumbos natively are not yet widely available. Hence, IP
parcels provide a packaging that can be considered in the near term
under current deployment limitations.
Templin Expires 30 July 2023 [Page 6]
Internet-Draft IP Parcels January 2023
A limiting consideration for sending large packets is that they are
often lost at links with MTU restrictions, and the resulting Packet
Too Big (PTB) message [RFC1191][RFC8201] may be lost somewhere in the
return path to the original source. This "Path MTU black hole"
condition can degrade performance unless robust path probing
techniques are used, however the best case performance always occurs
when loss of packets due to size restrictions is minimized.
These considerations therefore motivate a design where transport
protocols should employ a maximum segment size no larger than 65535
octets (minus headers), while parcels that carry multiple segments
may themselves be significantly larger. Then, even if the network
needs to sub-divide the parcels into smaller sub-parcels for further
forwarding toward the final destination, an important performance
optimization for the original source, final destination and network
path as a whole can be realized. This performance advantage is
accompanied by an overall improvement in integrity and efficiency.
An analogy: when a consumer orders 50 small items from a major online
retailer, the retailer does not ship the order in 50 separate small
boxes. Instead, the retailer packs as many of the small items as
possible into one or a few larger boxes (i.e., parcels) then places
the parcels on a semi-truck or airplane. The parcels may then pass
through one or more regional distribution centers where they may be
repackaged into different parcel configurations and forwarded further
until they are finally delivered to the consumer. But most often,
the consumer will only find one or a few parcels at their doorstep
and not 50 separate small boxes. This flexible parcel delivery
service greatly reduces shipping and handling cost for all including
the retailer, regional distribution centers and finally the consumer.
4. IP Parcel Formation
An upper layer protocol entity (identified by the 5-tuple as above)
forms a parcel body when it prepares a data buffer containing the
concatenation of an Integrity Block of up to 256 2-octet Checksums
followed by their corresponding upper layer protocol segments (with
each TCP non-first segment preceded by a 4-octet Sequence Number).
All non-final segments MUST be equal in length while the final
segment MUST NOT be larger and MAY be smaller. Each non-final
segment MUST NOT be larger than the minimum of 65535 octets and the
path MTU, minus the length of the {TCP,UDP} header, minus the length
of the IP header (plus options/extensions), minus 2 octets for the
per-segment Checksum. (Note that this also satisfies the case of
ingress middlebox OMNI interfaces in the path that would process the
headers as upper layer protocol payload during IPv6 encapsulation/
fragmentation.)
Templin Expires 30 July 2023 [Page 7]
Internet-Draft IP Parcels January 2023
The upper layer protocol entity then presents the buffer and non-
final segment size L to lower layers (noting that the buffer may be
larger than 65535 octets if it includes sufficient segments of a
large enough size to exceed that value). If the next hop link is not
parcel capable, the lower layer prepares each segment from the buffer
as an individual IP packet as will be discussed further below.
Otherwise, if the buffer plus headers would together be no larger
than the next hop link MTU or path MTU, the lower layer then appends
a single full {TCP,UDP} header (plus options) and a single full IP
header (plus options/extensions). If the buffer would cause a single
parcel to exceed the link/path MTU, the lower layer instead breaks
the buffer up into multiple smaller buffers (each with an integral
number of segments) and appends separate {TCP,UDP}/IP headers for
each as sub-parcels of the same original parcel.
Upper layers request lower layers to include a specially-formatted
"Jumbo Payload" option (e.g., by asserting a socket option) as an
extension to the IP header of each parcel prior to transmission over
a network interface. For IPv4, the Jumbo Payload option is included
as an IPv4 header option with format derived from [RFC2675] except
that the IP layer sets option type to '00001011' and option length to
'00010000' (noting that the length also distinguishes this type from
its obsoleted use as the "IPv4 Probe MTU" option [RFC1063]). The
option is formed as shown in Figure 1:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Opt Data Len | Code | Check |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Nsegs | Jumbo Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identification |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|P|S| Reserved | Path MTU (PMTU) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1: IPv4 Jumbo Payload Option Format
Templin Expires 30 July 2023 [Page 8]
Internet-Draft IP Parcels January 2023
Following {TCP,UDP} parcel assembly but prior to transmission, the IP
layer sets Code to 255 and sets Check to the same value that will
appear in the TTL of the outgoing IPv4 header. The IP layer next
sets Nsegs to a value J between 0 and 255 and sets Jumbo Payload
Length to a 3-octet value M that encodes the length of the IPv4
header plus the length of the {TCP,UDP} header plus the combined
length of the Integrity Block plus all concatenated segments. Next,
the IP layer sets Identification as discussed in Section 7, sets the
"(P)robe Path MTU" flag to '1' for probes or '0' for non-probes and
sets the "(S)ub-parcel" flag to '1' for non-final sub-parcels or '0'
for the final (sub-)parcel. The IP layer finally sets the IPv4
header DF bit to 1 and Total Length field to the non-final segment
size L.
For IPv6, the Jumbo Payload option is included as an IPv6 Hop-by-Hop
option formatted the same as for IPv4 above, but with option type set
to '11001110', option length set to '00001100' and with the Code/
Check fields omitted. The option is formed as shown in Figure 2:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Opt Data Len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Nsegs | Jumbo Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identification |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|P|S| Reserved | Path MTU (PMTU) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: IPv6 Jumbo Payload Option Format
Following {TCP,UDP} parcel assembly but prior to transmission, the IP
layer sets Nsegs to a 1-octet value J between 0 and 255 and sets the
Jumbo Payload Length field to a 3-octet value M that encodes the
lengths of all IPv6 extension headers present plus the length of the
{TCP,UDP} header plus the combined length of the Integrity Block plus
all concatenated segments. Next, the IP layer sets Identification as
discussed in Section 7, sets the P flag to '1' for probes or '0' for
non-probes and sets the S flag to '1' for non-final sub-parcels or
'0' for the final (sub-)parcel. The IP layer finally sets the IPv6
header Payload Length field to L.
The lower layers then assemble the {TCP,UDP}/IP parcel according to
the formats shown in Figure 3:
Templin Expires 30 July 2023 [Page 9]
Internet-Draft IP Parcels January 2023
TCP/IP Parcel Structure UDP/IP Parcel Structure
+------------------------------+ +------------------------------+
|IP Hdr plus options/extensions| |IP Hdr plus options/extensions|
~ {Total, Payload} Length = L ~ ~ {Total, Payload} Length = L ~
| Nsegs = J; Jumbo Length = M | | Nsegs = J; Jumbo Length = M |
+------------------------------+ +------------------------------+
| | | |
~ TCP header (plus options) ~ ~ UDP header ~
| (Includes Sequence Number 0) | | |
+------------------------------+ +------------------------------+
| | | |
~ Integrity Block ~ ~ Integrity Block ~
| | | |
+------------------------------+ +------------------------------+
~ ~ ~ ~
~ Segment 0 (L-4 octets) ~ ~ Segment 0 (L octets) ~
+------------------------------+ +------------------------------+
~ Sequence Number 1 followed ~ ~ ~
~ by Segment 1 (L octets) ~ ~ Segment 1 (L octets) ~
+------------------------------+ +------------------------------+
~ Sequence Number 2 followed ~ ~ ~
~ by Segment 2 (L octets) ~ ~ Segment 2 (L octets) ~
+------------------------------+ +------------------------------+
~ ... ~ ~ ... ~
~ ... ~ ~ ... ~
+------------------------------+ +------------------------------+
~ Sequence Number J followed ~ ~ ~
~ by Segment J (K octets) ~ ~ Segment J (K octets) ~
+------------------------------+ +------------------------------+
Figure 3: {TCP,UDP}/IP Parcel Structure
where the total number of segments is (J + 1), L is the length of
each non-final segment which MUST NOT be larger than 65535 octets
(minus headers) and K is the length of the final segment which MUST
NOT be larger than L. (Note that when J is 0, K and L are one and
the same value.)
The {TCP,UDP} header is then immediately followed by an Integrity
Block containing (J + 1) 2-octet Checksums concatenated in numerical
order as shown in Figure 4:
Templin Expires 30 July 2023 [Page 10]
Internet-Draft IP Parcels January 2023
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum (0) | Checksum (1) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum (2) | ... ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ... ~
~ ... ... ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum (J-1) | Checksum (J) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4: Integrity Block Format
The Integrity Block is then followed by (J + 1) upper layer protocol
segments. For TCP, the TCP header Sequence Number field encodes a
4-octet starting sequence number for the first segment only, while
each additional segment is preceded by its own 4-octet Sequence
Number field. For this reason, the length of the first segment is
only (L-4) octets since the 4-octet TCP header Sequence Number field
applies to that segment. (All non-first TCP segments instead begin
with their own Sequence Numbers, with the 4-octet length included in
L and K.)
Following parcel construction, the Nsegs value unambiguously
determines the number of 2-octet Checksums present in the Integrity
Block and (together with the IP {Total, Payload} length and Jumbo
Payload Length) also determines the number of parcel data segments
present. Receiving nodes that process IP parcels therefore observe
the following requirements:
* if the Jumbo Payload Length indicates insufficient space for the
full Integrity Block plus at least one data segment of length K,
the receiver discards the parcel.
* if the length of the payload following the Integrity Block is (J *
L) or less, the receiver processes all initial Checksums along
with their corresponding segments up to the end of the payload and
ignores any remaining Checksums.
* if the length of the payload following the Integrity Block is
greater than ((J + 1) * L) the receiver processes all Checksums
with their corresponding segments and ignores any remaining
payload beyond the end of the final segment.
Note: per-segment Checksums appear in a contiguous Integrity Block
immediately following the {TCP,UDP}/IP headers instead of inline with
the parcel segments to greatly increase the probability that they
will appear in the contiguous head of a kernel receive buffer even if
the parcel was subject to OMNI interface IPv6 fragmentation. This
Templin Expires 30 July 2023 [Page 11]
Internet-Draft IP Parcels January 2023
condition may not always hold if the IPv6 fragments also incur IPv4
encapsulation and fragmentation over paths that traverse slow IPv4
links with small MTUs. In that case, performance is bounded by the
unavoidable slow link traversal and not the overhead for pulling a
fragmented Integrity Block into the contiguous head of a kernel
receive buffer.
Note: For IPv4 parcels, the first 2 octets of the Jumbo Payload
option include Code and Check fields in case a router on the path
overwrites the values in a wayward attempt to implement [RFC1063].
IPv4 parcel recipients should therefore regard an incorrect Code or
Check value as evidence that the field was either intentionally or
accidentally altered by a previous hop node, and mark the previous
hop link as non-parcel-capable.
5. TCP Parcels
A TCP Parcel is an IP Parcel that includes an IP header plus
extensions with a Jumbo Payload option formed as shown in Section 4
with Nsegs/J encoding one less than the number of segments and Jumbo
Payload length encoding a value up to 16,777,215 (2**24 - 1). The IP
header plus extensions is then followed by a TCP header plus options
(20 or more octets), which is then followed by an Integrity Block
with (J + 1) consecutive 2-octet Checksums. The Integrity Block is
then followed by (J + 1) consecutive segments, where the first
segment is (L-4) octets in length and uses the 4-octet sequence
number found in the TCP header, each intermediate segment is L octets
in length (including its own 4-octet Sequence Number segment header)
and the final segment is K octets in length (including its own
4-octet Sequence Number segment header). The value L is encoded in
the IP header {Total, Payload} Length field while J is encoded in the
Nsegs octet. The overall length of the parcel as well as final
segment length K are determined by the Jumbo Payload length M as
discussed above.
The source prepares TCP Parcels in a similar fashion as for simple
TCP jumbograms [RFC2675]. The source calculates a checksum of the
TCP header plus IP pseudo-header only (see: Section 9), but with the
TCP header Sequence Number field temporarily set to 0 during the
calculation since the true sequence number will be included as a
pseudo header for the first segment. The source then writes the
calculated value in the TCP header Checksum field as-is (i.e.,
without converting calculated '0' values to 'ffff') and finally re-
writes the actual sequence number back into the Sequence Number
field. (Nodes that verify the header checksum first perform the same
operation of temporarily setting the Sequence Number field to 0 and
then resetting to the actual value following checksum verification.)
Templin Expires 30 July 2023 [Page 12]
Internet-Draft IP Parcels January 2023
The source then calculates the checksum of the first segment
beginning with the sequence number found in the full TCP header as a
4-octet pseudo-header then extending over the remaining (L-4) octet
length of the segment. The source next calculates the checksum for
each L octet intermediate segment independently over the length of
the segment (beginning with its sequence number), then finally
calculates the checksum of the K octet final segment (beginning with
its sequence number). As the source calculates each segment(i)
checksum (for i = 0 thru J), it writes the value into the
corresponding Integrity Block Checksum(i) field as-is.
See: Section 9 for additional integrity considerations.
6. UDP Parcels
A UDP Parcel is an IP Parcel that includes an IP header plus
extensions with a Jumbo Payload option formed as shown in Section 4
with Nsegs/J encoding one less than the number of segments and Jumbo
Payload length encoding a value up to 16,777,215 (2**24 - 1). The IP
header plus extensions is then followed by an 8-octet UDP header
followed by an Integrity Block with (J + 1) consecutive 2-octet
Checksums followed by (J + 1) upper layer protocol segments. Each
segment must begin with a transport-specific start delimiter (e.g., a
segment identifier) included by the transport layer user of UDP. The
length of the first segment L is encoded in the IP {Total, Payload}
Length field while J is encoded in the Nsegs octet. The overall
length of the parcel as well as the final segment length are
determined by the Jumbo Payload length M as discussed above.
The source prepares UDP Parcels in a similar fashion as for simple
UDP jumbograms [RFC2675] and therefore MUST set the UDP header length
field to 0. The source then calculates the checksum of the UDP
header plus IP pseudo-header (see: Section 9) and writes the
calculated value in the UDP header Checksum field as-is (i.e.,
without converting calculated '0' values to 'ffff').
The source then calculates a separate checksum for each segment for
which checksums are enabled independently over the length of the
segment. As the source calculates each segment(i) checksum (for i =
0 thru J), it writes the value into the corresponding Integrity Block
Checksum(i) field with calculated '0' values converted to 'ffff'; for
segments with checksums disabled, the source instead writes the value
'0'.
See: Section 9 for additional integrity considerations.
Templin Expires 30 July 2023 [Page 13]
Internet-Draft IP Parcels January 2023
7. Transmission of IP Parcels
Following {TCP,UDP} parcel assembly, the IP layer of the source fully
populates all IP header fields including source address, destination
address and the Jumbo Payload option as discussed above. The source
also maintains a randomly-initialized 32-bit cached Identification
value for each destination. For each parcel transmission, the IP
layer sets the Jumbo Payload Identification field to the current
cached value for this destination then increments the cached value by
1 (modulo 2**32). The IP layer can subsequently reset each cached
value to a new random value at any time (e.g., to maintain an
unpredictable profile) noting that resetting too frequently may
interfere with opportunistic reassembly.
The IP layer of the source next presents each parcel to a network
interface for transmission to the next hop. For ordinary IP
interface attachments to parcel-capable links, the interface simply
admits each parcel into the link the same as for any IP packet after
which it may then be forwarded by one or more routers over additional
consecutive parcel-capable links possibly even traversing the entire
forward path to the final destination. If any node in the path does
not recognize the parcel construct, it may drop the parcel and return
an ICMP "Parameter Problem" message.
When the next hop link is parcel-capable but configures an MTU that
is too small to pass the entire parcel, or when the next hop link
does not support parcels at all, the source breaks the parcel up into
smaller sub-parcels (in the first case) or into individual IP packets
(in the second case). In the first case, each sub-parcel will
contain the same Identification value and with the S flag set
appropriately. The final destination can then reassemble to deliver
the largest possible parcel buffers to its upper layer protocols. In
the second case, the parcel is replaced by individual IP packets, but
the process engages Generic Segment Offload (GSO) and the final
destination can apply Generic Receive Offload (GRO) to recombine the
packets into a larger buffer for delivery to upper layers. In all
other ways, the source processes of breaking a larger parcel up into
smaller sub-parcels or individual IP packets entails the same
considerations as for a router on the path that invokes these
processes as discussed in the following subsections.
Each parcel serves as an implicit probe that tests the forward path's
ability to pass parcels. Each parcel also includes a trailing 24-bit
"Path MTU (PMTU)" field into which the source writes the minimum of
the next hop link MTU and (2**24 -1 ) and each router in the path
writes the minimum MTU in a similar fashion as for [RFC1063] and
[I-D.ietf-6man-mtu-option]. In particular, each router compares the
parcel PMTU value with the next hop link MTU for the parcel and MUST
Templin Expires 30 July 2023 [Page 14]
Internet-Draft IP Parcels January 2023
(re)set PMTU to the minimum value. (Note that the fact that the
parcel traversed a previous hop link should provide acceptable
evidence of forward progress since parcel path MTU determination is
unidirectional in the forward path only. In cases where this may not
be true, nodes can optionally include the previous hop link MTU in
their minimum PMTU calculations.) Each parcel also includes one or
more upper layer protocol segments corresponding to the 5-tuple for
the flow, which may also include {TCP,UDP} segment size probes used
for packetization layer path MTU discovery [RFC4821] [RFC8899]. (See
Section 8 for further details on implicit/explicit path probing.)
When a router receives an IPv4 parcel it first compares Code with 255
and Check with the IPv4 header TTL; if either value differs, the
router marks the previous hop link as non-parcel-capable, drops the
parcel and returns a negative Parcel Reply (see Section 8). For all
IP parcels, the router next compares the value L with the next hop
link MTU. If the next hop link MTU is too small to pass either a
parcel or an individual IP packet with a single segment of length L
the router discards the parcel and returns a positive Parcel Reply
with MTU set to the next hop link MTU. For IPv4 parcels, if the next
hop link is parcel capable the router MUST then reset Check to the
same value that would appear in the TTL of the outgoing IPv4 header
for forwarding the parcel to the next hop.
If the router recognizes parcels but the next hop link in the path
does not, or if the entire parcel would exceed the next hop link MTU,
the router instead opens the parcel. The router then forwards each
enclosed segment in individual IP packets or in a set of smaller sub-
parcels that each contain a subset of the original parcel's segments.
If the next hop link is via an OMNI interface, the router instead
proceeds according to OMNI Adaptation Layer procedures. These
considerations are discussed in detail in the following sections.
7.1. Individual IP Packets over Non-Parcel Links
For transmission of individual IP packets over links that do not
support parcels, the source or router (i.e., the node) engages GSO.
The node first determines whether a singleton parcel with segment of
length L can fit within the next hop link MTU. If not, the node
drops the parcel and returns a positive Parcel Reply message with MTU
set to the next hop link MTU and with the leading portion of the
parcel beginning with the IP header as the "packet in error".
Otherwise, the node removes the Jumbo Payload option, sets aside and
remembers the Integrity Block (and for TCP also truncates the
Sequence Number headers of each non-first segment while remembering
their values) then copies the {TCP,UDP}/IP headers (but with the
Jumbo Payload option removed) followed by segment(i) (for i= 0 thru
J) into 'i' individual IP packets ("packet(i)"). The node then sets
Templin Expires 30 July 2023 [Page 15]
Internet-Draft IP Parcels January 2023
IP {Total, Payload} length for each packet(i) based on the length of
segment(i) according to the IP protocol standards [RFC0791]
[RFC8200].
For each IPv6 packet(i), the node includes an IPv6 Fragment Header
and sets the Identification field to the value found in the parcel
header. For each IPv4 packet(i), the node sets the Identification
field to the least significant 16 bits of the value found in the
parcel header and sets the (D)ont Fragment flag to '1'. For each IP
packet(i), the node then sets both the Fragment Offset field and
(M)ore fragments flag to '0' to produce an unfragmented IP packet.
IPv6 destinations should process these "atomic fragments" as whole
packets instead of admitting them into the reassembly cache (i.e.,
the same as for IPv4).
The node then processes further according to upper layer protocol
conventions. For TCP, the node clears the SYN/ACK flags in all
except packet(0) then calculates the checksum for packet(0)'s TCP/IP
headers only according to [RFC9293] but with the Sequence Number
value saved and the field set to 0. The node then adds Integrity
Block Checksum(0) to the calculated value and writes the sum into
packet(0)'s TCP checksum field. The node then resets the Sequence
Number field to packet(0)'s saved sequence number and forwards
packet(0) to the next hop. The node next calculates the checksum of
packet(1)'s TCP/IP headers with the Sequence Number field set to 0
and saves the calculated value. In each non-first packet(i) (for i =
1 thru J), the node then adds the saved value to Integrity Block
Checksum(i), writes the sum into packet(i)'s TCP checksum field, sets
the TCP Sequence Number field to packet(i)'s sequence number then
forwards packet(i) to the next hop.
For UDP, the node sets the UDP length field according to [RFC0768] in
each packet(i) (for i= 0 thru J). If Integrity Block Checksum(i) is
0, the node then sets the UDP header checksum to 0, forwards
packet(i) to the next hop and continues to the next. The node next
calculates the checksum over packet(i)'s UDP/IP headers only
according to [RFC0768]. If Integrity Block Checksum(i) is not
'ffff', the node then adds the value to the header checksum;
otherwise, the node re-calculates the checksum for segment(i). If
the re-calculated segment(i) checksum value is 'ffff' or '0' the node
adds the value to the header checksum; otherwise, it continues to the
next packet(i). The node finally writes the total checksum value
into the UDP checksum field for packet(i) (or writes 'ffff' if the
total was '0') and forwards packet(i) to the next hop.
Note: for each UDP packet(i), the node must recalculate the segment
checksum if Checksum(i) is 'ffff', since that value is shared by both
'0' and 'ffff' calculated checksums. If recalculating the checksum
Templin Expires 30 July 2023 [Page 16]
Internet-Draft IP Parcels January 2023
produces an incorrect value, segment(i) is considered errored and the
node can optionally drop or forward (noting that the forwarded packet
would simply be discarded as an error by the final destination).
Note: for each {TCP,UDP} packet(i), the node can optionally re-
calculate and verify the segment checksum unconditionally before
forwarding, but this may introduce undesirable extra delay and
processing overhead.
7.2. Sub-Parcels over Parcel-capable Links
For transmission of smaller sub-parcels over parcel-capable links,
the source or router (i.e., the node) first determines whether a
single segment of length L can fit within the next hop link MTU if
packaged as a (singleton) sub-parcel. If not, the node returns a
positive Parcel Reply message with MTU set to the next hop link MTU
and containing the leading portion of the parcel beginning with the
IP header, then drops the parcel. Otherwise, the node breaks the
original parcel into smaller groups of segments that would fit within
the path MTU by determining the number of segments of length L that
can fit into each sub-parcel under the size constraints. For
example, if the node determines that a sub-parcel can contain 3
segments of length L, it creates sub-parcels with the first
containing Integrity Block Checksums/Segments 0-2, the second
containing Checksums/Segments 3-5, etc., and with the final
containing any remaining Checksums/Segments.
When the node breaks an original parcel into sub-parcels, it first
checks the "(S)ub-parcel" flag in the Jumbo Header. If the S flag is
'0', the node sets S to '1' in all resulting sub-parcels except the
last (i.e., the one containing the final segment of length K, which
may be shorter than L) for which it sets S to '0'. If the S flag is
'1', the node instead sets S to '1' in all resulting sub-parcels
including the last. The node finally sets PMTU to the next hop link
MTU.
Templin Expires 30 July 2023 [Page 17]
Internet-Draft IP Parcels January 2023
The node then appends identical {TCP,UDP}/IP headers (including the
Jumbo Payload option and any other extensions) to each sub-parcel
while resetting L and M in each according to the above equations with
Nsegs/J set to 2 for each intermediate sub-parcel and with Nsegs/J
set to one less than the remaining number of segments for the final
sub-parcel. For TCP, the node then sets the TCP Sequence Number
field to the value that appears in the first sub-parcel segment while
removing the first segment Sequence Number field (if present) and
also clears the SYN/ACK flags in all sub-parcels except the first.
For both TCP and UDP, the node finally resets the {TCP,UDP} header
checksum according to ordinary parcel formation procedures (see
above) then forwards each (sub-)parcel over the parcel-capable next
hop link.
Note: sub-dividing a larger parcel into two or more sub-parcels
entails replication of the {TCP,UDP}/IP headers (including the Jumbo
Payload option and any other extensions). For TCP, the process
entails copying the full TCP/IP header from the original parcel while
writing the sequence number of the first sub-parcel segment into the
TCP Sequence Number field, clearing the SYN/ACK flags if necessary as
discussed above and truncating the (new) first segment Sequence
Number field. For UDP, the process entails copying the full UDP/IP
header from the original parcel into each sub-parcel. For both TCP
and UDP, the process finally includes recalculating and resetting
Nsegs and Jumbo Payload Length then recalculating the {TCP,UDP}
header checksum. Note that the per-segment Integrity Block Checksum
values in the sub-parcel segments themselves are still valid and need
not be recalculated.
7.3. Parcels/Sub-Parcels over OMNI Interfaces
For transmission of original parcels or sub-parcels over OMNI
interfaces, the node admits all parcels into the interface
unconditionally since the OMNI interface MTU is unrestricted. The
OMNI Adaptation Layer (OAL) of this First Hop Segment (FHS) OAL
source node then forwards the parcel to the next OAL hop which may be
either an OAL intermediate node or a Last Hop Segment (LHS) OAL
destination. OMNI interface upper layer protocol processing
procedures are specified in detail in the remainder of this section,
while lower layer encapsulation and fragmentation procedures are
specified in [I-D.templin-intarea-omni].
When the OAL source forwards a parcel or sub-parcel (whether
generated by a local application or forwarded by other nodes over one
or more parcel-capable links), it first assigns a monotonically-
incrementing (modulo 255) "Parcel ID" for adaptation layer
processing. If the parcel is larger than the OAL maximum segment
size of 65535 octets, the OAL source then subdivides the parcel into
Templin Expires 30 July 2023 [Page 18]
Internet-Draft IP Parcels January 2023
sub-parcels the same as for the IP layer procedures discussed above.
The OAL source next assigns a different monotonically-incrementing
adaptation layer Identification value for each sub-parcel of the same
"Parcel ID" then performs adaptation layer encapsulation and
fragmentation and finally forwards each fragment to the next OAL hop
which forwards them further toward the OAL destination as necessary.
(During encapsulation, the OAL source examines the Jumbo Payload
option S flag to determine the setting for the adaptation layer
fragment header S flag according to the same rules specified in
Section 7.2.)
When the sub-parcels arrive at the OAL destination, the node can
optionally retain them along with their Parcel ID and Identifications
for a brief time to support re-combining with peer sub-parcels of the
same original parcel identified by the adaptation layer 4-tuple
(source, destination, Identification, Parcel ID). This re-combining
entails the concatenation of Checksums/Segments included in sub-
parcels with the same Parcel ID and with Identification values within
255 of one another to create a larger sub-parcel possibly even as
large as the entire original parcel. Order of concatenation need not
be strictly enforced, with the exception that the sub-parcel with S
flag set to '0' must occur as a final concatenation and not as an
intermediate. The recombined (sub)parcel then sets the S flag to '0'
if and only if one of its recombined elements also had the S flag set
to '0'; otherwise, it sets the S flag to '1'.
The OAL destination then appends a common {TCP,UDP}/IP header plus
extensions to each re-combined sub-parcel while resetting J, K, L and
M in each. For TCP, if any sub-parcels have the SYN/ACK flags set
the OAL destination also sets the SYN/ACK flags in the re-combined
sub-parcel TCP header. The OAL destination then resets the
{TCP,UDP}/IP header checksum for each re-combined sub-parcel. If the
OAL destination is also the final destination, it then delivers the
sub-parcels to the IP layer which processes them according to the
5-tuple information supplied by the original source. Otherwise, the
OAL destination forwards each sub-parcel toward the final destination
the same as for an ordinary IP packet as discussed above.
Note: sub-dividing of IP parcels over OMNI links occurs only at an
OAL ingress node while re-combining of IP parcels occurs only at an
OAL egress node. Therefore, intermediate OAL nodes do not
participate in the sub-dividing or recombining processes. For TCP,
the SYN/ACK flags must be managed as specified above to avoid
confusing receivers with gratuitous duplicate ACKs.
Note: re-combining two or more sub-parcels into a larger parcel
entails a process in which the {TCP,UDP}/IP headers of non-first sub-
parcels are discarded and their included segments concatenated
Templin Expires 30 July 2023 [Page 19]
Internet-Draft IP Parcels January 2023
following those of a first sub-parcel. For TCP, the process includes
setting the SYN/ACK flags in the TCP header only if SYN/ACK were set
in any of the original sub-parcels. For both TCP and UDP, the
process finally includes recalculating and resetting Nsegs and Jumbo
Payload Length then recalculating the {TCP,UDP} header checksum as
discussed above (the per-segment Integrity Block Checksums need not
be recalculated). The OAL destination can instead avoid this process
if it would negatively impact performance, noting that forwarding
individual sub-parcels without delay and without re-combining is
always acceptable (but not always optimal).
Note: sub-dividing and re-combining of IP parcels over OMNI links
occurs is based on the adaptation layer 4-tuple and not the network
layer 5-tuple. The adaptation layer must adhere to this discipline
even if network layer information is available, since some sub-
parcels of the same original parcel may be forwarded over different
network paths.
7.4. Final Destination Reassembly
When a large parcel transits a path that includes links with
restrictive MTUs, the final destination may receive multiple sub-
parcels having the same 5-tuple and Identification value. The final
destination can hold the sub-parcels in a reassembly buffer for a
short time or until a sub-parcel with the S flag set to '0' arrives.
The final destination then concatenates the segments of all non-final
sub-parcels and finally concatenates the segments of the final sub-
parcel then passes the reassembled parcel to upper layers.
Due to the possibility of network loss and/or reordering, the final
destination may receive a sub-parcel with S set to '0' before all
other sub-parcels of the same original parcel have arrived. This
condition does not constitute an error, but in some cases may cause
the IP layer to deliver partial reassemblies to upper layers that are
smaller than the original parcel. Upper Layers simply process any
segments received from all such lower layer deliveries and will
request retransmission of any segments that were lost and/or damaged.
If the original source or a router on the path opens a parcel and
forwards its contents as individual IP packets, these packets will
arrive at the final destination which may collectively reassemble
them using GRO. The 5-tuple information plus the Identification
value provides sufficient context for GRO reassembly, which practical
implementations have proven can provide a robust service at high data
rates even for IPv4 with its 16-bit Identification limitation.
Templin Expires 30 July 2023 [Page 20]
Internet-Draft IP Parcels January 2023
Note: in both the sub-parcel and GRO reassembly cases, reassembly
entails concatenation of the segments in the order they were received
even though some small degree of reordering and/or loss may have
occurred in the networked path. This eliminates the need for a
reassembly offset value, since each sub-parcel or individual IP
packet contains an integral number of whole upper layer protocol
segments which are not themselves fragmented. The IP layer can then
present the reassembled parcel contents to upper layers with segments
arranged in roughly the same order in which they were originally
transmitted, but strict ordering is not required since each segment
will include an upper layer protocol-specific start delimiter with
positional coordinates.
Note: as an implementation option, if the final destination's
reassembly buffer holds sub-parcels of "adjacent" parcels (i.e.,
those with identical 5-tuples, L values, and with Identification
values in close proximity) the destination can optionally recombine
sub-parcels of adjacent parcels for delivery to upper layers. If so,
however, the destination must avoid recombining sub-parcels
containing final segments of multiple original parcels.
8. Parcel Path Probing
All parcels serve as implicit probes and may cause either a router in
the path or the final destination to return an ordinary ICMP error
[RFC0792][RFC4443] and/or Packet Too Big (PTB) message
[RFC1191][RFC8201] concerning the parcel. A router in the path or
the final destination may also return an unsolicited negative "Parcel
Reply" if the parcel cannot make further forward progress.
To unambiguously determine whether parcels can transit at least an
initial portion of the forward path toward the final destination, the
original source can also send IP parcels with the Jumbo Payload
option P flag set to '1' as an explicit "Parcel Probe". The probe
will elicit a Parcel Reply from a router or the final destination
(and possibly also one or more upper layer protocol-specific probe
replys from the final destination) while the parcel itself may
continue to make forward progress.
If the original source receives a positive Parcel Reply, it marks the
path as "parcels supported" and ignores any ordinary ICMP and/or PTB
messages concerning the probe. If the original source instead
receives a negative Parcel Reply or no reply, it marks the path as
"parcels not supported" and may regard any ordinary ICMP and/or PTB
messages concerning the probe (or its contents) as indications of a
possible path limitation.
Templin Expires 30 July 2023 [Page 21]
Internet-Draft IP Parcels January 2023
The original source can therefore send Parcel Probes in the same IP
parcels used to carry real data. The probes will traverse parcel-
capable links joined by routers on the forward path possibly
extending all the way to the destination. If the original source
receives a positive Parcel Reply, it can continue using IP parcels
(while also adjusting its current segment size if necessary).
The original source sends Parcel Probes unidirectionally in the
forward path toward the final destination to elicit a Parcel Reply,
since it will often be the case that IP parcels are supported only in
the forward path and not in the return path. Parcel Probes may be
dropped in the forward path by any node that does not recognize IP
parcels, but Parcel Replys must be packaged to avoid filtering since
parcels may not be recognized along portions of the return path. For
this reason, the Jumbo Payload options included in Parcel Probes are
always packaged as IPv4 header options or IPv6 Hop-by-Hop options
while Parcel Replys are returned as UDP/IP encapsulated ICMPv6 PTB
messages with a "Parcel Reply" Code value (see:
[I-D.templin-intarea-omni]).
Original sources send ordinary parcels as explicit Parcel Probes by
setting the Jumbo Payload P flag to '1' and PMTU to the minimum of
the next hop link MTU and (2**24 -1 ). The source can also form a
NULL probe/parcel by setting Protocol to "No Next Header (59)" and
including an Integrity Block with one or more Checksum fields set to
'0' followed by a corresponding number of NULL segments with zero,
random and/or other disposable payloads. The source then sets
{Nsegs, Jumbo Payload Length, IPv4 Total Length} and calculates the
header and per-segment checksums the same as for an ordinary parcel.
The source finally sends the Parcel Probe via the outbound IP
interface.
According to [RFC7126], IPv4 middleboxes (i.e., routers, security
gateways, firewalls, etc.) that do not observe this specification
SHOULD drop IPv4 packets that contain option type '00001011' ("IPv4
Probe MTU") but some might instead either attempt to implement
[RFC1063] or ignore the option altogether. IPv4 middleboxes that
observe this specification instead MUST process the option as an
implicit or explicit Parcel Probe as specified below.
According to [RFC2675], IPv6 middleboxes (i.e., routers, security
gateways, firewalls, etc.) that recognize the IPv6 Jumbo Payload
option but do not observe this specification SHOULD return an ICMPv6
Parameter Problem message (and presumably also drop the packet) due
to the different option length. IPv6 middleboxes that observe this
specification instead MUST process the option as an implicit or
explicit Parcel Probe as specified below.
Templin Expires 30 July 2023 [Page 22]
Internet-Draft IP Parcels January 2023
When a router that observes this specification receives an IPv4
Parcel Probe it first compares Code with 255 and Check with the IP
header TTL; if either value differs, the router marks the previous
hop link as non-parcel-capable then MUST drop the probe and return a
negative Parcel Reply (see below). For all other IP Parcel Probes,
if the next hop link is non-parcel-capable the router compares the
PMTU value with the next hop link MTU for the probe and MUST (re)set
PMTU to the minimum value. The router then MUST return a positive
Parcel Reply (see below) and convert the probe into individual IP
packet(s) the same as specified in Section 7. If the next hop link
configures a sufficiently large MTU to pass the packet(s), the router
converts the probe and MUST forward each packet to the next hop;
otherwise, it drops the probe. If the next hop link both supports
parcels and configures an MTU that is large enough to pass the
parcel, the router instead compares the probe PMTU value with the
next hop link MTU for the probe and MUST (re)set PMTU to the minimum
value. The router then MUST forward the Parcel Probe to the next hop
(and for IPv4 while resetting Check to the same value that will
appear in the outgoing IPv4 TTL). If the next hop link supports
parcels but configures an MTU that is too small to pass the probe, it
resets PMTU (and Check if necessary) the same as above then
subdivides the probe into multiple smaller sub-parcels that can
traverse the link while setting the P flag to '1' only for the first
sub-parcel.
The final destination may therefore receive one or more individual IP
packets or intact Parcel Probes. If the final destination receives
individual IP packets, it performs any necessary integrity checks,
applies GRO if possible then delivers the (reassembled) buffer
contents to upper layers which will return one or more upper layer
probe response(s) if necessary. If the final destination receives an
IPv4 Parcel Probe, it first compares Code with 255 and Check with the
IPv4 header TTL; if either value differs, the final destination marks
the previous hop link as non-parcel-capable then MUST drop the probe
and return a negative Parcel Reply. Otherwise, the final destination
then MUST return a positive Parcel Reply and deliver the
(reassembled) buffer contents to upper layers the same as for an
ordinary IP parcel.
When a router or final destination returns a Parcel Reply, it
prepares an ICMPv6 PTB message [RFC4443] with Code set to "Parcel
Reply" (see: [I-D.templin-intarea-omni]) and with MTU set to either
the PMTU value reported in the probe/parcel for a positive reply or
to the value '0' for a negative reply. The node then writes its own
IP address as the Parcel Reply source and writes the source of the
Parcel Probe as the Parcel Reply destination (for IPv4 Parcel Probes,
the node writes the Parcel Reply address as an IPv4-Compatible IPv6
address [RFC4291]). The node next copies as much of the leading
Templin Expires 30 July 2023 [Page 23]
Internet-Draft IP Parcels January 2023
portion of the probe/parcel (beginning with the IP header) as
possible into the "packet in error" field without causing the Parcel
Reply to exceed 512 octets in length, then calculates the ICMPv6
header checksum. Since IPv6 packets cannot traverse IPv4 paths, and
since middleboxes often filter ICMPv6 messages as they traverse IPv6
paths, the node next wraps the Parcel Reply in UDP/IP headers of the
correct IP version with the IP source and destination addresses
copied from the Parcel Reply and with UDP port numbers set to the UDP
port number for OMNI [I-D.templin-intarea-omni]. In the process, the
node either calculates or omits the UDP checksum as appropriate and
(for IPv4) clears the DF bit. The node finally sends the prepared
Parcel Reply to the original source of the probe.
After sending a Parcel Probe (or an ordinary parcel) the original
source may therefore receive a UDP/IP encapsulated Parcel Reply (see
above) and/or one or more upper layer protocol probe replies. If the
source receives a Parcel Reply, it verifies the checksum and matches
the enclosed PTB message with an original probe/parcel by examining
the Identification echoed in the ICMPv6 "packet in error" containing
the leading portion of the probe. If the Identification does not
match, the source discards the Parcel Reply; otherwise, it continues
to process. If the Parcel Reply MTU is '0', the source marks the
path as "parcels not supported"; otherwise, it marks the path as
"parcels supported" and also records the MTU value as the parcel path
MTU (i.e., the portion of the path up to and including the node that
returned the Parcel Reply). If the MTU value is 65535 or larger, the
MTU determines the largest whole parcel size that can traverse the
parcel path without subdivision while using any segment size up to
and including the maximum. If the MTU value is smaller than 65535,
the MTU represents both the largest whole parcel size and a maximum
segment size limitation. In both cases, the maximum parcel size that
can traverse the initial portion of the path may be larger than the
maximum segment size that can continue to traverse the remaining path
to the final destination, which can only be determined through upper
layer protocol probes (i.e., either as individual probe packets or as
payloads of the Parcel Probes).
Note: If a router or final destination receives a Parcel Probe but
does not recognize the parcel construct, it drops the probe without
further processing (and may return an ICMP error). The original
source will then consider the probe as lost, but may attempt to probe
again later, e.g., in case the path may have changed.
Note: On links that include a forward error correction capability,
in-transit damage to the Parcel Probe headers may be corrected as a
lower-layer function of the receiver before the headers are examined
by the network layer.
Templin Expires 30 July 2023 [Page 24]
Internet-Draft IP Parcels January 2023
9. Integrity
The {TCP,UDP}/IP header plus each segment of a (multi-segment) IP
parcel includes its own integrity check. This means that IP parcels
can support stronger and more discrete integrity checks for the same
amount of upper layer protocol data compared to an individual IP
packet or jumbogram. The {TCP/UDP} header integrity checks can be
verified at each hop to ensure that parcels with errored headers are
detected. The per-segment Integrity Block Checksums are set by the
source and verified by the final destination, noting that TCP parcels
must honor the sequence number discipline discussed in Section 5.
IP parcels can range in length from as small as only the {TCP,UDP}/IP
headers plus a single Integrity Block Checksum with a non-zero length
segment to as large as the headers plus (256 * (65535 minus headers))
octets. Although 32-bit link layer integrity checks provide
sufficient protection for contiguous data blocks up to approximately
9KB, reliance on link-layer integrity checks may be inadvisable for
links with significantly larger MTUs and may not be possible at all
for links such as tunnels over IPv4 that invoke fragmentation.
Moreover, the segment contents of a received parcel may arrive in an
incomplete and/or rearranged order with respect to their original
packaging.
Lower layer protocol entities calculate and verify {TCP,UDP}/IP
parcel header Checksums at their layer, since an errored header could
result in mis-delivery to the wrong upper layer protocol entity. If
a lower layer protocol entity on the path detects an incorrect
{TCP,UDP}/IP Checksum it discards the entire IP parcel unless the
header(s) can somehow be repaired.
To support the parcel header checksum calculation, lower layer
protocol entities use modified versions of the {TCP,UDP}/IPv4
"pseudo-header" found in [RFC0768][RFC9293], or the {TCP,UDP}/IPv6
"pseudo-header" found in Section 8.1 of [RFC8200]. Note that while
the contents of the two IP protocol version-specific pseudo-headers
beyond the address fields are the same, the order in which the
contents are arranged differs and must be honored according to the
specific IP protocol version as shown in Figure 5. This allows for
maximum reuse of widely deployed code while ensuring
interoperability.
Templin Expires 30 July 2023 [Page 25]
Internet-Draft IP Parcels January 2023
IPv4 Parcel Pseudo-Header
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPv4 Source Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPv4 Destination Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| zero | Next Header | Segment Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Nsegs | Upper-Layer Packet Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
IPv6 Parcel Pseudo-Header
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
~ IPv6 Source Address ~
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
~ IPv6 Destination Address ~
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Nsegs | Upper-Layer Packet Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Segment Length | zero | Next Header |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: {TCP,UDP}/IP Parcel Pseudo-Header Formats
where the following fields appear in both pseudo-headers (but
arranged in the order specified in Figure 5):
* Source Address is the 4-octet IPv4 or 16-octet IPv6 source address
of the prepared parcel.
* Destination Address is the 4-octet IPv4 or 16-octet IPv6
destination address of the prepared parcel.
* zero encodes the constant value '0'.
* Next Header is the IP protocol number corresponding to the upper
layer protocol, i.e., TCP or UDP.
* Segment Length is the value that appears in the IPv4 Total Length
or IPv6 Payload Length field of the prepared parcel.
Templin Expires 30 July 2023 [Page 26]
Internet-Draft IP Parcels January 2023
* Nsegs is a 1-octet value one less than the number of segments
included, and must contain a number between 0 and 255 (this is the
same value that appears in the Jumbo Payload Option Nsegs field).
* Upper-Layer Packet Length is the 3-octet length of the {TCP,UDP}
header plus data (this value can be derived from the Jumbo Payload
Length by subtracting the IPv4 header length for IPv4 or IPv6
extension header length for IPv6).
Upper layer protocol entities use socket options to coordinate per-
segment checksum processing with lower layers. If the upper layer
sets a SO_NO_CHECK(TX) socket option, the upper layer is responsible
for supplying per-segment checksums on transmission and the lower
layer forwards the IP parcel to the next hop without further
processing; otherwise, the lower layer supplies the per-segment
checksums before forwarding. If the upper layer sets a
SO_NO_CHECK(RX) socket option, the upper layer is responsible for
verifying per-segment checksums on reception and the lower layer
delivers each received parcel body to the upper layer without further
processing; otherwise, the lower layer verifies the per-segment
parcel checksums before delivering.
When the upper layer protocol entity of the source sends a parcel
body to lower layers, it prepends an Integrity Block of (J + 1)
2-octet Checksum fields and includes a 4-octet Sequence Number field
with each TCP non-first segment. If the SO_NO_CHECK(TX) socket
option is set, the upper layer protocol either calculates each
segment checksum and writes the value into the corresponding Checksum
field (and for UDP with '0' values written as 'ffff') or writes the
value '0' to disable checksums for specific UDP segments. If the
SO_NO_CHECK(TX) socket options is clear, for UDP the upper layer
instead writes the value '0' to disable or any non-zero value to
enable checksums for specific segments (for TCP, the upper layer
instead writes any zero or non-zero value).
When the lower layer protocol entity of the source receives the
parcel body from upper layers, if the SO_NO_CHECK(TX) socket option
is set the lower layer appends the {TCP,UDP}/IP headers and forwards
the parcel to the next hop without further processing. If the
SO_NO_CHECK(TX) socket option is clear, the lower layer instead
calculates the checksum for each TCP segment (or each UDP segment
with a non-zero value in the corresponding Integrity Block Checksum
field) and overwrites the calculated value into the Checksum field
(and for UDP with '0' values written as 'ffff').
When the lower layer protocol entity of the destination receives a
parcel from the source, if the SO_NO_CHECK(RX) socket option is set
the lower layer delivers the parcel body to the upper layer without
Templin Expires 30 July 2023 [Page 27]
Internet-Draft IP Parcels January 2023
further processing, and the upper layer is responsible for per-
segment checksum verification. If the SO_NO_CHECK(RX) socket option
is clear, the lower layer instead verifies the checksum for each TCP
segment (or each UDP segment with a non-zero value in the
corresponding Integrity Block Checksum field) and marks a
corresponding field for the segment in an ancillary data structure as
either "correct" or "incorrect". (For UDP, if the Checksum is '0'
the lower layer protocol unconditionally marks the segment as
"correct".) The lower layer then delivers both the parcel body
(beginning with the Integrity block) and ancillary data to the upper
layer which can then determine which segments have correct/incorrect
checksums.
Note: The Integrity Block itself is intentionally omitted from the IP
Parcel {TCP,UDP} header checksum calculation. This permits
destinations to accept as many intact segments as possible from
received parcels with checksum block bit errors, whereas the entire
parcel would need to be discarded if the header checksum also covered
the Integrity Block.
Note: IP parcels that set {Protocol, Next Header} to "No Next Header
(59)" do not include a {TCP,UDP} Checksum field and therefore do not
include a header checksum. Intermediate nodes simply forward these
NULL parcels without verifying a header checksum, while destination
nodes simply discard them after returning a Parcel Reply, if
necessary.
10. IP Jumbograms
True IPv6 jumbograms are distinguished from IPv6 parcels by including
a zero IPv6 Payload Length and an IPv6 Hop-by-Hop Option with type
'11001110' and length '00000100'. The Jumbo Payload option format
and all aspects of IPv6 jumbogram processing are exactly as specified
in [RFC2675].
True IPv4 jumbograms are distinguished from IPv4 parcels by including
a zero IPv4 Total Length and an IPv4 option with type '00001011' and
length '00000110'. The Jumbo Payload option format and all aspects
of IPv4 jumbogram processing are exactly the same as for IPv6
jumbograms.
This specification augments IP jumbograms by also providing a Jumbo
Path Qualification function using the mechanisms specified in
Section 8. The function employs a "Jumbo Probe" formed exactly the
same as for Parcel Probes, but with Nsegs/Jumbo Payload Length set to
'0' and with the final 4 octets converted to a single 32-bit PMTU
field. The Jumbo Probe also sets the IP {Total, Payload} length
fields to '0', sets {Protocol, Next Header} to "No Next Header (59)"
Templin Expires 30 July 2023 [Page 28]
Internet-Draft IP Parcels January 2023
and includes no octets beyond the IP header. The purpose of the
Jumbo Probe is to determine whether the entire path from the source
to the destination is jumbo-capable (i.e., one in which all links
recognize jumbograms and configure an MTU larger than 65535 octets)
as well as to determine the jumbo path MTU.
The source sets the Jumbo Probe PMTU to the full 32-bit MTU of the
(jumbo-capable) next hop link, (and for IPv4 sets Code to 255 and
Check to the next hop TTL) then sends the probe via the link toward
the final destination. At each IPv4 forwarding hop, the router
examines Code and Check and returns a negative "Jumbo Reply" (i.e.,
prepared the same as a Parcel Reply) if either value is incorrect.
Otherwise, if the next hop link is jumbo-capable the router compares
PMTU to the next hop link MTU, resets PMTU to the minimum value (and
for IPv4 sets Check to the next hop TTL) then silently forwards the
probe to the next hop. If the next hop link is not jumbo-capable,
the router instead drops the probe and returns a negative Jumbo
Reply.
If the Jumbo Probe encounters an OMNI link, the OAL source can either
drop the probe and return a negative Jumbo Reply or forward the probe
further toward the OAL destination using adaptation layer
encapsulation. In a first option, if the OAL source has a table of
known PMTUs for selected OAL destinations it can encapsulate and
forward the Jumbo Probe with PMTU set to the known value and with no
additional padding octets. In a second option, the OAL source can
encapsulate the Jumbo Probe in the adaptation layer IPv6 header with
a jumbo payload option and with NULL padding octets added beyond the
end of the encapsulated Jumbo Probe to form a large adaptation layer
probe. The OAL source then forwards the probe via the path toward
the OAL destination, where it may be lost due to a link restriction.
If the probe somehow traverses the path, the OAL destination then
removes the adaptation layer encapsulation (while discarding the
padding), resets the Jumbo Probe PMTU according to the padding size,
then forwards the probe further toward the final destination.
If the Jumbo Probe reaches the final destination, the final
destination returns a positive Jumbo Reply with the PMTU set to the
maximum-sized jumbogram that can transit the path. (Note that the
jumbo probing process is conducted independently of any parcel
probing, and the two processes may yield very different results.)
11. Implementation Status
Common widely-deployed implementations include services such as TCP
Segmentation Offload (TSO) and Generic Segmentation/Receive Offload
(GSO/GRO). These services support a robust (but non-standard)
service that has been shown to improve performance in many instances.
Templin Expires 30 July 2023 [Page 29]
Internet-Draft IP Parcels January 2023
UDP/IPv4 parcels have been implemented in the linux-5.10.67 kernel
and ION-DTN ion-open-source-4.1.0 source distributions. Patch
distribution found at: "https://github.com/fltemplin/ip-parcels.git".
Performance analysis with a single-threaded receiver has shown that
including increasing numbers of segments in a single parcel produces
measurable performance gains over fewer numbers of segments due to
more efficient packaging and reduced system calls/interrupts. For
example, sending parcels with 30 2000-octet segments shows a 48%
performance increase in comparison with ordinary IP packets with a
single 2000-octet segment.
Since performance is strongly bounded by single-segment receiver
processing time (with larger segments producing dramatic performance
increases), it is expected that parcels with increasing numbers of
segments will provide a performance multiplier on multi-threaded
receivers in parallel processing environments.
12. IANA Considerations
The IANA is instructed to change the "MTUP - MTU Probe" entry in the
'ip option numbers' registry to the "JUMBO - IPv4 Jumbo Payload"
option. The Copy and Class fields must both be set to 0, and the
Number and Value fields must both be set to '11'. The reference must
be changed to this document [RFCXXXX].
13. Security Considerations
In the control plane, original sources match the Identification
values in received Parcel Replys with their corresponding Parcels or
Parcel Probes. If the values match, the reply is likely authentic.
In environments where stronger authentication is necessary, nodes
that send Parcel Replys can apply the message authentication services
specified for AERO/OMNI.
In the data plane, multi-layer security solutions may be needed to
ensure confidentiality, integrity and availability. Since parcels
are defined only for TCP and UDP, IP layer securing services such as
IPsec-AH/ESP [RFC4301] cannot be applied directly to parcels,
although they can certainly be used at lower layers such as for
transmission of parcels over VPNs and/or OMNI link secured spanning
trees. Since the IP layer does not manipulate segments exchanged
with upper layers, parcels do not interfere with transport- or
higher-layer security services such as (D)TLS/SSL [RFC8446] which may
provide greater flexibility in some environments.
Further security considerations related to IP parcels are found in
the AERO/OMNI specifications.
Templin Expires 30 July 2023 [Page 30]
Internet-Draft IP Parcels January 2023
14. Acknowledgements
This work was inspired by ongoing AERO/OMNI/DTN investigations. The
concepts were further motivated through discussions on the IETF
intarea and 6man lists as well as with Boeing colleagues.
A considerable body of work over recent years has produced useful
"segmentation offload" facilities available in widely-deployed
implementations.
15. References
15.1. Normative References
[RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768,
DOI 10.17487/RFC0768, August 1980,
<https://www.rfc-editor.org/info/rfc768>.
[RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791,
DOI 10.17487/RFC0791, September 1981,
<https://www.rfc-editor.org/info/rfc791>.
[RFC0792] Postel, J., "Internet Control Message Protocol", STD 5,
RFC 792, DOI 10.17487/RFC0792, September 1981,
<https://www.rfc-editor.org/info/rfc792>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC2675] Borman, D., Deering, S., and R. Hinden, "IPv6 Jumbograms",
RFC 2675, DOI 10.17487/RFC2675, August 1999,
<https://www.rfc-editor.org/info/rfc2675>.
[RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing
Architecture", RFC 4291, DOI 10.17487/RFC4291, February
2006, <https://www.rfc-editor.org/info/rfc4291>.
[RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet
Control Message Protocol (ICMPv6) for the Internet
Protocol Version 6 (IPv6) Specification", STD 89,
RFC 4443, DOI 10.17487/RFC4443, March 2006,
<https://www.rfc-editor.org/info/rfc4443>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
Templin Expires 30 July 2023 [Page 31]
Internet-Draft IP Parcels January 2023
[RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", STD 86, RFC 8200,
DOI 10.17487/RFC8200, July 2017,
<https://www.rfc-editor.org/info/rfc8200>.
[RFC9293] Eddy, W., Ed., "Transmission Control Protocol (TCP)",
STD 7, RFC 9293, DOI 10.17487/RFC9293, August 2022,
<https://www.rfc-editor.org/info/rfc9293>.
15.2. Informative References
[BIG-TCP] Dumazet, E., "BIG TCP, Netdev 0x15 Conference (virtual),
https://netdevconf.info/0x15/session.html?BIG-TCP", 31
August 2021.
[I-D.ietf-6man-mtu-option]
Hinden, R. M. and G. Fairhurst, "IPv6 Minimum Path MTU
Hop-by-Hop Option", Work in Progress, Internet-Draft,
draft-ietf-6man-mtu-option-15, 10 May 2022,
<https://www.ietf.org/archive/id/draft-ietf-6man-mtu-
option-15.txt>.
[I-D.templin-dtn-ltpfrag]
Templin, F., "LTP Fragmentation", Work in Progress,
Internet-Draft, draft-templin-dtn-ltpfrag-09, 25 July
2022, <https://www.ietf.org/archive/id/draft-templin-dtn-
ltpfrag-09.txt>.
[I-D.templin-intarea-aero]
Templin, F., "Automatic Extended Route Optimization
(AERO)", Work in Progress, Internet-Draft, draft-templin-
intarea-aero-14, 10 January 2023,
<https://www.ietf.org/archive/id/draft-templin-intarea-
aero-14.txt>.
[I-D.templin-intarea-omni]
Templin, F. L., "Transmission of IP Packets over Overlay
Multilink Network (OMNI) Interfaces", Work in Progress,
Internet-Draft, draft-templin-intarea-omni-15, 23 January
2023, <https://datatracker.ietf.org/api/v1/doc/document/
draft-templin-intarea-omni/>.
[QUIC] Ghedini, A., "Accelerating UDP packet transmission for
QUIC, https://blog.cloudflare.com/accelerating-udp-packet-
transmission-for-quic/", 8 January 2020.
Templin Expires 30 July 2023 [Page 32]
Internet-Draft IP Parcels January 2023
[RFC1063] Mogul, J., Kent, C., Partridge, C., and K. McCloghrie, "IP
MTU discovery options", RFC 1063, DOI 10.17487/RFC1063,
July 1988, <https://www.rfc-editor.org/info/rfc1063>.
[RFC1071] Braden, R., Borman, D., and C. Partridge, "Computing the
Internet checksum", RFC 1071, DOI 10.17487/RFC1071,
September 1988, <https://www.rfc-editor.org/info/rfc1071>.
[RFC1191] Mogul, J. and S. Deering, "Path MTU discovery", RFC 1191,
DOI 10.17487/RFC1191, November 1990,
<https://www.rfc-editor.org/info/rfc1191>.
[RFC4301] Kent, S. and K. Seo, "Security Architecture for the
Internet Protocol", RFC 4301, DOI 10.17487/RFC4301,
December 2005, <https://www.rfc-editor.org/info/rfc4301>.
[RFC4821] Mathis, M. and J. Heffner, "Packetization Layer Path MTU
Discovery", RFC 4821, DOI 10.17487/RFC4821, March 2007,
<https://www.rfc-editor.org/info/rfc4821>.
[RFC5326] Ramadas, M., Burleigh, S., and S. Farrell, "Licklider
Transmission Protocol - Specification", RFC 5326,
DOI 10.17487/RFC5326, September 2008,
<https://www.rfc-editor.org/info/rfc5326>.
[RFC7126] Gont, F., Atkinson, R., and C. Pignataro, "Recommendations
on Filtering of IPv4 Packets Containing IPv4 Options",
BCP 186, RFC 7126, DOI 10.17487/RFC7126, February 2014,
<https://www.rfc-editor.org/info/rfc7126>.
[RFC8201] McCann, J., Deering, S., Mogul, J., and R. Hinden, Ed.,
"Path MTU Discovery for IP version 6", STD 87, RFC 8201,
DOI 10.17487/RFC8201, July 2017,
<https://www.rfc-editor.org/info/rfc8201>.
[RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol
Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
<https://www.rfc-editor.org/info/rfc8446>.
[RFC8899] Fairhurst, G., Jones, T., Tüxen, M., Rüngeler, I., and T.
Völker, "Packetization Layer Path MTU Discovery for
Datagram Transports", RFC 8899, DOI 10.17487/RFC8899,
September 2020, <https://www.rfc-editor.org/info/rfc8899>.
[RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
Multiplexed and Secure Transport", RFC 9000,
DOI 10.17487/RFC9000, May 2021,
<https://www.rfc-editor.org/info/rfc9000>.
Templin Expires 30 July 2023 [Page 33]
Internet-Draft IP Parcels January 2023
[RFC9171] Burleigh, S., Fall, K., and E. Birrane, III, "Bundle
Protocol Version 7", RFC 9171, DOI 10.17487/RFC9171,
January 2022, <https://www.rfc-editor.org/info/rfc9171>.
Appendix A. IP Parcel Futures
Both historic and modern-day data links configure Maximum
Transmission Units (MTUs) that are far smaller than the desired state
for IP parcel transmission futures. When the first Ethernet data
links were deployed many decades ago, their 1500 octet MTU set a
strong precedent that was widely adopted. This same size now appears
as the predominant MTU limit for most paths in the Internet today,
although modern link deployments with MTUs as large as 9KB have begun
to emerge.
In the late 1980's, the Fiber Distributed Data Interface (FDDI)
standard defined a new link type with MTU slightly larger than 4500
octets. The goal of the larger MTU was to increase performance by a
factor of 10 over the ubiquitous 10Mbps and 1500-octet MTU Ethernet
technologies of the time. Many factors including a failure to
harmonize MTU diversity and an Ethernet performance increase to
100Mbps led to poor FDDI market reception. In the next decade, the
1990's saw new initiatives including ATM/AAL5 (9KB MTU) and HiPPI
(64KB MTU) which offered high-speed data link alternatives with
larger MTUs but again the inability to harmonize diversity derailed
their momentum. By the end of the 1990s and leading into the 2000's,
emergence of the 1Gbps, 10Gbps and even faster Ethernet performance
levels seen today has obscured the fact that the modern Internet of
the 21st century is still operating with 20th century MTUs!
To bridge this gap, increased OMNI interface deployment in the near
future will provide a virtual link type that can pass IP parcels over
paths that traverse traditional data links with small MTUs.
Performance analysis has proven that (single-threaded) receive-side
performance is bounded by upper layer protocol segment size, with
performance increasing in direct proportion with segment size.
Experiments have also shown measurable (single-threaded) performance
increases by including larger numbers of segments per parcel, with
steady increases for including increasing number of segments.
However, parallel receive-side processing will provide performance
multiplier benefits since the multiple segments that arrive in a
single parcel can be processed simultaneously instead of serially.
In addition to the clear near-term benefits, IP parcels will increase
performance to new levels as future parcel-capable links with very
large MTUs begin to emerge. These links will provide MTUs far in
excess of 64KB to as large as 16MB. With such large MTUs, the
traditional CRC-32 (or even CRC-64) error checking with errored
Templin Expires 30 July 2023 [Page 34]
Internet-Draft IP Parcels January 2023
packet discard discipline will no longer apply for large parcels.
Instead, parcels larger than a link-specific threshold will include
Forward Error Correction (FEC) codes so that errored parcels can be
repaired at the receiver's data link layer then delivered to upper
layers rather than being discarded and triggering retransmission of
large amounts of data. Even if the FEC repairs are incomplete or
imperfect, all parcels can still be delivered to upper layers where
the individual segment checksums will detect and discard any damaged
data not repaired by lower layers.
These new "super-links" will appear mostly in the network edges
(e.g., high-performance data centers) and not as often in the middle
of the Internet. (However, some space-domain links that extend over
enormous distances may also benefit.) For this reason, a common use
case will include parcel-capable super-links in the edge networks of
both parties of an end-to-end session with an OMNI link connecting
the two over wide area Internetworks. Medium- to moderately large-
sized IP parcels over OMNI links will already provide considerable
performance benefits for wide-area end-to-end communications while
truly large IP parcels over super-links can provide boundless
increases for localized bulk transfers in edge networks or for deep
space long haul transmissions. The ability to grow and adapt without
practical bound enabled by IP parcels will inevitably encourage new
data link development leading to future innovations in new markets
that will revolutionize the Internet.
Until these new links begin to emerge, however, parcels will already
provide a tremendous benefit to end systems by allowing applications
to send and receive segment buffers larger than 65535 octets in a
single system call. By expanding the current operating system call
data copy limit from its current 16-bit length to a 32-bit length,
applications will be able to send and receive maximum-length parcel
buffers even if lower layers need to break them into multiple parcels
to fit within the underlying interface MTU. For applications such as
the Delay Tolerant Networking (DTN) Bundle Protocol [RFC9171], this
will allow applications to send and receive entire large upper layer
protocol constructs (such as DTN bundles) in a single system call.
Appendix B. Change Log
<< RFC Editor - remove prior to publication >>
Changes from earlier versions:
* Submit for Intarea Standards Track RFC Publication.
Author's Address
Templin Expires 30 July 2023 [Page 35]
Internet-Draft IP Parcels January 2023
Fred L. Templin (editor)
Boeing Research & Technology
P.O. Box 3707
Seattle, WA 98124
United States of America
Email: fltemplin@acm.org
Templin Expires 30 July 2023 [Page 36]