IP Parcels and Advanced Jumbos
draft-templin-intarea-parcels-88
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft whose latest revision state is "Replaced".
|
|
|---|---|---|---|
| Author | Fred Templin | ||
| Last updated | 2023-11-17 (Latest revision 2023-11-16) | ||
| Replaced by | draft-templin-intarea-parcels2 | ||
| RFC stream | (None) | ||
| Formats | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-templin-intarea-parcels-88
Network Working Group F. L. Templin, Ed.
Internet-Draft Boeing Research & Technology
Updates: 2675, 9268 (if approved) 17 November 2023
Intended status: Standards Track
Expires: 20 May 2024
IP Parcels and Advanced Jumbos
draft-templin-intarea-parcels-88
Abstract
IP packets (both IPv4 and IPv6) contain a single unit of transport
layer protocol data which becomes the retransmission unit in case of
loss. Transport layer protocols including the Transmission Control
Protocol (TCP) and reliable transport protocol users of the User
Datagram Protocol (UDP) prepare data units known as segments which
the network layer packages into individual IP packets each containing
only a single segment. This specification presents new packet
constructs known as IP Parcels and Advanced Jumbos with different
properties. IP parcels permit a single packet to include multiple
segments as a "packet-of-packets", while advanced jumbos offer
significant operational advantages over basic jumbograms for
transporting truly large singleton segments. IP parcels and advanced
jumbos provide essential building blocks for improved performance,
efficiency and integrity while encouraging larger Maximum
Transmission Units (MTUs) in the Internet.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 20 May 2024.
Templin Expires 20 May 2024 [Page 1]
Internet-Draft IP Parcels November 2023
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Requirements . . . . . . . . . . . . . . . . . . . . . . . . 8
4. Background and Motivation . . . . . . . . . . . . . . . . . . 9
5. IP Parcel and Advanced Jumbo Link Service Model . . . . . . . 10
6. IP Parcel Formation . . . . . . . . . . . . . . . . . . . . . 12
6.1. TCP Parcels . . . . . . . . . . . . . . . . . . . . . . . 15
6.2. UDP Parcels . . . . . . . . . . . . . . . . . . . . . . . 16
6.3. Calculating J and K . . . . . . . . . . . . . . . . . . . 16
7. Transmission of IP Parcels . . . . . . . . . . . . . . . . . 17
7.1. Packetization over Non-Parcel Links . . . . . . . . . . . 19
7.2. Parcellation over Parcel-capable Links . . . . . . . . . 21
7.3. OMNI Interface Parcellation and Reunification . . . . . . 22
7.4. Final Destination Restoration/Reunification . . . . . . . 24
7.5. Parcel/Jumbo Reports . . . . . . . . . . . . . . . . . . 26
7.6. Parcel/Jumbo Path Probing . . . . . . . . . . . . . . . . 26
8. Advanced Jumbos . . . . . . . . . . . . . . . . . . . . . . . 32
9. Minimal IPv6 Parcels/Advanced Jumbos . . . . . . . . . . . . 36
10. OMNI IP Parcels/Advanced Jumbos . . . . . . . . . . . . . . . 37
11. Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . 39
12. Implementation Status . . . . . . . . . . . . . . . . . . . . 43
13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 43
14. Security Considerations . . . . . . . . . . . . . . . . . . . 45
15. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 46
16. References . . . . . . . . . . . . . . . . . . . . . . . . . 47
16.1. Normative References . . . . . . . . . . . . . . . . . . 47
16.2. Informative References . . . . . . . . . . . . . . . . . 48
Appendix A. TCP Extensions for High Performance . . . . . . . . 51
Appendix B. Extreme L Value Implications . . . . . . . . . . . . 52
Appendix C. Additional Parcel/Jumbo Probe Considerations . . . . 53
Appendix D. Advanced Jumbo Cyclic Redundancy Check (CRC128J) . . 54
Appendix E. Change Log . . . . . . . . . . . . . . . . . . . . . 54
Templin Expires 20 May 2024 [Page 2]
Internet-Draft IP Parcels November 2023
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 54
1. Introduction
IP packets (both IPv4 [RFC0791] and IPv6 [RFC8200]) contain a single
unit of transport layer protocol data which becomes the
retransmission unit in case of loss. Transport layer protocols such
as the Transmission Control Protocol (TCP) [RFC9293] and reliable
transport protocol users of the User Datagram Protocol (UDP)
[RFC0768] (including QUIC [RFC9000], LTP [RFC5326] and others)
prepare data units known as segments which the network layer packages
into individual IP packets each containing only a single segment.
This document presents a new construct known as the IP Parcel which
permits a single packet to include multiple segments. The parcel is
essentially a "packet-of-packets" with the full {TCP,UDP}/IP headers
appearing only once but with possibly multiple segments included.
Transport layer protocol entities form parcels by preparing a data
buffer (or buffer chain) containing at most 64 consecutive transport
layer protocol segments that can be broken out into individual
packets and/or smaller sub-parcels if necessary. All segments except
the final one must be equal in length and no larger than 65535
octets, while the final segment must be no larger than the others.
The transport layer protocol entity then delivers the buffer(s),
number of segments and non-final segment size to the network layer.
The network layer next appends an Internet Checksum header and a
Cyclic Redundancy Check (CRC) trailer to each segment, merges the
segments into the parcel body, appends a {TCP,UDP} header and finally
appends an IP header plus extensions that identify this as a parcel
and not an ordinary packet.
The network layer then forwards each parcel over consecutive parcel-
capable links in a path until they arrive at a node with a next hop
link that does not support parcels, a parcel-capable link with a size
restriction, or an ingress Overlay Multilink Network (OMNI) Interface
[I-D.templin-intarea-omni] connection to an OMNI link that spans
intermediate Internetworks. In the first case, the original source
or next hop router applies packetization to break the parcel into
individual IP packets. In the second case, the node applies network
layer parcellation to form smaller sub-parcels. In the final case,
the OMNI interface applies adaptation layer parcellation to form
still smaller sub-parcels, then applies adaptation layer IPv6
encapsulation and fragmentation if necessary. The node then forwards
the resulting packets/parcels/fragments to the next hop.
Following IPv6 reassembly if necessary, an egress OMNI interface
applies adaptation layer reunification if necessary to merge multiple
sub-parcels into a minimum number of larger (sub-)parcels then
Templin Expires 20 May 2024 [Page 3]
Internet-Draft IP Parcels November 2023
delivers them to the network layer which either processes them
locally or forwards them via the next hop link toward the final
destination. The final destination can then apply network layer
(parcel-based) reunification or (packet-based) restoration if
necessary to deliver a minimum number of larger (sub-)parcels to the
transport layer. Reordering, loss or corruption of individual
segments within the network is therefore possible, but most
importantly the parcels delivered to the final destination's
transport layer should be the largest practical size for best
performance, and loss or receipt of individual segments (rather than
parcel size) determines the retransmission unit.
This document further introduces an advanced jumbo service that
provides essential extensions beyond the basic IPv6 jumbogram service
defined in [RFC2675]. Advanced jumbos are defined for both IP
protocol versions and provide end systems and routers with a more
robust service when the transmission of truly large singleton
segments is necessary.
The following sections discuss rationale for creating and shipping IP
parcels and advanced jumbos as well as actual protocol constructs and
procedures involved. IP parcels and advanced jumbos provide
essential building blocks for improved performance, efficiency and
integrity while encouraging larger Maximum Transmission Units (MTUs).
These services should further inspire future innovation in
applications, transport protocols, operating systems, network
equipment and data links in ways that promise to transform the
Internet architecture.
2. Terminology
The Oxford Languages dictionary defines a "parcel" as "a thing or
collection of things wrapped in paper in order to be carried or sent
by mail". Indeed, there are many examples of parcel delivery
services worldwide that provide an essential transit backbone for
efficient business and consumer transactions.
In this same spirit, an "IP parcel" is simply a collection of at most
64 transport layer protocol segments wrapped in an efficient package
for transmission and delivery as a "packet-of-packets", with each
segment preceded by an end-to-end integrity check to detect link
errors. IP parcels are distinguished from ordinary packets and
various forms of jumbograms through the constructs specified in this
document.
The IP parcel construct is defined for both IPv4 and IPv6. Where the
document refers to "IPv4 header length", it means the total length of
the base IPv4 header plus all included options, i.e., as determined
Templin Expires 20 May 2024 [Page 4]
Internet-Draft IP Parcels November 2023
by consulting the Internet Header Length (IHL) field. Where the
document refers to "IPv6 header length", however, it means only the
length of the base IPv6 header (i.e., 40 octets), while the length of
any extension headers is referred to separately as the "IPv6
extension header length". Finally, the term "IP header plus
extensions" refers generically to an IPv4 header plus all included
options or an IPv6 header plus all included extension headers.
The term "advanced jumbo" refers to a new type of IP jumbogram
defined for both IP protocol versions and modeled from the basic IPv6
jumbogram construct defined in [RFC2675]. Advanced jumbos include a
32-bit Jumbo Payload Length field the same as for basic IPv6
jumbograms, but are differentiated from parcels and other jumbogram
types by including an "Advanced Jumbo Type" value in the IP {Total,
Payload} Length field and an end-to-end segment integrity check to
detect link errors.
Where the document refers to "{TCP,UDP} header length", it means the
length of either the TCP header plus options (20 or more octets) or
the UDP header (8 octets). It is important to note that only a
single IP header and a single full {TCP,UDP} header appears in each
parcel regardless of the number of segments included. This
distinction often provides a significant overhead savings advantage
made possible only by IP parcels.
Where the document refers to checksum calculations, it means the
standard Internet checksum unless otherwise specified. The same as
for TCP [RFC9293], UDP [RFC0768] and IPv4 [RFC0791], the standard
Internet checksum is defined as (sic) "the 16-bit one's complement of
the one's complement sum of all (pseudo-)headers plus data, padded
with zero octets at the end (if necessary) to make a multiple of two
octets". A notional Internet checksum algorithm can be found in
[RFC1071], while practical implementations require detailed attention
to network byte ordering to ensure interoperability between diverse
architectures.
The term Cyclic Redundancy Check (CRC) is used consistently with its
application in widely deployed Internetworking services. The CRC32C
[RFC3385] and CRC64E [ECMA-182] standards are selected for IP parcels
according to non-final segment length "L" (see: Section 11).
Advanced jumbos include either a CRC or message digest calculated
according to the MD5 [RFC1321], SHA1 [RFC3174] or US Secure Hash
[RFC6234] algorithms. In all cases, the CRC or message digest is
appended as a per-segment trailer arranged for transmission in
network byte order per standard Internetworking conventions.
Templin Expires 20 May 2024 [Page 5]
Internet-Draft IP Parcels November 2023
The terms "application layer (L5 and higher)", "transport layer
(L4)", "network layer (L3)", "(data) link layer (L2)" and "physical
layer (L1)" are used consistently with common Internetworking
terminology, with the understanding that reliable delivery protocol
users of UDP are considered as transport layer elements. The OMNI
specification further defines an "adaptation layer" logically
positioned below the network layer but above the link layer (which
may include physical links and Internet- or higher-layer tunnels).
The adaptation layer is not associated with a layer number itself and
is simply known as "the layer below L3 but above L2". A network
interface is a node's attachment to a link (via L2), and an OMNI
interface is therefore a node's attachment to an OMNI link (via the
adaptation layer).
The term "parcel/jumbo-capable link/path" refers to paths that
transit interfaces to adaptation layer and/or link layer media
(either physical or virtual) capable of transiting {TCP,UDP}/IP
packets that employ the parcel/jumbo constructs specified in this
document. The source and each router in the path has a "next hop
link" that forwards parcels/jumbos toward the final destination,
while each router and the final destination has a "previous hop link"
that accepts en route parcels/jumbos. Each next hop link must be
capable of forwarding parcels/jumbos (after first applying
parcellation if necessary) with segment lengths no larger than can
transit the link. Currently only the OMNI link satisfies these
properties, while other link types that support parcels/jumbos should
soon follow.
The term "5-tuple" refers to a transport layer protocol entity
identifier that includes the network layer (source address,
destination address, source port, destination port, protocol number).
The term "4-tuple" refers to a network layer parcel entity identifier
that includes the adaptation layer (source address, destination
address, Parcel ID, Identification).
Templin Expires 20 May 2024 [Page 6]
Internet-Draft IP Parcels November 2023
The Internetworking term "Maximum Transmission Unit (MTU)" is widely
understood to mean the largest packet size that can transit a single
link ("link MTU") or an entire path ("path MTU") without requiring
network layer IP fragmentation. If the MTU value returned during
parcel path qualification is larger than 65535 (plus the length of
the parcel headers), it determines the maximum-sized parcel or jumbo
that can transit the link/path without requiring a router to perform
packetization/parcellation. If the MTU is no larger than 65535, the
value instead determines the "Maximum Segment Size (MSS)" for the
leading portion of the path up to a router that cannot forward the
parcel further. (Note that this size may still be larger than the
MSS that can transit the remainder of the path to the final
destination, which can only be determined through explicit MSS
probing.)
The terms "packetization" and "restoration" refer to a network layer
process in which the original source or a router on the path breaks a
parcel out into individual IP packets that can transit the remainder
of the path without loss due to a size restriction. The final
destination then restores the combined packet contents into a parcel
before delivery to the transport layer. In current practice,
packetization/restoration can be considered as functional equivalents
to the well-known Generic Segmentation/Receive Offload (GSO/GRO)
services.
The terms "parcellation" and "reunification" refer to either network
layer or adaptation layer processes in which the original source or a
router on the path breaks a parcel into smaller sub-parcels that can
transit the path without loss due to a size restriction. These sub-
parcels are then reunified into larger (sub-)parcels before delivery
to the transport layer. As a network layer process, the sub-parcels
resulting from parcellation may only be reunified at the final
destination. As an adaptation layer process, the resulting sub-
parcels may be first reunified at an adaptation layer egress node
then possibly further reunified by the network layer of the final
destination.
The terms "fragmentation" and "reassembly" follow exactly from their
definitions in the IPv4 [RFC0791] and IPv6 [RFC8200] standards. In
particular, OMNI interfaces support IPv6 encapsulation and
fragmentation as an adaptation layer process that can transit packets
or (sub-)parcels of sizes that exceed the underlying Internetwork
path MTU. OMNI fragmentation/reassembly occurs at a lower layer of
the protocol stack than restoration and/or reunification and
therefore provides a complimentary service.
Templin Expires 20 May 2024 [Page 7]
Internet-Draft IP Parcels November 2023
"Automatic Extended Route Optimization (AERO)"
[I-D.templin-intarea-aero] and the "Overlay Multilink Network
Interface (OMNI)" [I-D.templin-intarea-omni] provide an adaptation
layer framework for transmission of IP parcels and advanced jumbos
over one or more concatenated Internetworks. AERO/OMNI will provide
an operational environment for IP parcels beginning from the earliest
deployment phases and extending indefinitely to accommodate
continuous future growth. As more and more parcel/jumbo-capable
links are enabled (e.g., in data centers, edge networks, space-
domain, and other high data rate services) AERO/OMNI will continue to
provide an essential service for Internetworking performance
maximization.
The parcel sizing variables "J", "K", "L" and "M" are cited
extensively throughout this document. "J" denotes the number of non-
final segments included in the parcel, "K" is the length of the final
segment, "L" is the length of each non-final segment and "M" is
termed the "Parcel Payload Length".
3. Requirements
IP parcels and advanced jumbos are derived from the basic jumbogram
specification found in [RFC2675], but the specifications in this
document take precedence whenever they differ from the basic
requirements. Most notably, IPv4 parcels use the IPv4 Probe MTU
option [RFC1063] while IPv6 parcels and advanced jumbos use one of
either the IPv6 Minimum Path MTU [RFC9268] or basic IPv6 jumbogram
[RFC2675] Hop-by-Hop option. IP parcels and advanced jumbos are
further permitted to encode values other than 0 in the IP {Total,
Payload} length field and they are not limited to packet sizes that
exceed 65535 octets. (Instead, IP parcels can be as small as the
packet headers plus a singleton segment while advanced jumbos can be
as small as the headers plus a NULL payload.)
Each IPv4 parcel/advanced jumbo includes at most one Probe MTU option
and each IPv6 parcel/advanced jumbo includes at most one IPv6 Minimum
Path MTU or Jumbo Payload Hop-by-Hop option. Intermediate and end
systems therefore silently drop any IP parcels/advanced jumbos that
include multiple. For further Hop-by-Hop option considerations, see:
[I-D.ietf-6man-hbh-processing]. For IPv6 extension header limits,
see: [I-D.ietf-6man-eh-limits].
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119][RFC8174] when, and only when, they appear in all
capitals, as shown here.
Templin Expires 20 May 2024 [Page 8]
Internet-Draft IP Parcels November 2023
4. Background and Motivation
Studies have shown that applications can improve their performance by
sending and receiving larger packets due to reduced numbers of system
calls and interrupts as well as larger atomic data copies between
kernel and user space. Larger packets also result in reduced numbers
of network device interrupts and better network utilization (e.g.,
due to header overhead reduction) in comparison with smaller packets.
A first study [QUIC] involved performance enhancement of the QUIC
protocol [RFC9000] using the linux Generic Segment/Receive Offload
(GSO/GRO) facility. GSO/GRO provides a robust service that has shown
significant performance increases based on a multi-segment transfer
capability between the operating system kernel and QUIC applications.
GSO/GRO performs (virtual) fragmentation and reassembly at the
transport layer with the transport protocol segment size limited by
the path MTU (typically 1500 octets or smaller in today's Internet).
A second study [I-D.templin-dtn-ltpfrag] showed that GSO/GRO also
improves performance for the Licklider Transmission Protocol (LTP)
[RFC5326] used for the Delay Tolerant Networking (DTN) Bundle
Protocol [RFC9171] for segments larger than the actual path MTU
through the use of OMNI interface encapsulation and fragmentation.
Historically, the NFS protocol also saw significant performance
increases using larger (single-segment) UDP datagrams even when IP
fragmentation is invoked, and LTP still follows this profile today.
Moreover, LTP shows this (single-segment) performance increase
profile extending to the largest possible segment size which suggests
that additional performance gains are possible using (multi-segment)
IP parcels that approach or even exceed 65535 octets in total length.
TCP also benefits from larger packet sizes and efforts have
investigated TCP performance using jumbograms internally with changes
to the linux GSO/GRO facilities [BIG-TCP]. The approach proposed to
use the Jumbo Payload option internally and to allow GSO/GRO to use
buffer sizes that exceed 65535 octets, but with the understanding
that links that support jumbograms natively are not yet widely
deployed and/or enabled. Hence, IP parcels provide a packaging that
can be considered in the near term under current deployment
limitations.
A limiting consideration for sending large packets is that they are
often lost at links with MTU restrictions, and the resulting Packet
Too Big (PTB) messages [RFC1191][RFC8201] may be lost somewhere in
the return path to the original source. This path MTU "black hole"
condition can degrade performance unless robust path probing
techniques are used, however the best case performance always occurs
when loss of packets due to size restrictions is minimized.
Templin Expires 20 May 2024 [Page 9]
Internet-Draft IP Parcels November 2023
These considerations therefore motivate a design where transport
protocols can employ segment sizes as large as 65535 octets (minus
headers) while parcels that carry multiple segments may themselves be
significantly larger. This would allow the receiving transport layer
protocol entity to process multiple segments in parallel instead of
one at a time per existing practices. Parcels therefore support
improvements in performance, integrity and efficiency for the
original source, final destination and networked path as a whole.
This is true even if the network and lower layers need to apply
packetization/restoration, parcellation/reunification and/or
fragmentation/reassembly.
An analogy: when a consumer orders 50 small items from a major online
retailer, the retailer does not ship the order in 50 separate small
boxes. Instead, the retailer packs as many of the small items as
possible into one or a few larger boxes (i.e., parcels) then places
the parcels on a semi-truck or airplane. The parcels may then pass
through one or more regional distribution centers where they may be
repackaged into different parcel configurations and forwarded further
until they are finally delivered to the consumer. But most often,
the consumer will only find one or a few parcels at their doorstep
and not 50 separate small boxes. This flexible parcel delivery
service greatly reduces shipping and handling cost for all including
the retailer, regional distribution centers and finally the consumer.
5. IP Parcel and Advanced Jumbo Link Service Model
The classical Internetworking link service model requires each link
in the path to apply a link-layer frame integrity check often termed
a "Frame Check Sequence (FCS)". The link near-end calculates and
appends a FCS trailer to each packet pending transmission, and the
link far-end verifies the FCS upon packet reception. If verification
fails, the link far-end unconditionally discards the packet. This
process is repeated for each link in the path so that only packets
that pass all link-layer checks are delivered to the final
destination.
While this link service model has contributed to the unparalleled
success of terrestrial Internetworks (including the global public
Internet), new uses in which significant delays or disruptions can
occur are not as well supported. For example, a path that contains
links with higher bit error rates may be unable to pass a majority
percentage of packets since loss due to link errors can occur at any
hop. Moreover, packets that incur link errors but somehow pass the
link integrity check will be forwarded by all remaining links in the
path exposing the final destination to undetected errors. Advanced
error detection and correction services not typically associated with
packets are therefore necessary; especially with the advent of space-
Templin Expires 20 May 2024 [Page 10]
Internet-Draft IP Parcels November 2023
domain Internetworking, the long delays associated with
interplanetary signal propagation are often intolerant of
retransmissions.
IP parcels and advanced jumbos include an end-to-end Cyclic
Redundancy Check (CRC) with each segment that is calculated and
inserted by the original source and verified by the final
destination. For each IP parcel or advanced jumbo admitted into a
parcel-capable link, the link near-end applies its standard link-
layer FCS upon transmission which the link far-end then verifies upon
reception. Instead of unconditionally discarding frames with link
errors, however, the link far-end delivers all parcel/advanced jumbo
frames to upper layers along with an error flag that is set if a link
error was detected or cleared otherwise.
Each link along the path simply discards any ordinary IP packets that
have incurred link errors according to current practice. For IP
parcels and advanced jumbos received with link errors, however, each
intermediate hop SHOULD and the final destination MUST first verify
the parcel/jumbo header Checksum to protect against mis-delivery.
Each intermediate hop then unconditionally forwards the parcel/
advanced jumbo to the next hop even though it includes link errors.
The IP parcel/advanced jumbo segments may therefore acquire
cumulative link errors along the path, but these will be detected by
the per segment end-to-end CRC and/or Internet checksums performed by
the final destination. The final destination in turn delivers each
segment to the local transport layer along with an error flag that is
set if an end-to-end CRC or Internet checksum error was detected
(otherwise the flag is cleared). The error indication is then taken
under advisement by the transport layer, which should employ
transport or higher-layer integrity checks to guide any corrective
actions.
IP parcels and advanced jumbos therefore provide a revolutionary
advancement for delay/disruption tolerance in air/land/sea/space
mobile Internetworking applications. As the Internet continues to
evolve from its more stable fixed terrestrial network origins to one
where more and more nodes operate in the mobile edge, this new link
service model relocates error detection and correction
responsibilities from intermediate systems to the end systems that
are uniquely capable of take corrective actions.
Note: IP parcels and advanced jumbos may already be compatible with
widely-deployed link types such as 1/10/100-Gbps Ethernet. Each
Ethernet frame is identified by a preamble followed by a Start Frame
Delimiter (SFD) followed by the frame data itself followed by the FCS
and finally an Inter Packet Gap (IPG). Since no length field is
Templin Expires 20 May 2024 [Page 11]
Internet-Draft IP Parcels November 2023
included, however, the frame can theoretically extend as long as
necessary for transmission of IP parcels and advanced jumbos that are
much larger than the typical 1500 octet Ethernet MTU as long as the
time duration on the link media is properly bounded. Widely-deployed
links may therefore already include all of the necessary features to
natively support large IP parcels and advanced jumbos with no
additional extensions, while operating systems may need to be
modified to post larger receive buffers.
6. IP Parcel Formation
A transport protocol entity identified by its 5-tuple forms a parcel
body by preparing a data buffer (or buffer chain) containing at most
64 transport layer protocol segments, with each TCP segment preceded
by a 4-octet Sequence Number header and with each segment (plus
Sequence Number) preceded by a 2-octet Internet Checksum header and
followed by a 4-octet or 8-octet CRC trailer. All non-final segments
MUST be equal in length while the final segment MUST NOT be larger
and MAY be smaller. The number of non-final segments is represented
as J; the total number of segments is therefore (J + 1).
The non-final segment size L is set to a 16-bit value that MUST be no
smaller than 256 octets and SHOULD be no larger than 65535 octets
minus the length of the {TCP,UDP} header (plus options), minus the
length of the IP header (plus options/extensions), minus 2 octets for
the Checksum header minus 4/8 octets for the CRC trailer (see:
Appendix B). The final segment length K MUST NOT be larger than L
but MAY be smaller. The transport layer protocol entity then
presents the buffer(s) and size L to the network layer, noting that
the combined buffer length(s) may exceed 65535 octets when there are
sufficient segments of a large enough size.
If the next hop link is not parcel capable, the network layer
performs packetization to package each segment as an individual IP
packet as discussed in Section 7.1. If the next hop link is parcel
capable, the network layer instead completes the parcel by appending
an Internet Checksum header and CRC trailer to each segment then
appending a single full {TCP,UDP} header (plus options) and a single
full IP header (plus options/extensions). The network layer finally
includes a specially-formatted Parcel Payload option as an extension
to the IP header of each parcel prior to transmission over a network
interface.
The Parcel Payload option format for both IP protocol versions
appears as shown in Figure 1:
Templin Expires 20 May 2024 [Page 12]
Internet-Draft IP Parcels November 2023
IPv4 Parcel Payload Option Format
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| option-type | option-length | Code | Check |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Index |P|S| Parcel Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
IPv6 Parcel Payload Option Format
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Opt Data Len | Code | Check |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Index |P|S| Parcel Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1: Parcel Payload Option
For IPv4, the network layer includes the Parcel Payload option as an
IPv4 header option with option-type set to '0x0B' and option-length
set to 8. The length also distinguishes this type from its obsoleted
use as the IPv4 Probe MTU option [RFC1063]. The network layer next
sets Parcel Payload Length to a 3-octet value M that encodes the
length of the IPv4 header plus the length of the {TCP,UDP} header
plus the combined length of all concatenated segments with their CRC
(and for TCP also Sequence Number) headers. The network layer then
sets the IPv4 header DF bit to 1 and Total Length field to the non-
final segment size L.
For IPv6, the network layer includes the Parcel Payload option as an
IPv6 Hop-by-Hop option with Option Type set to '0x30' and Opt Data
Len set to 6. The length also distinguishes this type from its use
as the IPv6 Minimum Path MTU Hop-by-Hop Option [RFC9268]. The
network layer next sets Parcel Payload Length to a 3-octet value M
that encodes the lengths of all IPv6 extension headers present plus
the length of the {TCP,UDP} header plus the combined length of all
concatenated segments with their CRC (and for TCP also Sequence
Number) headers. The network layer then sets the IPv6 header Payload
Length field to L.
Templin Expires 20 May 2024 [Page 13]
Internet-Draft IP Parcels November 2023
For both IP protocol versions, the network layer then sets Index to
an ordinal segment "Parcel Index" value between 0 and 63, sets the
"(P)arcel" flag to 1 and sets the "More (S)egments" flag to 1 for
non-final sub-parcels or 0 for the final (sub-)parcel. (Note that
non-zero Index values identify the initial segment index in non-first
sub-parcels of a larger original parcel while the value 0 denotes the
first sub-parcel.) The network layer finally sets Code to 255 and
sets Check to the same value that will appear in the IP header TTL/
Hop Limit field on transmission. These values provide hop-by-hop
assurance that previous hops correctly implement the parcel protocol
without applying [RFC1063][RFC9268] processing.
Following this transport and network layer processing, {TCP,UDP}/IP
parcels therefore have the structures shown in Figure 2:
TCP/IP Parcel Structure UDP/IP Parcel Structure
+------------------------------+ +------------------------------+
| | | |
~ IP Hdr plus extensions ~ ~ IP Hdr plus extensions ~
| | | |
+------------------------------+ +------------------------------+
| | | |
~ TCP header (plus options) ~ ~ UDP header ~
| | | |
+------------------------------+ +------------------------------+
| Checksum 0 followed by | | Checksum 0 followed by |
~ Sequence Number 0 followed ~ ~ Segment 0 (L octets) ~
~ by Segment 0 (L octets) ~ ~ followed by ~
| followed by CRC 0 | | CRC 0 |
+------------------------------+ +------------------------------+
| Checksum 1 followed by | | Checksum 1 followed by |
~ Sequence Number 1 followed ~ ~ Segment 1 (L octets) ~
~ by Segment 1 (L octets) ~ ~ followed by ~
| followed by CRC 1 | | CRC 1 |
+------------------------------+ +------------------------------+
~ ... ~ ~ ... ~
~ More Segments ~ ~ More Segments ~
~ ... ~ ~ ... ~
+------------------------------+ +------------------------------+
| Checksum J followed by | | Checksum J followed by |
~ Sequence Number J followed ~ ~ Segment J (K octets) ~
~ by Segment J (K octets) ~ ~ followed by ~
| followed by CRC J | | CRC J |
+------------------------------+ +------------------------------+
Figure 2: {TCP,UDP}/IP Parcel Structure
Templin Expires 20 May 2024 [Page 14]
Internet-Draft IP Parcels November 2023
The {TCP,UDP}/IP header is then followed by (J + 1) transport layer
segments. For TCP, the TCP header Sequence Number field encodes the
value 0, and each segment is preceded by its own 4-octet Sequence
Number field with the 4-octet length included in L and K. Each
segment (and its Sequence Number) is then preceded by a 2-octet
Internet Checksum header and followed by a 4/8-octet CRC trailer but
without the Checksum/CRC lengths included in L and K.
6.1. TCP Parcels
A TCP Parcel is an IP Parcel that includes an IP header plus
extensions with a Parcel Payload option formed as shown in Section 6
with Parcel Payload Length encoding a value no larger than 16,777,215
(2**24 - 1) octets. The IP header plus extensions is then followed
by a TCP header plus options (20 or more octets) followed by (J + 1)
consecutive segments that each include a 2-octet Internet Checksum
header and 4/8-octet CRC trailer. The sequence number found in the
TCP header is set to 0, each non-final segment is L octets in length
(including its own 4-octet Sequence Number) and the final segment is
K octets in length (including its own 4-octet Sequence Number). The
value L is encoded in the IP header {Total, Payload} Length field
while the overall length of the parcel is determined by the Parcel
Payload Length M.
The source prepares TCP Parcels in an alternative adaptation of TCP
jumbograms [RFC2675]. The source calculates a checksum of the TCP
header plus IP pseudo-header only (see: Section 11). The source then
writes the exact calculated value in the TCP header Checksum field
(i.e., without converting calculated 0 values to '0xffff').
The source next calculates the Internet checksum for each segment
independently over the length of the segment (beginning with its
sequence number) and writes the value into the 2-octet per-segment
Checksum header. The source then calculates the CRC over the segment
beginning with the Checksum header and writes the value into the
4/8-octet CRC trailer.
Note: The parcel TCP header Source Port, Destination Port and (per-
segment) Sequence Number fields apply to each parcel segment, while
the TCP control bits and all other fields apply only to the first
segment (i.e., "segment(0)"). Therefore, only parcel segment(0) may
be associated with control bit settings while all other segment(i)'s
must be simple data segments.
See Appendix A for additional TCP considerations. See Section 11 for
additional integrity considerations.
Templin Expires 20 May 2024 [Page 15]
Internet-Draft IP Parcels November 2023
6.2. UDP Parcels
A UDP Parcel is an IP Parcel that includes an IP header plus
extensions with a Parcel Payload option formed as shown in Section 6
with Parcel Payload Length encoding a value no larger than 16,777,215
(2**24 - 1) octets. The IP header plus extensions is then followed
by an 8-octet UDP header followed by (J + 1) transport layer segments
with their Checksums and CRCs. Each segment must begin with a
transport-specific start delimiter (e.g., a segment identifier, a
sequence number, etc.) included by the transport layer user of UDP.
The length of the first segment L is encoded in the IP {Total,
Payload} Length field while the overall length of the parcel is
determined by the Parcel Payload Length M as above.
The source prepares UDP Parcels in an alternative adaptation of UDP
jumbograms [RFC2675]. The source first sets the UDP header length
field to 0, then calculates the checksum of the UDP header plus IP
pseudo-header (see: Section 11) and writes the exact calculated value
into the UDP header Checksum field (i.e., without converting
calculated 0 values to '0xffff'). If UDP checksums are enabled, the
source also calculates a separate checksum for each segment while
writing the values into the corresponding per-segment Checksum header
with calculated 0 values converted to '0xffff' (if UDP checksums are
disabled, the source instead writes the value 0). The source then
calculates the CRC over each segment beginning with the segment
Checksum field and writes the value into the 4/8-octet CRC trailer.
See: Section 11 for additional integrity considerations.
6.3. Calculating J and K
The IP parcel source unambiguously encodes the values L and M in the
corresponding header fields as specified above. The values J and K
are not encoded in header fields and must therefore be calculated by
intermediate and final destination nodes as follows:
Templin Expires 20 May 2024 [Page 16]
Internet-Draft IP Parcels November 2023
/* L is non-final segment length (256 or greater);
M is parcel payload length;
H is length of {TCP,UDP}/IP headers plus extensions;
T is parcel payload length minus headers;
C is the combined length of the CRC and Checksum fields;
integer arithmetic assumed.*/
if ((L < 256) || ((T = (M - H)) <= 0))
drop parcel;
if ((J = (T / (L + C))) > 64)
drop parcel;
if ((K = (T % (L + C))) == 0) {
J--; K = L;
} else {
if ((J > 63) || ((K -= C) <= 0))
drop parcel;
}
Figure 3: Calculating J and K
Note: from the above calculations, a minimal IP parcel is one that
sets L to at least 256 and includes at least one segment no larger
than L along with its CRC and checksum. In addition, all IP parcels
set L to at most 65535 and contain at most 64 segments along with
their corresponding CRCs/checksums.
7. Transmission of IP Parcels
When the network layer of the source assembles a {TCP,UDP}/IP parcel
it fully populates all IP header fields including the source address,
destination address and Parcel Payload option as above. The source
also sets IP {Total, Payload} Length to L (between 256 and 65535) to
distinguish the parcel from other jumbogram types (see: Section 8).
The network layer of the source also maintains a randomly-initialized
4/8/12/16-octet (32/64/96/128-bit) (extended) Identification value
for each destination expressed in an Identification Extension Option
for the Internet Protocol and includes an Identification in each
parcel (see: [I-D.templin-intarea-ipid-ext]). For each packet or
parcel transmission, the source sets the (extended) Identification to
the current cached value for this destination and increments the
cached value by 1 (modulo 2**32/64/96/128) for each successive
transmission. (The source can then reset the cached value to a new
random number when necessary, e.g., to maintain an unpredictable
profile.)
Templin Expires 20 May 2024 [Page 17]
Internet-Draft IP Parcels November 2023
The network layer of the source finally presents the parcel to an
interface for transmission to the next hop. For ordinary interface
attachments to parcel-capable links, the source simply admits each
parcel into the interface the same as for any IP packet where it may
be forwarded by one or more routers over additional consecutive
parcel-capable links possibly even traversing the entire forward path
to the final destination. Note that any node in the path that does
not recognize the parcel construct may either drop it and return an
ICMP Parameter Problem message or (erroneously) attempt to forward it
as an ordinary packet.
Most importantly, each parcel-capable link in the path forwards the
parcel even if link errors were detected since IP parcels and
advanced jumbos include end-to-end CRC and Checksum integrity checks.
This ensures that the majority of good data is delivered to the final
destination instead of being discarded along with a small amount of
errored data at an intermediate hop.
When the next hop link does not support parcels at all, or when the
next hop link is parcel-capable but configures an MTU that is too
small to pass the entire parcel, the source breaks the parcel up into
individual IP packets (in the first case) or into smaller sub-parcels
(in the second case). In the first case, the source can apply
packetization using Generic Segment Offload (GSO), and the final
destination can apply restoration using Generic Receive Offload (GRO)
to deliver the largest possible parcel buffer(s) to the transport
layer. In the second case, the source can apply parcellation to
break the parcel into sub-parcels with each containing the same
(extended) Identification value and with the S flag set
appropriately. The final destination can then apply reunification to
deliver the largest possible parcel buffer(s) to the transport layer.
In all other ways, the source processes of breaking a parcel up into
individual IP packets or smaller sub-parcels entail the same
considerations as for a router on the path that invokes these
processes as discussed in the following subsections.
Parcel probes that test the forward path's ability to pass parcels
set a Path MTU (PMTU field) to a non-zero value as discussed in
Section 7.6. Each router in the path then rewrites PMTU in a similar
fashion as for [RFC1063][RFC9268]. Specifically, each router
compares the parcel PMTU value with the next hop link MTU in the
parcel path and MUST (re)set PMTU to the minimum value. The fact
that the parcel transited a previous hop link provides sufficient
evidence of forward progress (since parcel path MTU determination is
unidirectional in the forward path only), but nodes can also include
the previous hop link MTU in their minimum PMTU calculations in case
the link may have an ingress size restriction (such as a receive
buffer limitation). Each parcel also includes one or more transport
Templin Expires 20 May 2024 [Page 18]
Internet-Draft IP Parcels November 2023
layer segments corresponding to the 5-tuple for the flow, which may
include {TCP,UDP} segment size probes used for packetization layer
path MTU discovery [RFC4821][RFC8899]. (See: Section 7.6 for further
details on parcel path probing.)
When a router receives an IP parcel it first compares Code with 255
and Check with the IP header TTL/Hop Limit; if either value differs,
the router drops the parcel and returns a negative Jumbo Report (see:
Section 7.5) subject to rate limiting. (Note that the IP parcel may
also have been truncated in length by a previous-hop router that does
not recognize the construct.) For all other intact IP parcels, the
router next compares the value L with the next hop link MTU. If the
next hop link is parcel capable but configures an MTU too small to
admit a parcel with a single segment of length L the router returns a
positive Jumbo Report (subject to rate limiting) with MTU set to the
next hop link MTU. If the next hop link is not parcel capable and
configures an MTU too small to pass an individual IP packet with a
single segment of length L the router instead returns a positive
Parcel Report (subject to rate limiting) with MTU set to the next hop
link MTU. If the next hop link is parcel capable the router MUST
reset Check to the same value that would appear in the IP header TTL/
Hop Limit field upon transmission to the next hop.
If the router recognizes parcels but the next hop link in the path
does not, or if the entire parcel would exceed the next hop link MTU,
the router instead opens the parcel. The router then forwards each
enclosed segment in individual IP packets or in a set of smaller sub-
parcels that each contain a subset of the original parcel's segments.
If the next hop link is via an OMNI interface, the router instead
follows OMNI Adaptation Layer procedures. These considerations are
discussed in detail in the following sections.
7.1. Packetization over Non-Parcel Links
For transmission of individual IP packets over links that do not
support parcels, or for transmission of (sub-)parcels larger than the
next-hop link MTU, the source or router (i.e., the node) engages GSO
to perform packetization. The node first determines whether an
individual packet with segment of length L can fit within the next
hop link/path MTU. If an individual packet would be too large (and
if source fragmentation is not an option), the node drops the parcel
and returns a positive Parcel Report message (subject to rate
limiting) with MTU set to the next hop link/path MTU and with the
leading portion of the parcel beginning with the IP header as the
"packet in error". If an individual packet can be accommodated, the
node removes the Parcel Payload option and caches the per-segment
Checksum header values (and for TCP also caches the Sequence
Numbers). The node then removes the Parcel Payload option, verifies
Templin Expires 20 May 2024 [Page 19]
Internet-Draft IP Parcels November 2023
the CRCs of each segment(i) (for i = 0 thru j) and discards any
segment(i)'s with incorrect CRCs. The node then copies the
{TCP,UDP}/IP headers followed by segment (i) (i.e., while discarding
the per-segment Checksum, Sequence Number and CRC fields) into as
many as 'j' individual IP packets ("packet(i)"). Each such packet(i)
will be subject to the independent CRC verifications of each
remaining link in the path.
For each packet(i), the node then clears the TCP control bits in all
but packet(0), and includes only those TCP options that are permitted
to appear in data segments in all but packet(0) which may also
include control segment options (see: Appendix A for further
discussion). The node then sets IP {Total, Payload} Length for each
packet(i) based on the length of segment(i) according to the IP
protocol standards [RFC0791] [RFC8200].
For each IPv6 packet(i), the node includes an "augmented" IPv6
(Extended) Fragment Header that replaces the "Reserved" octet with a
"Parcel Index" octet as shown in Figure 4. The node then sets the
(extended) Identification field to the value found in the parcel
header and writes the value 'i' in the Index field. The node finally
sets the "(P)arcel" bit to 1, and sets the "More (S)egments" bit to 1
for each non-final segment or 0 for the final segment.
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Next Header | Index |P|S| Fragment Offset |Res|M|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ Identification ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4: IPv6 (Extended) Fragment Header with Parcel Index
For each IPv4 packet(i), the node instead includes an Identification
Extension Option with Parcel Index extension octet as specified in
[I-D.templin-intarea-ipid-ext]. The node then sets the Parcel Index
octet values the same as for IPv6 above, sets the (extended)
Identification field to the value found in the parcel header and sets
the (D)ont Fragment flag to 1.
For each TCP/IP packet, the node sets the IP length then calculates/
sets the checksum for the packet according to [RFC9293]. For each
UDP/IP packet, the node instead sets the IP length and UDP length
fields then calculates/sets the checksum according to [RFC0768]. The
node reuses the cached checksum value for each segment in the
checksum calculation process. The node first calculates the Internet
checksum over the new packet {TCP,UDP}/IP headers then adds the
cached segment checksum value. For TCP, the node finally writes the
cached Sequence Number value for each segment into the TCP Sequence
Templin Expires 20 May 2024 [Page 20]
Internet-Draft IP Parcels November 2023
Number field which initially encoded the value 0 (note that this
permits the node to use the cached segment checksum without having to
recalculate). For UDP, if a per-segment Checksum was 0 the node
instead writes the value 0 in the Checksum field of the corresponding
UDP/IP packet.
For each IP packet, the node then sets both the Fragment Offset field
and (M)ore fragments flag to 0, and also sets the IP protocol-
specific flag to permit network fragmentation. The node then
performs source fragmentation if necessary while using both the
(extended) Identification and Parcel Index fields to identify the
fragments of the same packet. (This means that destinations must
consult both the Identification and Parcel Index in order to prevent
reassembly misassociations.) The node finally forwards each packet
or all of its constituent fragments to the next hop.
Note: Packets resulting from packetization may be too large to
transit the remaining path to the final destination, such that a
router may drop the packet(s) and possibly also return an ordinary
ICMP PTB message. Since these messages cannot be authenticated or
may be lost on the return path, the original source should take care
in setting a segment size larger than the known path MTU unless as
part of an active probing service.
7.2. Parcellation over Parcel-capable Links
For transmission of smaller sub-parcels over parcel-capable links,
the source or router (i.e., the node) first determines whether a
single segment of length L can fit within the next hop link MTU if
packaged as a (singleton) sub-parcel and possibly subject to IP
fragmentation. If a singleton sub-parcel would be too large, the
node returns a positive Jumbo Report message (subject to rate
limiting) with MTU set to the next hop link MTU and containing the
leading portion of the parcel beginning with the IP header, then
performs packetization as discussed in Section 7.1. Otherwise, the
node employs network layer parcellation to break the original parcel
into smaller groups of segments that can traverse the path as a whole
packet. The node first determines the number of segments of length L
that can fit into each sub-parcel under the size constraints. For
example, if the node determines that each sub-parcel can contain 3
segments of length L, it creates sub-parcels with the first
containing Segments 0-2, the second containing 3-5, the third
containing 6-8, etc., and with the final containing any remaining
Segments (where each segment includes its Checksum header and CRC
trailer from the original (sub-)parcel).
Templin Expires 20 May 2024 [Page 21]
Internet-Draft IP Parcels November 2023
If the original parcel's Parcel Payload option has S set to 0, the
node then sets S to 1 in all resulting sub-parcels except the last
(i.e., the one containing the final segment of length K, which may be
shorter than L) for which it sets S to 0. If the original parcel has
S set to 1, the node instead sets S to 1 in all resulting sub-parcels
including the last. The node next sets the Index field to the value
'i' which is the ordinal number of the first segment included in each
sub-parcel. (In the above example, the first sub-parcel sets Index
to 0, the second sets Index to 3, the third sets Index to 6, etc.).
If another router further down the path toward the final destination
forwards the sub-parcel(s) over a link that configures a smaller MTU,
the router may break it into even smaller sub-parcels each with Index
set to the ordinal number of the first segment included.
The node next appends identical {TCP,UDP}/IP headers (including the
Parcel Payload option, (extended) Identification and any other
extensions) to each sub-parcel while resetting Index, S, {Total,
Payload} Length (L) and Parcel Payload Length (M) in each as above.
For TCP, the node then clears the TCP control bits in all but the
first sub-parcel and includes only those TCP options that are
permitted to appear in data segments in all but the first sub-parcel
(which may also include control segment options). For both TCP and
UDP, the node then resets the {TCP,UDP} Checksum according to
ordinary parcel formation procedures (see above). The node finally
sets PMTU to the next hop link MTU then forwards each (sub-)parcel to
the parcel-capable next hop.
7.3. OMNI Interface Parcellation and Reunification
For transmission of original parcels or sub-parcels over OMNI
interfaces, the node admits all parcels into the interface
unconditionally since the OMNI interface MTU is unrestricted. The
OMNI Adaptation Layer (OAL) of this First Hop Segment (FHS) OAL
source node then forwards the parcel to the next OAL hop which may be
either an intermediate node or a Last Hop Segment (LHS) OAL
destination. OMNI interface parcellation and reunification
procedures are specified in detail in the remainder of this section,
while parcel encapsulation and fragmentation procedures are specified
in [I-D.templin-intarea-omni].
When the OAL source forwards a parcel (whether generated by a local
application or forwarded over a network path that transited one or
more parcel-capable links), it first assigns a monotonically-
incrementing (modulo 64) adaptation layer Parcel ID (note that this
value differs from the (Parcel) Index encoded in the Parcel Payload
option). If the parcel is larger than the OAL maximum segment size
of 65535 octets, the OAL source next employs parcellation to break
the parcel into sub-parcels the same as for the above network layer
Templin Expires 20 May 2024 [Page 22]
Internet-Draft IP Parcels November 2023
procedures. This includes re-setting the Index, P, S, {Total,
Payload} Length (L) and Parcel Payload Length (M) fields in each sub-
parcel the same as specified in Section 7.2.
The OAL source next assigns a different monotonically-incrementing
adaptation layer (extended) Identification value for each sub-parcel
of the same Parcel ID then performs adaptation layer encapsulation
while writing the Parcel ID into the OAL IPv6 Fragment Header. The
OAL source then performs OAL fragmentation if necessary and finally
forwards each fragment to the next OAL hop toward the OAL
destination. (During encapsulation, the OAL source examines the
Parcel Payload option S flag to determine the setting for the
adaptation layer fragment header S flag according to the same rules
specified in Section 7.2.)
When the sub-parcels arrive at the OAL destination, it retains them
along with their Parcel IDs and (extended) Identifications for a
short time to support reunification with peer sub-parcels of the same
original (sub-)parcel identified by the 4-tuple information
corresponding to the OAL source. This reunification entails the
concatenation of Checksums/Segments included in sub-parcels with the
same Parcel ID and with (extended) Identification values within
modulo (64) of one another to create a larger sub-parcel possibly
even as large as the entire original parcel. The OAL destination
concatenates the segments (plus their checksums and CRCs) for each
sub-parcel in ascending (extended) Identification value order, while
ensuring that any sub-parcel with TCP control bits set appears as the
first concatenated element in a reunified larger parcel and any sub-
parcel with S flag set to 0 appears as the final concatenation. The
OAL destination then sets S to 0 in the reunified (sub-)parcel if and
only if one of its constituent elements also had S set to 0;
otherwise, it sets S to 1.
The OAL destination then appends a common {TCP,UDP}/IP header plus
extensions to each reunified sub-parcel while resetting Index, S,
{Total, Payload} Length (L) and Parcel Payload Length (M) in the
corresponding header fields of each. For TCP, if any sub-parcel has
TCP control bits set the OAL destination regards it as sub-parcel(0)
and uses its TCP header as the header of the reunified (sub-)parcel
with the TCP options including the union of the TCP options of all
reunified sub-parcels. The OAL destination then resets the
{TCP,UDP}/IP header checksum. If the OAL destination is also the
final destination, it then delivers the sub-parcels to the network
layer which processes them according to the 5-tuple information
supplied by the original source. If the OAL destination is not the
final destination, it instead forwards each sub-parcel toward the
final destination the same as for an ordinary IP packet.
Templin Expires 20 May 2024 [Page 23]
Internet-Draft IP Parcels November 2023
Note: Adaptation layer parcellation over OMNI links occurs only at
the OAL source while adaptation layer reunification occurs only at
the OAL destination (intermediate OAL nodes do not engage in the
parcellation/reunification processes). The OAL destination should
retain sub-parcels in the reunification buffer only for a short time
(e.g., 1 second) or until all sub-parcels of the original parcel have
arrived. The OAL destination then delivers full and/or incomplete
reunifications to the network layer (in cases where loss and/or
delayed arrival interfere with full reunification).
Note: OMNI interface parcellation and reunification is an OAL process
based on the adaptation layer 4-tuple and not the network layer
5-tuple. This is true even if the OAL has visibility into network
layer information since some sub-parcels of the same original parcel
may be forwarded over different network paths.
Note: Some implementations may encounter difficulty in applying
adaptation layer reunification for sub-parcels that have already
incurred lower layer fragmentation and reassembly (e.g., due to
network kernel buffer structure limitations). In that case, the
adaptation layer can either linearize each sub-parcel before applying
reunification or deliver incomplete reunifications or even individual
sub-parcels to upper layers.
7.4. Final Destination Restoration/Reunification
When the original source or a router on the path opens a parcel and
forwards its contents as individual IP packets, these packets will
arrive at the final destination which can reassemble each packet if
necessary then hold them in a restoration buffer for a short time
before restoring the original parcel using GRO. The 5-tuple
information plus the (extended) Identification and (Parcel) Index
values provide sufficient context for GRO restoration which practical
implementations have proven as a robust service at high data rates.
When the original source or a router on the path opens a parcel and
forwards its contents as smaller sub-parcels, these sub-parcels will
arrive at the final destination which can hold them in a
reunification buffer for a short time or until all sub-parcels have
arrived. The 5-tuple information plus the Parcel ID, S flag and
(extended) Identification values provide sufficient context for
reunification.
In both the restoration and reunification cases, the final
destination concatenates segments according to ascending Index
numbers to preserve segment ordering even if a small degree of
reordering and/or loss may have occurred in the networked path. When
the final destination performs restoration/reunification on TCP
Templin Expires 20 May 2024 [Page 24]
Internet-Draft IP Parcels November 2023
segments, it must include the one with any TCP flag bits set as the
first concatenation and with the TCP options including the union of
the TCP options of all concatenated packets or sub-parcels. For both
TCP and UDP, any packet or sub-parcel containing the final segment
must appear as a final concatenation.
The final destination can then present the concatenated parcel
contents to the transport layer with segments arranged in (nearly)
the same order in which they were originally transmitted. Strict
ordering is not mandatory since each segment will include a transport
layer protocol specific start delimiter with positional coordinates.
However, the Index field includes an ordinal value that preserves
ordering since each sub-parcel or individual IP packet contains an
integral number of whole transport layer protocol segments.
Note: Restoration and/or reunification buffer management is based on
a hold timer during which singleton packets or sub-parcels are
retained until all members of the same original parcel have arrived.
Implementations should maintain a short hold timer (e.g., 1 second)
and advance any restorations/reunifications to upper layers when the
hold timer expires even if incomplete.
Note: Since loss and/or reordering may occur in the network, the
final destination may receive a packet or sub-parcel with S set to 0
before all other elements of the same original parcel have arrived.
This condition does not represent an error, but in some cases may
cause the network layer to deliver sub-parcels that are smaller than
the original parcel to the transport layer. The transport layer
simply accepts any segments received from all such deliveries and
will request retransmission of any segments that were lost and/or
damaged.
Note: Restoration and/or reunification buffer congestion may indicate
that the network layer cannot sustain the service(s) at current
arrival rates. The network layer should then begin to deliver
incomplete restorations/reunifications or even individual segments to
the receive queue (e.g., a socket buffer) instead of waiting for all
segments to arrive. The network layer can manage restoration/
reunification buffers, e.g., by maintaining buffer occupancy high/low
watermarks.
Note: Some implementations may encounter difficulty in applying
network layer restoration/reunification for packets/sub-parcels that
have already incurred adaptation layer reassembly/reunification. In
that case, the network layer can either linearize each packet/sub-
parcel before applying restoration/reunification or deliver
incomplete restorations/reunifications or even individual packets/
sub-parcels to upper layers.
Templin Expires 20 May 2024 [Page 25]
Internet-Draft IP Parcels November 2023
7.5. Parcel/Jumbo Reports
When a router or final destination returns a Parcel/Jumbo Report, it
prepares an ICMPv6 PTB message [RFC4443] with Code set to either
Parcel Report or Jumbo Report (see: [I-D.templin-intarea-ipid-ext])
and with MTU set to either the minimum MTU value for a positive
report or to 0 for a negative report. The node then writes its own
IP address as the Parcel/Jumbo Report source and writes the source
address of the packet that invoked the report as the Parcel/Jumbo
Report destination (for IPv4 Parcel Probes, the node writes the
Parcel/Jumbo Report address as an IPv4-Compatible IPv6 address
[I-D.templin-intarea-omni]). The node next copies as much of the
leading portion of the invoking packet as possible (beginning with
the IP header) into the "packet in error" field without causing the
entire Parcel/Jumbo Report (beginning with the IPv6 header) to exceed
512 octets in length. The node then sets the Checksum field to 0
instead of calculating and setting a true checksum since the UDP
checksum (see below) already provides an integrity check.
Since IPv6 packets cannot transit IPv4 paths, and since middleboxes
often filter ICMPv6 messages as they transit IPv6 paths, the node
next wraps the Parcel/Jumbo Report in UDP/IP headers of the correct
IP version with the IP source and destination addresses copied from
the Parcel/Jumbo Report and with UDP port numbers set to the OMNI UDP
port number [I-D.templin-intarea-omni]. The node then calculates and
sets the UDP Checksum (and for IPv4 clears the DF bit). The node
finally sends the prepared Parcel/Jumbo Report to the original source
of the probe.
Note: This implies that original sources that send IP parcels or
advanced jumbos must be capable of accepting and processing these
OMNI protocol UDP messages. A source that sends IP parcels or
advanced jumbos must therefore implement enough of the OMNI interface
to be able to recognize and process these messages.
7.6. Parcel/Jumbo Path Probing
All parcels also serve as implicit probes and may cause either a
router in the path or the final destination to return an ordinary
ICMP error [RFC0792][RFC4443] and/or Packet Too Big (PTB) message
[RFC1191] [RFC8201] concerning the parcel. A router in the path or
the final destination may also return a Parcel/Jumbo Report (subject
to rate limiting per [RFC4443]) as discussed in Section 7.5.
To determine whether parcels can transit at least an initial portion
of the forward path toward the final destination, the original source
can also send IP parcels with a Parcel Payload option PMTU field
included and set to the next hop link MTU as an explicit Parcel
Templin Expires 20 May 2024 [Page 26]
Internet-Draft IP Parcels November 2023
Probe. The Parcel Probe option format is shown in Figure 5, where
option-length is set to 12 for IPv4 and Opt Data Len is set to 10 for
IPv6:
IPv4 Parcel Probe Option Format
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| option-type | option-length | Code | Check |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Index |P|S| Parcel Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Path MTU (PMTU) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
IPv6 Parcel Probe Option Format
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Opt Data Len | Code | Check |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Index |P|S| Parcel Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Path MTU (PMTU) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: Parcel Probe Option
The parcel probe will cause the final destination or a router on the
path to return a Parcel/Jumbo Report or cause the final destination
to return an ordinary data packet with an IP Jumbo Reply MTU option
(see: Section 7.5).
A Parcel Probe can be included either in an ordinary data parcel or a
{TCP,UDP}/IP parcel with destination port set to 9 (discard)
[RFC0863]. The probe will still contain a valid {TCP,UDP} parcel
header Checksum that any intermediate hops as well as the final
destination can use to detect mis-delivery, while the final
destination will process any parcel data in probes with correct
Checksums/CRCs.
If the original source receives a positive Parcel/Jumbo Report or an
ordinary data packet/parcel with an IP Jumbo Reply MTU option, it
marks the path as "parcels supported" and ignores any ordinary ICMP
and/or PTB messages concerning the probe. If the original source
instead receives a negative Jumbo Report or no report/reply, it marks
the path as "parcels not supported" and may regard any ordinary ICMP
and/or PTB messages concerning the probe (or its contents) as
indications of a possible path limitation.
Templin Expires 20 May 2024 [Page 27]
Internet-Draft IP Parcels November 2023
The original source can therefore send Parcel Probes in the same IP
parcels used to carry real data. The probes will transit parcel-
capable links joined by routers on the forward path possibly
extending all the way to the destination. If the original source
receives a positive Parcel/Jumbo Report or an ordinary data packet/
parcel with an IP Jumbo Reply MTU option, it can continue using IP
parcels after adjusting its segment size if necessary.
The original source sends Parcel Probes unidirectionally in the
forward path toward the final destination to elicit a report/reply,
since it will often be the case that IP parcels are supported only in
the forward path and not in the return path. Parcel Probes may be
dropped in the forward path by any node that does not recognize IP
parcels, but Parcel/Jumbo Reports and/or IP Jumbo Reply MTU options
must be packaged to reduce the risk of return path filtering. For
this reason, the Parcel Payload options included in Parcel Probes and
IP Jumbo Reply MTU options are always packaged as IPv4 header or IPv6
Hop-by-Hop options while Parcel/Jumbo Reports are returned as UDP/IP
encapsulated ICMPv6 PTB messages with a Parcel/Jumbo Report Code
value (see: [I-D.templin-intarea-omni]).
Original sources send ordinary parcels or discard parcels as explicit
Parcel Probes by setting the Parcel Payload PMTU to the (non-zero)
next hop link MTU. The source then sets Index, Parcel Payload
Length, and {Total, Payload} Length, then calculates the header
Checksum and per-segment Checksums/CRCs the same as for an ordinary
parcel. The source finally sends the Parcel Probe via the outbound
IP interface.
Original sources can send Parcel Probes that include a large segment
size, but these may be dropped by a router on the path even if the
next hop link is parcel-capable. The original source may then
receive a Jumbo Report that contains only the MTU of the leading
portion of the path up to the router with the restrictive link. The
original source can instead send Parcel Probes with smaller segments
that would be likely to transit the entire forward path to the final
destination if all links are parcel-capable. For parcel-capable
paths, this may allow the original source to discover both the path
MTU and the MSS in a single message exchange instead of multiple.
According to [RFC7126], IPv4 middleboxes (i.e., routers, security
gateways, firewalls, etc.) that do not observe this specification
should drop IPv4 packets that contain option type '0x0B' (IPv4 Probe
MTU) but some might instead either attempt to implement [RFC1063] or
ignore the option altogether. IPv4 middleboxes that observe this
specification instead MUST process the option as an implicit or
explicit Parcel Probe as specified below.
Templin Expires 20 May 2024 [Page 28]
Internet-Draft IP Parcels November 2023
According to [RFC9268], IPv6 middleboxes (i.e., routers, security
gateways, firewalls, etc.) that do not observe this specification
will either ignore the option altogether or notice that the option
length differs from its base definition and presumably ignore the
option or drop the packet. IPv6 middleboxes that observe this
specification instead MUST process the option as an implicit or
explicit Parcel Probe.
When a router that observes this specification receives an IP Parcel
Probe it first compares Code with 255 and Check with the IP header
TTL/Hop Limit; if either value differs, the router drops the probe
and returns a negative Jumbo Report subject to rate limiting. (Note
that the Parcel Probe may also have been truncated in length by a
previous-hop router that does not recognize the construct.) For all
other intact IP Parcel Probes, if the next hop link is non-parcel-
capable the router compares PMTU with the next hop link MTU and
returns a positive Parcel Report subject to rate limiting with MTU
set to the minimum value. The router then applies packetization to
convert the probe into individual IP packet(s) and forwards each
packet to the next hop; otherwise, it drops the probe.
If the next hop link both supports parcels and configures an MTU that
is large enough to pass the probe, the router instead compares the
probe PMTU with the next hop link MTU. The router next MUST (re)set
PMTU to the minimum value then forward the probe to the next hop (and
also reset Check to the same value that will appear in the IP header
TTL/Hop Limit upon transmission to the next hop). If the next hop
link supports parcels but configures an MTU that is too small to pass
the probe, the router then applies parcellation to break the probe
into multiple smaller sub-parcels that can transit the link. In the
process, the router sets PMTU to the minimum link MTU value in the
first sub-parcel and omits the PMTU field in all non-first sub-
parcels (and also resets Check in all sub-parcels). If the next hop
link supports parcels but configures an MTU that is too small to pass
a singleton sub-parcel of the probe, the router instead drops the
probe and returns a positive Jumbo Report subject to rate limiting
with MTU set to the next hop link MTU.
The final destination may therefore receive one or more individual IP
packets or sub-parcels including an intact Parcel Probe. If the
final destination receives individual IP packets, it performs any
necessary integrity checks, applies restoration if possible then
delivers the (restored) parcel contents to the transport layer. If
the final destination receives a Parcel Probe, it first compares Code
with 255 and Check with the IP header TTL/Hop Limit; if either value
differs, the final destination drops the probe and returns a negative
Jumbo Report. (Note that the Parcel Probe may also have been
truncated in length by a previous-hop router that does not recognize
Templin Expires 20 May 2024 [Page 29]
Internet-Draft IP Parcels November 2023
the construct.) For all other intact Parcel Probes, if the {TCP,UDP}
port number is 9 (discard) the final destination instead returns a
positive Jumbo Report and discards the probe and any of its
associated sub-parcels without applying reunification.
If the final destination receives a Parcel Probe (plus any of its
associated sub-parcels) for any other {TCP,UDP} port number, it
applies reunification and delivers the (reunified) parcel contents to
the transport layer. The destination then arranges to include an IP
Jumbo Reply MTU option in a return data packet/parcel associated with
the flow according to the format shown in Figure 6:
IPv4 Jumbo Reply MTU Option Format
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| option-type | option-length | 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Path MTU (PMTU) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ Identification ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
IPv6 Jumbo Reply MTU Option Format
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Opt Data Len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Path MTU (PMTU) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ Identification ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: IP Jumbo Reply MTU Option
For IPv4, the destination sets option-type to '0x0C' and option-
length to 16/20/24/28 according to the length of the (extended)
Identification field.
For IPv6, the destination sets Option Type to '0x30' and Opt Data Len
to 12/16/20 according to the length of the (extended) Identification
field.
For both IP protocol versions, the Code and Check fields are omitted
since hop-by-hop determination of protocol recognition are not
required. The destination instead sets the Path MTU and (extended)
Templin Expires 20 May 2024 [Page 30]
Internet-Draft IP Parcels November 2023
Identification fields to the values received in the Parcel Probe,
then sets other unused fields to 0. Note that the option lengths
differentiate the options from the shorter forms of the same Option
Types that appear in [RFC1063] and [RFC9268] as well as in other
option formats specified in this document.
After sending Parcel Probes (or ordinary parcels) the original source
may therefore receive UDP/IP encapsulated Parcel/Jumbo Reports,
ordinary data packets with IP Jumbo Reply MTU options, and/or
transport layer protocol probe replies. If the source receives a
Parcel/Jumbo Report, it verifies the UDP Checksum then verifies that
the ICMPv6 Checksum is 0. If both Checksum values are correct, the
node then matches the enclosed PTB message with an original probe/
parcel by examining the ICMPv6 "packet in error" containing the
leading portion of the invoking packet. If the "packet in error"
does not match one of its previous packets, the source discards the
Parcel/Jumbo Report; otherwise, it continues to process.
If the source receives a negative Parcel/Jumbo Report (i.e., one with
MTU set to 0), it marks the path as "parcels not supported".
Otherwise, the source marks the path as "parcels supported" and also
records the MTU value as the parcel path MTU (i.e., the portion of
the path up to and including the node that returned the Parcel/Jumbo
Report). If the MTU value is 65535 (plus headers) or larger, the MTU
determines the largest whole parcel that can transit the path without
packetization/parcellation while using any segment size up to and
including the maximum. For Reports that include a smaller MTU, the
value represents both the largest whole parcel size and a maximum
segment size limitation. In that case, the maximum parcel size that
can transit the initial portion of the path may be larger than the
maximum segment size that can continue to transit the remaining path
to the final destination.
If the source receives an ordinary data packet for the flow that
includes an IP Jumbo Reply MTU option, it examines the (extended)
Identification to ensure that the reply matches one of the Parcel
Probes it previously sent for this same data flow. It then records
the PMTU value as the parcel/jumbo path MTU for this flow and marks
the path as "parcels and jumbos supported".
Note: when a source sends a parcel probe into a new path that has not
been probed previously, it should include enough padding payload so
that the overall packet length is consistent with the value found in
the IP {Total, Payload} Length field. This allows legacy routers on
the path that do not recognize parcels to see a length that is
consistent with the value found in the IP header.
Templin Expires 20 May 2024 [Page 31]
Internet-Draft IP Parcels November 2023
Note: the path MTU discovered through a Parcel Probe exchange can
conceivably exceed the maximum-sized parcel, since link MTUs are
represented as 32-bit values whereas the maximum-sized parcel is
limited to 24 bits. For this reason, Parcel Probes can serve the
dual purpose of also determining the maximum jumbogram size that can
traverse the path.
Note: when an IP Jumbo Reply MTU option is included with an IPv6
parcel or advanced jumbo, the Parcel/Jumbo Payload length field will
contain a non-zero value. Implementations recognize this as a
combined parcel/jumbo plus IP Jumbo Reply MTU by examining both the
Option Data Length and the Parcel/Jumbo Payload Length.
For further discussion on parcel/jumbo probing alternatives, see:
Appendix C.
8. Advanced Jumbos
This specification introduces an IP advanced jumbo(gram) service as
an alternative to parcels and basic jumbograms that also includes a
path probing function based on the mechanisms specified in
Section 7.6. The function employs an Advanced Jumbo Option with the
same option type and length values as for the Parcel Payload/Probe
options, but with the Parcel Index and Parcel Payload Length fields
replaced by a 32-bit Jumbo Payload Length field as shown in Figure 7:
IPv4 Advanced Jumbo/Probe Option Format
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| option-type | option-length | Code | Check |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Jumbo Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ Path MTU (PMTU) (Probes Only) ~
+- - - - - - - -+- - - - - - - -+- - - - - - - -+- - - - - - - -+
IPv6 Advanced Jumbo/Probe Option Format
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Opt Data Len | Code | Check |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Jumbo Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ Path MTU (PMTU) (Probes Only) ~
+- - - - - - - -+- - - - - - - -+- - - - - - - -+- - - - - - - -+
Figure 7: Advanced Jumbo/Probe Option
Templin Expires 20 May 2024 [Page 32]
Internet-Draft IP Parcels November 2023
{TCP/UDP}/IP advanced jumbos/probes are formed the same as for
parcels as shown in Figure 2 except that they include only a single
segment ("Segment 0") preceded by a 2-octet Internet Checksum header
and followed by an N-octet message digest trailer. Unlike IP
parcels, TCP Advanced Jumbos do not include a Sequence Number for the
(single) segment since the sequence number is coded in the TCP header
the same as for an ordinary packet.
Advanced Jumbo implementations honor the message digest algorithms
specified for MD5 [RFC1321], SHA1 [RFC3174] and the advanced US
Secure Hash Algorithms [RFC6234] as selected by an Advanced Jumbo
Type value between 250 and 255. Advanced Jumbos can instead employ a
CRC32C/CRC64E integrity check by selecting a Type value of 247 or
248, where the CRC code appears instead of a message digest. (The
Advanced Jumbo Type value 249 is reserved as a non-functional
placeholder for a nominal CRC128J algorithm, which may be specified
in future documents - see: Appendix D.)
The source includes a message digest according to an algorithm
appropriate for the segment length while considering the error
characteristics of the path. The destination verifies the digest
according to the selected algorithm and uses local knowledge to
determine whether the integrity check strength is sufficient to relax
upper layer checking. Advanced Jumbo implementations MUST support
the following:
Type Algorithm Digest Length
---- --------- -------------
247 CRC32C 4 octets
248 CRC64E 8 octets
250 MD5 16 octets
251 SHA1 20 octets
252 SHA-224 28 octets
253 SHA-256 32 octets
254 SHA-384 48 octets
255 SHA-512 64 octets
Figure 8: Mandatory Advanced Jumbo Algorithms
The source prepares an advanced jumbo/probe by first setting the IP
{Total, Payload} Length field to an Advanced Jumbo Type value taken
from the above table to distinguish this from a basic jumbogram or
parcel. The source can begin by sending a Jumbo Probe to pre-qualify
the path for advanced jumbos if necessary.
To prepare a Jumbo Probe that will trigger a Jumbo Report, the source
can set {Protocol, Next Header} to {TCP,UDP}, set the {TCP,UDP} port
to 9 (discard) and either include no octets beyond the {TCP,UDP}
Templin Expires 20 May 2024 [Page 33]
Internet-Draft IP Parcels November 2023
header or a single discard segment of the desired probe size
immediately following the header. (The source can instead set the
{TCP,UDP} port to the port number for a current data flow in order to
receive IP Jumbo Reply MTU options in return packets as discussed in
Section 7.6.) The source then sets Jumbo Payload Length to the
length of the {TCP,UDP} header plus the length of the segment
Checksum header and message digest trailer plus the discard segment
plus the length of the full IP header for IPv4 or the extension
headers for IPv6.
The source next sets the (extended) Identification the same as for an
IP Parcel Probe, sets the Jumbo Probe PMTU to the next hop link MTU,
then sets Code to 255 and Check to the next hop TTL/Hop Limit. The
source then calculates the {TCP,UDP} Checksum based on the same
pseudo header as for an ordinary parcel (see: Figure 11) but with the
Parcel Index and Payload Length fields replaced with a 32-bit Jumbo
Payload Length field and with the Segment Length replaced with one of
the supported Advanced Jumbo Type values. The source then calculates
the checksum of the segment payload, writes the value into the
segment Checksum header, then calculates the message digest over the
length of the (single) segment beginning with the Checksum field and
writes the value into the trailer. The source then sends the Jumbo
Probe via the next hop link toward the final destination.
At each IP forwarding hop, the router examines Code and Check then
drops the Jumbo Probe and returns a negative Jumbo Report if either
value is incorrect. (Note that the Jumbo Probe may also have been
truncated in length by a previous-hop router that does not recognize
the construct.) For all other intact probes, if the next hop link is
jumbo-capable the router compares PMTU to the next hop link MTU,
resets PMTU to the minimum value, sets Check to the next hop TTL/Hop
Limit then forwards the probe to the next hop. If the next hop link
is not jumbo-capable, the router instead drops the probe and returns
a negative Jumbo Report.
If the Jumbo Probe encounters an OMNI link, the OAL source can either
drop the probe and return a negative Jumbo Report or forward the
probe further toward the OAL destination using adaptation layer
encapsulation. If the OAL source already knows the OAL path MTU for
this OAL destination, it can encapsulate and forward the Jumbo Probe
with PMTU set to the minimum of itself and the known value (minus the
adaptation layer header size), and without adding any padding octets.
Templin Expires 20 May 2024 [Page 34]
Internet-Draft IP Parcels November 2023
If the OAL path MTU is unknown, the OAL source can instead
encapsulate the Jumbo Probe in an adaptation layer IPv6 header with
an Advanced Jumbo option and with padding octets added beyond the end
of the encapsulated Jumbo Probe to form an adaptation layer jumbogram
as large as the minimum of PMTU and (2**24 - 1) octets (minus the
adaptation layer header size) as a form of "jumbo-in-jumbo"
encapsulation.
The OAL source then writes this size into the Jumbo Probe PMTU field
and forwards the newly-created adaptation layer jumbogram toward the
OAL destination. If the jumbogram somehow transits the path, the OAL
destination then removes the adaptation layer encapsulation, discards
the padding, then forwards the Jumbo Probe onward toward the final
destination (with each hop reducing PMTU if necessary).
When a router on the path forwards a Jumbo Probe, it drops and
returns a Jumbo Report if the next hop MTU is insufficient;
otherwise, it forwards to the next hop toward the final destination.
When the final destination receives the Jumbo Probe, it returns a
Jumbo Report with the PMTU set to the maximum-sized jumbo that can
transit the path.
When the Jumbo Probe reaches the final destination, the destination
first examines the {TCP,UDP} port number. If the port number is 9
(discard), the destination returns a Jumbo Report UDP message;
otherwise, the destination prepares an IP Jumbo Reply MTU option to
include in a data packet/parcel on the return path to the original
source. Detailed descriptions for these processes are found in
Section 7.6.
After successfully probing the path, the original source can begin
sending regular advanced jumbos by setting the IP {Total, Payload}
Length field to one of the supported Advanced Jumbo Type values,
omitting the PMTU field and calculating the (TCP,UDP}/IP header
checksum and per-segment Checksum header and message digest trailer
the same as described for probes above. When the network layer of
the final destination receives an advanced jumbo, it first verifies
the integrity checks then delivers the data (along with a CRC/
Checksum error flag) to the transport layer without returning a Jumbo
Report. The source can continue to send advanced jumbos into the
path with the possibility that the path may change. In that case, a
router in the network may return an ICMP error, an ICMPv6 PTB, or a
Jumbo Report if the path MTU decreases.
Templin Expires 20 May 2024 [Page 35]
Internet-Draft IP Parcels November 2023
Note: when a source sends a jumbo probe into a new path that has not
been probed previously, it should include enough padding payload so
that the overall packet length is consistent with the value found in
the IP {Total, Payload} Length field. This allows legacy routers on
the path that do not recognize jumbos to see a length that is
consistent with the value found in the IP header.
Note: If an OAL source can in some way determine that a very large
packet is likely to transit the OAL path, it can encapsulate a Jumbo
Probe to form an adaptation layer jumbogram even larger than (2**24 -
1) octets with the understanding that the time required to transit
the path plus the receive buffer size determine acceptable jumbogram
sizes.
Note: The Jumbo Report message types returned in response to both
Parcel and Jumbo Probes are one and the same, and signify that both
parcels and advanced jumbos at least as large as the reported MTU can
transit the path. However, only a Parcel Probe (i.e., and not a
Jumbo Probe) may elicit a Parcel Report. This may indicate a
preference to use Parcel Probes instead of Jumbo Probes for general-
purpose path probing.
Note: unlike basic jumbograms, advanced jumbos may encode values
smaller than 65536 in the Jumbo Payload Length. This means that
advanced jumbos can range in size from as small as the headers plus a
minimal or even null payload to as large as 2**32 octets minus
headers. This allows smaller advanced jumbos to operate within the
traditional realms of ordinary packets or singleton parcels.
9. Minimal IPv6 Parcels/Advanced Jumbos
The basic IPv6 parcel and advanced jumbo constructs specified in the
previous sections use the IPv6 Minimum Path MTU Hop-by-Hop option
[RFC9268] initially to allow each hop to participate in path
qualification. Once a path has been qualified to accept the basic
constructs, however, the source can begin sending minimal IPv6
parcels or advanced jumbos that instead use the IPv6 Jumbo Payload
Hop-by-Hop Option [RFC2675] to benefit from an 8-octet per packet
savings as shown in Figure 9:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Opt Data Len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Data (first four octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 9: IPv6 Minimal Parcel/Jumbo Option Format
Templin Expires 20 May 2024 [Page 36]
Internet-Draft IP Parcels November 2023
In this format, the network layer includes the IPv6 minimal Parcel/
Jumbo Option as an IPv6 Hop-by-Hop option with Option Type set to
'0xC2' and Opt Data Len set to 4. For parcels, the first four octets
of the Option Data are formatted exactly as shown in Figure 1 while
for advanced jumbos the first four octets are exactly as shown in
Figure 7. The network layer prepares all other aspects of IPv6
minimal parcels and advanced jumbos exactly the same as for the basic
specifications found in previous sections except the option type/
length are different and the Code/Check fields are omitted.
This implies that implementations that honor the basic IPv6 parcel
and advanced jumbo formats and processing specified in the previous
sections MUST also honor the IPv6 Minimal Parcel/Jumbo Option format
specified above as an equivalent construct. Therefore, the Parcel/
Jumbo probe results received for the basic formats also serve as
probe results for the minimal format.
Since the minimal format does not include Code and Check fields,
intermediate and end systems must monitor the lengths of minimal
parcels and advanced jumbos they receive in case the path changes and
an unqualified previous hop begins truncating them. In that case,
the node MUST drop the packet and return a negative Jumbo Report to
the source which must then re-initiate parcel/jumbo path probing.
10. OMNI IP Parcels/Advanced Jumbos
Network intermediate systems often drop IPv4 packets that contain IP
header options unconditionally. This presents an obstacle to
deploying new IPv4 options in the Internet, but may be less of a
concern within some limited domain networks. As a first alternative,
the source could encode IPv4 parcel and advanced jumbo options as
IPv6 extension headers; for example, the source could set the IPv4
header Protocol to 0 and include an IPv6 Hop-by-Hop option
immediately after the header. Since intermediate systems are also
known to drop packets with unusual or unrecognized IP protocols,
however, the source could instead employ a second alternative more
likely to provide service by concealing IPv6 options within the body
of a protocol data unit such as UDP.
End systems and intermediate systems that recognize the OMNI protocol
[I-D.templin-intarea-omni] can use the parcel, advanced jumbo and
minimal parcel/jumbo formats specified in this document as native
protocol extension headers coded within the body of the OMNI protocol
data unit. This is true for both IPv6 and IPv4, where IPv4 parcels
and advanced jumbos can use the same extension header formats defined
for IPv6.
Templin Expires 20 May 2024 [Page 37]
Internet-Draft IP Parcels November 2023
The section titled "OMNI L2 Extension Header Encapsulation" in
[I-D.templin-intarea-omni] depicts protocol layering for
encapsulation of IPv6 Extension Headers in IPv4 and IPv6 packets as
shown in Figure 10:
+---------------------------+
| L2 IP/Ethernet Header |
+---------------------------+
| L2 UDP Header (port 8060) |
+---------------------------+
~ L2 IPv6 Extension Headers ~
+---------------------------+
| OAL IPv6 Encapsulation |
+---------------------------+
~ OAL IPv6 Extensions ~
+---------------------------+
| |
~ ~
~ Original IP Packet ~
~ ~
| |
+---------------------------+
Figure 10: OMNI IP Parcels/Advanced Jumbos
In this encapsulation format, the IPv6 parcel, advanced jumbo and
minimal parcel/jumbo extension headers specified in previous sections
as well as the IPv6 (Extended) Fragment Header appear as IPv6
Extension Headers following the OMNI protocol UDP, IP or Ethernet
header. The OMNI protocol requires each node to honor and implement
the parcel and advanced jumbo constructs as specified in this
document with reference to [I-D.templin-intarea-omni]. This includes
the setting of the IP {Total, Payload} length fields as well as the
settings of the parcel/jumbo options themselves.
Intermediate systems that do not recognize the OMNI protocol are
likely to drop any OMNI packets that include parcel or advanced jumbo
options, but they may instead forward the packet without updating the
Code/Check values and/or while truncating the overall packet length.
Intermediate systems and end systems that recognize OMNI therefore
perform the checks specified in this document to determine whether
previous path hops correctly process parcels and advanced jumbos.
Since parcel and advanced jumbo options are coded within the OMNI
protocol data unit itself instead of as an IP header extension,
network intermediate systems must also reset the OMNI protocol
checksum if necessary when they alter the contents of an option (such
as when resetting Path MTU or Check). For this reason, sources MAY
Templin Expires 20 May 2024 [Page 38]
Internet-Draft IP Parcels November 2023
disable the OMNI protocol checksum in path probes and SHOULD advance
to using minimal parcels and advanced jumbos soon after probing the
path to minimize intermediate system checksum interactions.
See: [I-D.templin-intarea-omni] for the full specification of OMNI L2
Extension Header encapsulation and processing. All parcel and
advanced jumbo implementations that recognize the OMNI protocol are
required to implement those portions of the OMNI specification.
11. Integrity
IP parcel and advanced jumbo integrity assurance responsibility is
shared between lower layers of the protocol stack and the transport
layer where more discrete compensations for lost or corrupted data
recovery can be applied. In particular, intermediate system lower
layers forward parcels or advanced jumbos with correct headers to the
final destination transport layer even if there may have been
cumulative link errors incurred at intermediate hops. The
destination is then responsible for its own integrity assurance.
The {TCP,UDP}/IP header plus each segment of a (multi-segment) IP
parcel or advanced jumbo includes its own integrity checks. This
means that IP parcels and advanced jumbos offer stronger and more
discrete integrity checks for the same amount of transport layer
protocol data compared to an individual IP packet or jumbogram. The
{TCP,UDP} Checksum header integrity check SHOULD be verified at each
hop for which a link error is encountered to ensure that IP parcels
and advanced jumbos with errored addressing information are detected.
The per-segment Checksums and CRCs are set by the source and verified
by the final destination. Note that each segment includes both
checks since there will be many instances when errors missed by the
CRC are detected by the Checksum [STONE].
IP parcels can range in length from as small as only the {TCP,UDP}/IP
headers plus a single segment to as large as the headers plus (64 *
65535) octets, while advanced jumbos include only a single segment
that can be as large as 2**32 octets (minus headers). Due to
parcellation/packetization in the path, the segment contents of a
received parcel may arrive in an incomplete and/or rearranged order
with respect to their original packaging.
Templin Expires 20 May 2024 [Page 39]
Internet-Draft IP Parcels November 2023
IP parcels and advanced jumbos include a separate 2-octet Internet
Checksum header for each segment. The original source calculates the
checksum for each segment beginning with the first octet of the per-
segment Sequence Number for TCP or beginning with the first octet of
the segment for UDP (noting that per-segment Checksum values of 0
indicate that the segment checksum is disabled). The source extends
the checksum calculation over the entire length of the segment (plus
sequence number for TCP) but does not extend the calculation into the
trailing CRC field.
IP parcels employ two different CRC types according to the non-final
segment length "L". For values of L smaller than 8192 octets (8KB),
the CRC32C specification is used [RFC3385] and the CRC is encoded in
a 4 octet trailer. For larger L values, the CRC64E specification is
used [ECMA-182] and the CRC is encoded in an 8 octet trailer.
Advanced jumbos instead include an N-octet message digest trailer
calculated per [RFC1321], [RFC3174] or [RFC6234] where N is
determined according to the hash algorithm assigned to the Advanced
Jumbo type (see: IANA Considerations).
When link errors are detected, each network layer forwarding hop as
well as the final destination SHOULD verify the IP parcel or advanced
jumbo {TCP,UDP}/IP Checksum at its layer, since an errored header
could result in mis-delivery. If a network layer protocol entity on
the path detects an incorrect {TCP,UDP}/IP Checksum it should discard
the entire IP parcel or advanced jumbo unless the header(s) can
somehow first be repaired by lower layers.
To support the IP parcel and advanced jumbo {TCP,UDP}/IP header
checksum calculation, the network layer uses modified versions of the
{TCP,UDP}/IPv4 pseudo-header found in [RFC9293] [RFC0768], or the
{TCP,UDP}/IPv6 pseudo-header found in Section 8.1 of [RFC8200]. Note
that while the contents of the two IP protocol version-specific
pseudo-headers beyond the address fields are the same, the order in
which the contents are arranged differs and must be honored according
to the specific IP protocol version as shown in Figure 11. This
allows for maximum reuse of widely deployed code while ensuring
interoperability.
Templin Expires 20 May 2024 [Page 40]
Internet-Draft IP Parcels November 2023
IPv4 Parcel Pseudo-Header
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPv4 Source Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPv4 Destination Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| zero | Next Header | Segment Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Index |P|S| Parcel Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
IPv6 Parcel Pseudo-Header
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ IPv6 Source Address ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ IPv6 Destination Address ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Index |P|S| Parcel Payload Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Segment Length | zero | Next Header |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 11: {TCP,UDP}/IP Parcel Pseudo-Header Formats
where the following fields appear in both pseudo-headers:
* Source Address is the 4-octet IPv4 or 16-octet IPv6 source address
of the prepared parcel.
* Destination Address is the 4-octet IPv4 or 16-octet IPv6
destination address of the prepared parcel.
* zero encodes the constant value 0.
* Next Header is the IP protocol number corresponding to the
transport layer protocol, i.e., TCP or UDP.
* Segment Length is the value that appears in the IP {Total,
Payload} Length field of the prepared parcel.
* For IP parcels, [Index, P, S] is the combined 1-octet field and
Parcel Payload Length is the 3-octet field that appear in the
Parcel Payload Option fields of the same name. (For advanced
jumbos, these two fields are replaced by a single 4-octet Jumbo
Payload Option field.)
Templin Expires 20 May 2024 [Page 41]
Internet-Draft IP Parcels November 2023
When the transport layer protocol entity of the source delivers a
parcel body to the network layer, it presents the values L and J
along with the (J + 1) segments in canonical order as a list of data
buffers (and with each TCP segment preceded by a 4-octet Sequence
Number field). When the network layer of the source accepts the
parcel body from the transport layer protocol entity, it calculates
the Internet checksum for each segment and writes the value in the
per-segment Checksum header (or writes the value 0 when UDP checksums
are disabled). The network layer then calculates the CRC for each
segment beginning with the Checksum field, inserts the CRC result as
a segment trailer in network byte order, then concatenates all
segments and appends the necessary {TCP,UDP}/IP headers and
extensions to form a parcel. The network layer then calculates the
{TCP,UDP}/IP header checksum over the length of only the {TCP,UDP}
headers plus IP pseudo header then forwards the parcel to the next
hop without further processing.
When the network layer of the destination reunifies a parcel from one
or more sub-parcels received from the source it first verifies the
{TCP,UDP}/IP header checksum then verifies first the CRC and next the
Checksum for each segment and marks any with incorrect integrity
check values as errors. When the network layer restores a parcel
from one or more individual {TCP,UDP}/IP packets received from the
source, it instead marks the CRCs of each segment as correct since
the individual packets were subject to CRC checks at each hop along
the path. The network layer then verifies the Internet checksum of
each individual packet (except when UDP checksums are disabled),
restores the parcel, and delivers each parcel segment along with a
CRC/Checksum error flag to the transport layer.
When the transport layer of the destination processes parcel or
advanced jumbo segments, it can accept any with correct CRCs and
Checksums while optionally applying additional higher-layer integrity
checks. The transport layer can instead process any segments with
CRC/Checksum errors by either discarding the entire segment or
applying higher-layer integrity checks on the component elements of
the segment to accept as many non-errored elements as possible. The
transport layer can then either reconstruct from local information or
request retransmission for any segment elements that may have been
damaged in transit as necessary.
Note: when the destination network layer detects a per-segment CRC
error, it immediately posts the segment plus an error code for
delivery to the transport instead of continuing to verify the segment
Checksum. Performing a second integrity check on a segment already
determined to contain errors by a first check would serve no useful
purpose.
Templin Expires 20 May 2024 [Page 42]
Internet-Draft IP Parcels November 2023
Note: the source and destination network layers can often engage
hardware functions to greatly improve CRC/Checksum calculation
performance.
12. Implementation Status
Common widely-deployed implementations include services such as TCP
Segmentation Offload (TSO) and Generic Segmentation/Receive Offload
(GSO/GRO). These services support a robust service that has been
shown to improve performance in many instances.
An early prototype of UDP/IPv4 parcels (draft version -15) has been
implemented relative to the linux-5.10.67 kernel and ION-DTN ion-
open-source-4.1.0 source distributions. Patch distribution found at:
"https://github.com/fltemplin/ip-parcels.git".
Performance analysis with a single-threaded receiver has shown that
including increasing numbers of segments in a single parcel produces
measurable performance gains over fewer numbers of segments due to
more efficient packaging and reduced system calls/interrupts. For
example, sending parcels with 30 2000-octet segments shows a 48%
performance increase in comparison with ordinary IP packets with a
single 2000-octet segment.
Since performance is strongly bounded by single-segment receiver
processing time (with larger segments producing dramatic performance
increases), it is expected that parcels with increasing numbers of
segments will provide a performance multiplier on multi-threaded
receivers in parallel processing environments.
13. IANA Considerations
The IANA is instructed to add a reference to this document
([RFCXXXX]) in the "MTUP - MTU Probe" and "MTUR - MTU Reply" entries
in the "IP Option Numbers" section of the 'ip-parameters' registry.
The IANA is instructed to add a reference to this document
([RFCXXXX]) in the "Minimum Path MTU Hop-by-Hop Option" entry in the
"Destination Options and Hop-by-Hop Options" section of the
'ipv6-parameters' registry.
The IANA is instructed to create and maintain a new registry titled
"IP Parcel and Advanced Jumbo Formats and Types". For IPv4 parcels
and Advanced Jumbos, the value in the 'option-length' field of Probe/
Reply MTU options [RFC1063] serves as an "Option Format" code that
distinguishes the various IPv4 option formats specified in this
document. Initial values are given below:
Templin Expires 20 May 2024 [Page 43]
Internet-Draft IP Parcels November 2023
Value Option Format Reference
----- ------------- ---------
4 Probe/Reply MTU [RFC1063]
8 Parcel/Advanced Jumbo [RFCXXXX]
12 Parcel/Advanced Jumbo Probe [RFCXXXX]
16 Jumbo Reply MTU ( 4-octet ID) [RFCXXXX]
20 Jumbo Reply MTU ( 8-octet ID) [RFCXXXX]
24 Jumbo Reply MTU (12-octet ID) [RFCXXXX]
28 Jumbo Reply MTU (16-octet ID) [RFCXXXX]
3-7 Unassigned [RFCXXXX]
9-11 Unassigned [RFCXXXX]
13-15 Unassigned [RFCXXXX]
17-19 Unassigned [RFCXXXX]
21-23 Unassigned [RFCXXXX]
25-27 Unassigned [RFCXXXX]
29-253 Unassigned [RFCXXXX]
254 Reserved for Experimentation [RFCXXXX]
255 Reserved by IANA [RFCXXXX]
Figure 12: IPv4 Parcel/Jumbo Option Formats
For IPv6 parcels and Advanced Jumbos, the value in the 'Opt Data Len'
field of the IPv6 Minimum Path MTU Hop-by-Hop Option [RFC9268] serves
as an "Option Format" code that distinguishes the various IPv6 option
formats specified in this document. Initial values are given below:
Value Option Format Reference
----- ------------- ---------
4 IPv6 Minimum Path MTU [RFC9268]
6 Parcel/Advanced Jumbo [RFCXXXX]
10 Parcel/Advanced Jumbo Probe [RFCXXXX]
12 Jumbo Reply MTU ( 4-octet ID) [RFCXXXX]
16 Jumbo Reply MTU ( 8-octet ID) [RFCXXXX]
20 Jumbo Reply MTU (12-octet ID) [RFCXXXX]
24 Jumbo Reply MTU (16-octet ID) [RFCXXXX]
5 Unassigned [RFCXXXX]
7-9 Unassigned [RFCXXXX]
11 Unassigned [RFCXXXX]
13-15 Unassigned [RFCXXXX]
17-19 Unassigned [RFCXXXX]
21-23 Unassigned [RFCXXXX]
25-253 Unassigned [RFCXXXX]
254 Reserved for Experimentation [RFCXXXX]
255 Reserved by IANA [RFCXXXX]
Figure 13: IPv6 Parcel/Jumbo Option Formats
Templin Expires 20 May 2024 [Page 44]
Internet-Draft IP Parcels November 2023
For all Parcels/Advanced Jumbos and their corresponding probes, the
IP {Total, Header} Length field encodes a "Jumbo Type" value instead
of an ordinary total/payload length. Initial values are given below:
Value Jumbo Type Reference
----- ---------- ---------
0 Basic Jumbogram (IPv6 only) [RFC2675]
1 Reserved for Experimentation [RFCXXXX]
2 Reserved by IANA [RFCXXXX]
3-246 Unassigned [RFCXXXX]
247 Advanced Jumbo / CRC32C [RFCXXXX]
248 Advanced Jumbo / CRC64E [RFCXXXX]
249 Advanced Jumbo / CRC128J [RFCXXXX]
250 Advanced Jumbo / MD5 [RFCXXXX]
251 Advanced Jumbo / SHA1 [RFCXXXX]
252 Advanced Jumbo / SHA-224 [RFCXXXX]
253 Advanced Jumbo / SHA-256 [RFCXXXX]
254 Advanced Jumbo / SHA-384 [RFCXXXX]
255 Advanced Jumbo / SHA-512 [RFCXXXX]
256-8192 IP Parcel / CRC32C [RFCXXXX]
8193-65535 IP Parcel / CRC64E [RFCXXXX]
Figure 14: IP Advanced Jumbo Types
14. Security Considerations
In the control plane, original sources match any identifying
information in received Parcel/Jumbo Reports and IP Jumbo Reply MTU
options with their corresponding probes. If the information matches,
the report is likely authentic. In environments where stronger
authentication is necessary, nodes that send Parcel and/or Jumbo
Reports can apply the message authentication services specified for
AERO/OMNI.
In the data plane, multi-layer security solutions may be needed to
ensure confidentiality, integrity and availability. Since parcels
and advanced jumbos are defined only for TCP and UDP, IPsec-AH/ESP
[RFC4301] cannot be applied in transport mode although they can
certainly be used in tunnel mode at lower layers such as for
transmission of parcels and advanced jumbos over OMNI link secured
spanning trees, VPNs, etc. Since the network layer does not
manipulate transport layer segments, parcels and advanced jumbos do
not interfere with transport or higher-layer security services such
as (D)TLS/SSL [RFC8446] which may provide greater flexibility in some
environments.
Templin Expires 20 May 2024 [Page 45]
Internet-Draft IP Parcels November 2023
IPv4 fragment reassembly is known to be dangerous at high data rates
where undetected reassembly buffer corruptions can result from
fragment misassociations [RFC4963]. IPv6 is less subject to these
concerns when the 32-bit Identification field is managed responsibly.
However, both IPv4 and IPv6 can robustly support high data rate
reassembly using Identification Extension Options for the Internet
Protocol [I-D.templin-intarea-ipid-ext].
IP parcels and advanced jumbos present a new link service model for
the Internet in which intermediate systems may forward packets that
incurred link errors and end systems are responsible for detecting
any link errors incurred along the path. The destination end system
in particular is uniquely positioned to verify and/or correct the
integrity of any transport layer segments received. For this reason,
transport layer protocols that use IP parcels and/or advanced jumbos
should include higher layer error detection/correction codes in
addition to the per-segment link error integrity checks.
The message digests included with Advanced Jumbos are provided as
integrity checks and must not be considered as authentication codes
in the absence of other supporting security services. Further
security considerations related to IP parcels and Advanced Jumbos are
found in the AERO/OMNI specifications.
15. Acknowledgements
This work was inspired by ongoing AERO/OMNI/DTN investigations. The
concepts were further motivated through discussions with colleagues.
A considerable body of work over recent years has produced useful
segmentation offload facilities available in widely-deployed
implementations.
With the advent of networked storage, big data, streaming media and
other high data rate uses the early days of Internetworking have
evolved to accommodate the need for improved performance. The need
fostered a concerted effort in the industry to pursue performance
optimizations at all layers that continues in the modern era. All
who supported and continue to support advances in Internetworking
performance are acknowledged.
This work has been presented at working group sessions of the
Internet Engineering Task Force (IETF). The following individuals
are acknowledged for their contributions: Roland Bless, Scott
Burleigh, Madhuri Madhava Badgandi, Joel Halpern, Tom Herbert, Andy
Malis, Herbie Robinson, Bhargava Raman Sai Prakash.
Honoring life, liberty and the pursuit of happiness.
Templin Expires 20 May 2024 [Page 46]
Internet-Draft IP Parcels November 2023
16. References
16.1. Normative References
[I-D.templin-intarea-ipid-ext]
Templin, F., "Identification Extension for the Internet
Protocol", Work in Progress, Internet-Draft, draft-
templin-intarea-ipid-ext-23, 10 November 2023,
<https://datatracker.ietf.org/doc/html/draft-templin-
intarea-ipid-ext-23>.
[RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768,
DOI 10.17487/RFC0768, August 1980,
<https://www.rfc-editor.org/info/rfc768>.
[RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791,
DOI 10.17487/RFC0791, September 1981,
<https://www.rfc-editor.org/info/rfc791>.
[RFC0792] Postel, J., "Internet Control Message Protocol", STD 5,
RFC 792, DOI 10.17487/RFC0792, September 1981,
<https://www.rfc-editor.org/info/rfc792>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC2675] Borman, D., Deering, S., and R. Hinden, "IPv6 Jumbograms",
RFC 2675, DOI 10.17487/RFC2675, August 1999,
<https://www.rfc-editor.org/info/rfc2675>.
[RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing
Architecture", RFC 4291, DOI 10.17487/RFC4291, February
2006, <https://www.rfc-editor.org/info/rfc4291>.
[RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet
Control Message Protocol (ICMPv6) for the Internet
Protocol Version 6 (IPv6) Specification", STD 89,
RFC 4443, DOI 10.17487/RFC4443, March 2006,
<https://www.rfc-editor.org/info/rfc4443>.
[RFC7323] Borman, D., Braden, B., Jacobson, V., and R.
Scheffenegger, Ed., "TCP Extensions for High Performance",
RFC 7323, DOI 10.17487/RFC7323, September 2014,
<https://www.rfc-editor.org/info/rfc7323>.
Templin Expires 20 May 2024 [Page 47]
Internet-Draft IP Parcels November 2023
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", STD 86, RFC 8200,
DOI 10.17487/RFC8200, July 2017,
<https://www.rfc-editor.org/info/rfc8200>.
[RFC9293] Eddy, W., Ed., "Transmission Control Protocol (TCP)",
STD 7, RFC 9293, DOI 10.17487/RFC9293, August 2022,
<https://www.rfc-editor.org/info/rfc9293>.
16.2. Informative References
[BIG-TCP] Dumazet, E., "BIG TCP, Netdev 0x15 Conference (virtual),
https://netdevconf.info/0x15/session.html?BIG-TCP", 31
August 2021.
[ECMA-182] ECMA, E., "European Computer Manufacturers Association
(ECMA) Standard ECMA-182, https://ecma-international.org/
wp-content/uploads/ECMA-
182_1st_edition_december_1992.pdf", December 1992.
[I-D.ietf-6man-eh-limits]
Herbert, T., "Limits on Sending and Processing IPv6
Extension Headers", Work in Progress, Internet-Draft,
draft-ietf-6man-eh-limits-08, 4 October 2023,
<https://datatracker.ietf.org/doc/html/draft-ietf-6man-eh-
limits-08>.
[I-D.ietf-6man-hbh-processing]
Hinden, R. M. and G. Fairhurst, "IPv6 Hop-by-Hop Options
Processing Procedures", Work in Progress, Internet-Draft,
draft-ietf-6man-hbh-processing-11, 5 November 2023,
<https://datatracker.ietf.org/doc/html/draft-ietf-6man-
hbh-processing-11>.
[I-D.templin-dtn-ltpfrag]
Templin, F., "LTP Fragmentation", Work in Progress,
Internet-Draft, draft-templin-dtn-ltpfrag-16, 23 October
2023, <https://datatracker.ietf.org/doc/html/draft-
templin-dtn-ltpfrag-16>.
Templin Expires 20 May 2024 [Page 48]
Internet-Draft IP Parcels November 2023
[I-D.templin-intarea-aero]
Templin, F., "Automatic Extended Route Optimization
(AERO)", Work in Progress, Internet-Draft, draft-templin-
intarea-aero-50, 23 October 2023,
<https://datatracker.ietf.org/doc/html/draft-templin-
intarea-aero-50>.
[I-D.templin-intarea-omni]
Templin, F., "Transmission of IP Packets over Overlay
Multilink Network (OMNI) Interfaces", Work in Progress,
Internet-Draft, draft-templin-intarea-omni-50, 23 October
2023, <https://datatracker.ietf.org/doc/html/draft-
templin-intarea-omni-50>.
[QUIC] Ghedini, A., "Accelerating UDP packet transmission for
QUIC, https://blog.cloudflare.com/accelerating-udp-packet-
transmission-for-quic/", 8 January 2020.
[RFC0863] Postel, J., "Discard Protocol", STD 21, RFC 863,
DOI 10.17487/RFC0863, May 1983,
<https://www.rfc-editor.org/info/rfc863>.
[RFC1063] Mogul, J., Kent, C., Partridge, C., and K. McCloghrie, "IP
MTU discovery options", RFC 1063, DOI 10.17487/RFC1063,
July 1988, <https://www.rfc-editor.org/info/rfc1063>.
[RFC1071] Braden, R., Borman, D., and C. Partridge, "Computing the
Internet checksum", RFC 1071, DOI 10.17487/RFC1071,
September 1988, <https://www.rfc-editor.org/info/rfc1071>.
[RFC1191] Mogul, J. and S. Deering, "Path MTU discovery", RFC 1191,
DOI 10.17487/RFC1191, November 1990,
<https://www.rfc-editor.org/info/rfc1191>.
[RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321,
DOI 10.17487/RFC1321, April 1992,
<https://www.rfc-editor.org/info/rfc1321>.
[RFC3174] Eastlake 3rd, D. and P. Jones, "US Secure Hash Algorithm 1
(SHA1)", RFC 3174, DOI 10.17487/RFC3174, September 2001,
<https://www.rfc-editor.org/info/rfc3174>.
[RFC3385] Sheinwald, D., Satran, J., Thaler, P., and V. Cavanna,
"Internet Protocol Small Computer System Interface (iSCSI)
Cyclic Redundancy Check (CRC)/Checksum Considerations",
RFC 3385, DOI 10.17487/RFC3385, September 2002,
<https://www.rfc-editor.org/info/rfc3385>.
Templin Expires 20 May 2024 [Page 49]
Internet-Draft IP Parcels November 2023
[RFC4301] Kent, S. and K. Seo, "Security Architecture for the
Internet Protocol", RFC 4301, DOI 10.17487/RFC4301,
December 2005, <https://www.rfc-editor.org/info/rfc4301>.
[RFC4821] Mathis, M. and J. Heffner, "Packetization Layer Path MTU
Discovery", RFC 4821, DOI 10.17487/RFC4821, March 2007,
<https://www.rfc-editor.org/info/rfc4821>.
[RFC4963] Heffner, J., Mathis, M., and B. Chandler, "IPv4 Reassembly
Errors at High Data Rates", RFC 4963,
DOI 10.17487/RFC4963, July 2007,
<https://www.rfc-editor.org/info/rfc4963>.
[RFC5326] Ramadas, M., Burleigh, S., and S. Farrell, "Licklider
Transmission Protocol - Specification", RFC 5326,
DOI 10.17487/RFC5326, September 2008,
<https://www.rfc-editor.org/info/rfc5326>.
[RFC6234] Eastlake 3rd, D. and T. Hansen, "US Secure Hash Algorithms
(SHA and SHA-based HMAC and HKDF)", RFC 6234,
DOI 10.17487/RFC6234, May 2011,
<https://www.rfc-editor.org/info/rfc6234>.
[RFC7126] Gont, F., Atkinson, R., and C. Pignataro, "Recommendations
on Filtering of IPv4 Packets Containing IPv4 Options",
BCP 186, RFC 7126, DOI 10.17487/RFC7126, February 2014,
<https://www.rfc-editor.org/info/rfc7126>.
[RFC8201] McCann, J., Deering, S., Mogul, J., and R. Hinden, Ed.,
"Path MTU Discovery for IP version 6", STD 87, RFC 8201,
DOI 10.17487/RFC8201, July 2017,
<https://www.rfc-editor.org/info/rfc8201>.
[RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol
Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
<https://www.rfc-editor.org/info/rfc8446>.
[RFC8899] Fairhurst, G., Jones, T., Tüxen, M., Rüngeler, I., and T.
Völker, "Packetization Layer Path MTU Discovery for
Datagram Transports", RFC 8899, DOI 10.17487/RFC8899,
September 2020, <https://www.rfc-editor.org/info/rfc8899>.
[RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
Multiplexed and Secure Transport", RFC 9000,
DOI 10.17487/RFC9000, May 2021,
<https://www.rfc-editor.org/info/rfc9000>.
Templin Expires 20 May 2024 [Page 50]
Internet-Draft IP Parcels November 2023
[RFC9171] Burleigh, S., Fall, K., and E. Birrane, III, "Bundle
Protocol Version 7", RFC 9171, DOI 10.17487/RFC9171,
January 2022, <https://www.rfc-editor.org/info/rfc9171>.
[RFC9268] Hinden, R. and G. Fairhurst, "IPv6 Minimum Path MTU Hop-
by-Hop Option", RFC 9268, DOI 10.17487/RFC9268, August
2022, <https://www.rfc-editor.org/info/rfc9268>.
[STONE] Stone, J. and C. Partridge, "When the CRC and TCP Checksum
Disagree, ACM SIGCOMM Computer Communication Review,
Volume 30, Issue 4, October 2000, pp. 309-319,
https://doi.org/10.1145/347057.347561", October 2000.
Appendix A. TCP Extensions for High Performance
TCP Extensions for High Performance are specified in [RFC7323], which
updates earlier work that began in the late 1980's and early 1990's.
These efforts determined that the TCP 16-bit Window was too small to
accommodate sustained transmission at high data rates and devised a
TCP Window Scale option to allow window sizes up to 2^30. The work
also defined a Timestamp option used for round-trip time measurements
and as a Protection Against Wrapped Sequences (PAWS) at high data
rates. TCP users of IP parcels are strongly encouraged to adopt
these measures.
Since TCP/IP parcels only include control bits for the first segment
("segment(0)"), nodes must regard all other segments of the same
parcel as data segments. When a node breaks a TCP/IP parcel out into
individual packets or sub-parcels, only the first packet/sub-parcel
contains the original segment(0) and therefore only its TCP header
retains the control bit settings from the original parcel TCP header.
If the original TCP header included TCP options such as Maximum
Segment Size (MSS), Window Scale (WS) and/or Timestamp, the node
copies those same options into the options section of the new TCP
header.
For all other packets/sub-parcels, the note sets all TCP header
control bits to 0 as data segment(s). Then, if the original parcel
contained a Timestamp option, the node copies the Timestamp option
into the options section of the new TCP header. Appendix A of
[RFC7323] provides implementation guidelines for the Timestamp option
layout.
Appendix A of [RFC7323] also discusses Interactions with the TCP
Urgent Pointer as follows: "if the Urgent Pointer points beyond the
end of the TCP data in the current segment, then the user will remain
in urgent mode until the next TCP segment arrives. That segment will
update the Urgent Pointer to a new offset, and the user will never
Templin Expires 20 May 2024 [Page 51]
Internet-Draft IP Parcels November 2023
have left urgent mode". In the case of IP parcels, however, it will
often be the case that the next TCP segment is included in the same
(sub-)parcel as the segment that contained the urgent pointer such
that the urgent pointer can be updated immediately.
Finally, if the parcel contains more than 65535 octets of data (i.e.,
spread across multiple segments), then the Urgent Pointer can be
regarded in the same manner as for jumbograms as described in
Section 5.2 of [RFC2675].
Appendix B. Extreme L Value Implications
For each parcel, the transport layer can specify any L value between
256 and 65535 octets. Transport protocols that send isolated control
and/or data segments smaller than 256 octets should package them as
ordinary packets, singleton parcels or as the final segment of a
larger parcel. It is also important to note that segments smaller
than 256 octets are likely to include control information for which
timely delivery rather than bulk packaging is desired. Transport
protocol streams therefore often include a mix of (larger) parcels
and (smaller) ordinary packets or singleton parcels.
The transport layer should also specify an L value no larger than can
accommodate the maximum-sized transport and network layer headers
that the source will include without causing a single segment plus
headers to exceed 65535 octets. For example, if the source will
include a 28 octet TCP header plus a 40 octet IPv6 header with 24
extension header octets (plus 6/8 octets for the per-segment
Checksum/CRC) the transport should specify an L value no larger than
(65535 - 28 - 40 - 24 - 10) = 65433 octets.
The transport can specify still larger "extreme" L values up to 65535
octets, but the resulting parcels might be lost along some paths with
unpredictable results. For example, a parcel with an extreme L value
set as large as 65535 might be able to transit paths that can pass
jumbograms natively but might not be able to transit a path that
includes non-jumbo links. The transport layer should therefore
carefully consider the benefits of constructing parcels with extreme
L values larger than the recommended maximum due to high risk of loss
compared with only minor potential performance benefits.
Parcels that include extreme L values larger than the recommended
maximum and with a maximum number of included segments could also
cause a parcel to exceed 16,777,215 (2**24 - 1) octets in total
length. Since the Parcel Payload Length field is limited to 24 bits,
however, the largest possible parcel is also limited by this size.
See also the above risk/benefit analysis for parcels that include
extreme L values larger than the recommended maximum.
Templin Expires 20 May 2024 [Page 52]
Internet-Draft IP Parcels November 2023
Appendix C. Additional Parcel/Jumbo Probe Considerations
After sending a Parcel/Jumbo Probe, the source may receive a Parcel/
Jumbo Report from either a router on the path or from the final
destination itself. Alternatively, the source can shape its probes
to request IP Jumbo Reply MTU options carried by ordinary data
packets/parcels on the return path from the destination.
If a router or final destination receives a Parcel/Jumbo Probe but
does not recognize the parcel/jumbo constructs, it will likely drop
the probe without further processing and may return an ICMP error.
The original source will then consider the probe as lost, but may
attempt to probe again later, e.g., in case the path may have
changed.
When the source examines the "packet in error" portion of a Parcel/
Jumbo Report, it can easily match the Report against its recent
transmissions if the (extended) Identification value is available.
For each "packet in error" that does not include an (extended)
Identification, the source can attempt to match based on any other
identifying information; otherwise, it should discard the message.
If the source receives multiple Parcel/Jumbo Reports for a single
parcel/jumbo sent into a given path, it should prefer any information
reported by the final destination over information reported by a
router. For example, if a router returns a negative report while the
destination returns a positive report the latter should be considered
as more-authoritative. For this reason, the source should provide a
configuration knob allowing it to accept or ignore reports that
originate from routers, e.g., according to the network trust model.
When a destination returns a Parcel/Jumbo Report, it can optionally
pair the report with an ordinary data packet that it returns to the
original source. For example, the OMNI specification includes a
"super-packet" service that allows multiple independent IP packets to
be encapsulated as a single adaptation layer packet. This is
distinct from an IP parcel in that each packet member of the super-
packet includes its own IP (and possibly other upper layer) header.
A source can request to receive two different types of parcel/jumbo
path MTU feedback from the destination - a UDP encapsulated Parcel/
Jumbo Report in response to a probe sent to port 9 (discard), or an
ordinary data packet/parcel with an IP Jumbo Reply MTU option in
response to a probe sent into an ordinary transport layer protocol
flow. In some environments, one or both of these MTU feedback types
may be erroneously dropped by a router along the return path. The
source may therefore attempt to probe first using "method A", and
then try again using "method B", e.g., if there is no response. In
Templin Expires 20 May 2024 [Page 53]
Internet-Draft IP Parcels November 2023
environments where ongoing transport protocol sessions are
established, it is recommended that the source engage the IP Jumbo
Reply MTU option as "method A".
Appendix D. Advanced Jumbo Cyclic Redundancy Check (CRC128J)
This section postulates a 128-bit Cyclic Redundancy Check (CRC)
algorithm for Advanced Jumbos termed "CRC128J". An Advanced Jumbo
Type value 249 is reserved for CRC128J, but at the time of this
writing no algorithm specification exists. Future specifications of
CRC128J may update this document and provide instructions for
handling Advanced Jumbos with Type 249.
Appendix E. Change Log
<< RFC Editor - remove prior to publication >>
Changes from earlier versions:
* Submit for review.
Author's Address
Fred L. Templin (editor)
Boeing Research & Technology
P.O. Box 3707
Seattle, WA 98124
United States of America
Email: fltemplin@acm.org
Templin Expires 20 May 2024 [Page 54]