Skip to main content

Infight Removal of IPv6 Hop-by-Hop and Routing Headers
draft-herbert-eh-inflight-removal-00

The information below is for an old version of the document.
Document Type
This is an older version of an Internet-Draft whose latest revision state is "Expired".
Author Tom Herbert
Last updated 2023-08-22
RFC stream (None)
Formats
Stream Stream state (No stream defined)
Consensus boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-herbert-eh-inflight-removal-00
Network Working Group                                         T. Herbert
Internet-Draft                                                   SiPanda
Intended status: Experimental                             22 August 2023
Expires: 23 February 2024

         Infight Removal of IPv6 Hop-by-Hop and Routing Headers
                  draft-herbert-eh-inflight-removal-00

Abstract

   This document specifies an experimental method to allow intermediate
   nodes to remove IPv6 Hop-by-Hop Options Headers or Routing Headers
   from packets in flight.  The goal is to reduce the probability of
   packets being dropped, because they contain these extension headers,
   without impacting functionality.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 23 February 2024.

Copyright Notice

   Copyright (c) 2023 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Herbert                 Expires 23 February 2024                [Page 1]
Internet-Draft             Inflight-EH-Removal               August 2023

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Motivation  . . . . . . . . . . . . . . . . . . . . . . . . .   3
     2.1.  Hop-by-Hop Options drop rate  . . . . . . . . . . . . . .   3
     2.2.  Router Header domain firewall . . . . . . . . . . . . . .   4
     2.3.  Removing extension headers  . . . . . . . . . . . . . . .   4
       2.3.1.  Removal by egress routers . . . . . . . . . . . . . .   4
       2.3.2.  Removal by ingress routers  . . . . . . . . . . . . .   5
     2.4.  Alternatives to Extension Header removal  . . . . . . . .   5
       2.4.1.  Host routing  . . . . . . . . . . . . . . . . . . . .   5
       2.4.2.  Probing . . . . . . . . . . . . . . . . . . . . . . .   6
       2.4.3.  IPinIP Encapsulation from source  . . . . . . . . . .   6
       2.4.4.  IPinIP Encapsulation from egress router . . . . . . .   7
   3.  Considerations  . . . . . . . . . . . . . . . . . . . . . . .   8
     3.1.  Reflection of Hop-by-Hop Options  . . . . . . . . . . . .   8
     3.2.  End host processing of Routing Headers  . . . . . . . . .   8
     3.3.  ICMP errors . . . . . . . . . . . . . . . . . . . . . . .   9
   4.  Requirements  . . . . . . . . . . . . . . . . . . . . . . . .   9
   5.  Procedures  . . . . . . . . . . . . . . . . . . . . . . . . .  10
     5.1.  Removing a Hop-by-Hop Options Header  . . . . . . . . . .  10
     5.2.  Removing a Routing Header . . . . . . . . . . . . . . . .  12
     5.3.  Removing both Hop-by-Hop Options and a Routing Headers  .  15
   6.  Implementation Considerations . . . . . . . . . . . . . . . .  18
     6.1.  Copying the IPv6 Header . . . . . . . . . . . . . . . . .  19
     6.2.  Scatter/gather  . . . . . . . . . . . . . . . . . . . . .  19
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  19
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  19
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  19
     9.1.  Normative References  . . . . . . . . . . . . . . . . . .  19
     9.2.  Informative References  . . . . . . . . . . . . . . . . .  19
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  21

1.  Introduction

   This document specifies an experimental protocol to allow
   intermediate nodes to remove IPv6 Hop-by-Hop Options Headers or
   Routing Headers from packets in flight.

   Current data suggests that there are very high drop rates, nearing
   100%, for packets with Hop-by-Hop Options sent over the Internet.
   The goal of this protocol is to reduce the probability of the packet
   being dropped by a downstream node without reducing functionality,
   thereby improving the viability and usability for sending Hop-by-Hop
   Options.

Herbert                 Expires 23 February 2024                [Page 2]
Internet-Draft             Inflight-EH-Removal               August 2023

   A secondary goal is to allow removal of Hop-by-Hop Options Headers or
   Routing Headers when packets egress a limited domain, such as a
   segment routing domain, in order to limit exposure of data to only
   those nodes that legitimately need to process it.

1.1.  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

2.  Motivation

   This section provides the motivations for allowing intermediate nodes
   to remove Hop-by-Hop Options or Routing Headers from packets.

2.1.  Hop-by-Hop Options drop rate

   Current measurements indicate that packets with Hop-by-Hop Extension
   Headers have high drop rates when sent over the Internet.  From
   [APNIC-EH]:

      The HBH option was experiencing an average packet drop rate of
      99.5% across all HBH option sizes

   The reported drops rates for Hop-by-Hop Options are greater than that
   of packets with Destination Options Headers or Fragment Headers.  A
   possible explanation for this difference is that Hop-by-Hop Options
   are intended to be processed by intermediate nodes in a network, and
   hence a network operator may be motivated to drop packets with Hop-
   by-Hop options entering their network from untrusted sources to
   protect their network infrastructure.  This is mentioned in [RFC9098]
   as a reason that packets containing IPv6 Hop-by-Hop Options are
   dropped:

      The Hop-by-Hop Options header has been particularly challenging
      since, in most circumstances, the corresponding packet is punted
      to the control plane for processing.  As a result, many operators
      drop IPv6 packets containing this extension header [RFC7872].
      [RFC6192] provides advice regarding protection of a router's
      control plane.

   Given that there doesn't seem to be a easy fix to make Hop-by-Hop
   Options work over the Internet, the commonly proposed alternative is
   to limit use of Hop-by-Hop Options to limited domains [RFC8799].  It
   can be noted that Hop-by-Hop Options are only useful when at least
   some of nodes in the path process them, and a network operator would
   likely only deploy routers that process Hop-by-Hop Options if they

Herbert                 Expires 23 February 2024                [Page 3]
Internet-Draft             Inflight-EH-Removal               August 2023

   perceived Hop-by-Hop Options provide some value.  An example of such
   an option is FAST [I-D.herbert-fast] which allows the network
   infrastructure to provide fine grained QoS and monetize network
   services on a per packet basis.  If a network supports value add
   services that use Hop-by-Hop Options, it stands to reason that if
   packets with Hop-by-Hop Options wouldn't be dropped while their
   within the limited domain of the network operator.

   If a destination is not within the limited domain, a source might
   still desire to use Hop-by-Hop Options to affect packet processing in
   the part of the path that is within the limited domain.  In this
   case, a packet might be created with Hop-by-Hop Options, the packet
   traverses the local network to an egress router, and at the egress
   router the packet is forwarded outside of the limited domain without
   Hop-by-Hop Options.

2.2.  Router Header domain firewall

   When a host sends a packet with a Routing Header, for example a
   Segment Routing Header, the intermediate destinations are considered
   to be in the same limited domain; for example, in Segment Routing all
   of the intermediate destinations in the Segment Routing Header must
   be in the same segment routing domain.

   The final destination of a Routing Header might not be in the routing
   domain.  It may, in fact, be outside of the limited domain.  An
   example use case of this would be if routing was used to route the
   packet to an egress router of the domain.  The egress router would be
   the penultimate destination in the segment list such that the
   Segments Left field is set to zero and all downstream nodes would
   ignore the Routing Header.  In this case, the packets can forwarded
   beyond the limited domain without a routing header and no impact on
   behavior.

2.3.  Removing extension headers

2.3.1.  Removal by egress routers

   To contain the Hop-by-Hop Options and Routing Header to their limited
   domain, this specification proposes that egress routers could remove
   the extension headers from packets before forwarding them beyond the
   limited domain.

   Hop-by-Hop Options would be removed by an egress router in order to
   increase the likelihood that packets sent with Hop-by-Hop Options are
   successfully delivered.  The assumption is that the Hop-by-Hop
   Options are most likely not useful beyond the limited domain.  Option

Herbert                 Expires 23 February 2024                [Page 4]
Internet-Draft             Inflight-EH-Removal               August 2023

   reflection to affect processing in the reverse direction of a flow,
   such as defined in FAST [I-D.herbert-fast], is one case where it
   would be useful to send outside of a limited domain (discussed
   below).

   A Routing Header would be removed at an egress router when its being
   used to route a packet from a host beyond the limited domain.  Note
   that when the penultimate destination processes the routing header,
   it sets the final Destination Address and Segments Left to zero, so
   at that point the Routing Header can be removed without impacting
   downstream processing of the packet.

2.3.2.  Removal by ingress routers

   Hop-by-Hop Options could be removed from packets by ingress routers
   as an alternative to the current practice of dropping the packets
   with Hop-by-Hop Options.  In this case, the network operator doesn't
   process Hop-by-Hop Options, or it only processes Hop-by-Hop Options
   from source hosts in the local domain that it trusts.  Removing Hop-
   by-Hop Options instead of dropping them allows packets to be
   delivered without loss of functionality or risk to the network
   infrastructure.

2.4.  Alternatives to Extension Header removal

   This section discusses some of the alternatives to extension header
   removal that have been proposed.

2.4.1.  Host routing

   It is conceivable that a host network stack could maintain routes to
   destinations or networks with an indication that the destination is
   within the limited domain.  So when a packet is being created, the
   routing table could be consulted to determine if it's safe to send
   packets with Hop-by-Hop Options to the destination.

   The main drawback of this approach is that it requires significant
   changes to the host networking stack: in the routing infrastructure,
   the APIs presented to the application trying to set Hop-by-Hop
   Options, and probably applications themselves.  Additionally, in all
   but trivial network topologies, it won't be obvious just given an
   address whether the destination is in the same limited domain as the
   host.  In some simpler topologies, might be possible to configure
   hosts with all the network prefixes that belong to the limited
   domain, however for a more complex topology hosts may need to
   participate in a routing protocol or a discovery protocol with the
   network.

Herbert                 Expires 23 February 2024                [Page 5]
Internet-Draft             Inflight-EH-Removal               August 2023

2.4.2.  Probing

   Capabilities probing has been successfully employed in other contexts
   such as "Happy Eyeballs" for IPv6.  Probing could similarly be used
   to determine the viability of Hop-by-Hop Options to a destination.
   In this case, a host could probe each destination to determine if
   Hop-by-Hop Options are viable.  The advantage of this method is that
   it requires no special assistance from the network.

   The main drawback of this approach is the complexity in the host
   stack and applications.  Probing assumes bidirectional
   communications, state needs to be maintained for each desintion or
   flow, procedures need to be specified for probing, backoff, and
   continuous probing in the case of a route changes that might affect
   the disposition of packet with Hop-by-Hop Options in the network.
   Additionally, the implementation for probing would be different for
   UDP and TCP: probing in the UDP case would most likely need support
   in the application and userspace libraries, probing for TCP would
   likely need to be supported in the kernel itself.

2.4.3.  IPinIP Encapsulation from source

   In order to use Hop-by-Hop Options in the part of the path in a
   limited domain, a source host may encapsulate the packet in an IPinIP
   encapsulation [RFC2473].  The outer IPv6 header would contain the
   Hop-by-Hop Options header and the destination would be the address of
   an egress router for the limited domain.  At the egress router, the
   packet would be decapsulated and the packet can be forwarded without
   Hop-by-Hop Options.

   The main problem to this approach is that the sending host would need
   to known the correct Destination Address to set in the encapsulating
   header; that is, the host would need to know the address of the
   correct egress router for the packet.  That information is not
   available to hosts and might not even be available to intermediate
   nodes including the first hop router.  In a complex, multi-homed,
   network topology that might support mobile hosts, the only way to
   determine the current egress router for a packet may be to actually
   route through the network to the external destination address.

   If the network did maintain the association between destinations and
   the egress router, then conceptually it could share that information
   with hosts using a routing protocol or discovery protocol.  This
   information could be saved in an augmented routing table on the host
   similar to that described in the "Host routing" section.

Herbert                 Expires 23 February 2024                [Page 6]
Internet-Draft             Inflight-EH-Removal               August 2023

   If the network provides the addresses of egress routers that is
   potentially divulging network topology information to the hosts and
   could be considered a security risk.

   Conceivably, a host could be configured with a single anycast address
   to be used as Destination Address of the egress routier when
   encapsulating.  If the host routing table includes limited domain
   information, as described in the Host Routing section, then this
   would be sufficient to route packets to an egress router.  In this
   case though, the anycast address represents a default router which
   might not be the same one had the packet been routed based on its
   final destination-- this could be suboptimal routing or cause out-of-
   order packets if not all packets of a flow are encapsulated.

   This solution is complex from a host implementation point of view.
   An IPinIP encapsulation adds at least forty bytes of overhead to the
   packet, which reduces the effective MTU for the application and
   requires special end host processing that may be prohibitive on low
   end devices.  Even if an anycast address is configured, a host stack
   will need to maintain routing information to determine when packets
   need to be encapsulated.  Furthermore, setting the Hop-by-Hop Options
   is be done by the application without regard to whether the packet is
   being encapsulated.  When a packet is sent and it needs to
   encapsulated, the host stack will need to remove the Hop-by-Hop
   Options from the original packet and set them in the encapsulating
   IPv6 headers.

2.4.4.  IPinIP Encapsulation from egress router

   Another solution using IPinIP encapsulation would be for an egress
   router to encapsulate a packet containing Hop-by-Hop Options in
   IPinIP.  The outer IPv6 header contains no Hop-by-Hop Options and the
   inner IPv6 header contains the options.  The Destination Address in
   the outer and inner IP headers are the same.

   This solution is not robust since the encapsulation increases packet
   size and reduces the Path MTU seen by the sender which can cause
   systematic packet drops.  For example, suppose a host sends a packet
   with minimum MTU size of 1,280, and an egress router encapsulates the
   packet so that its length increase to 1,320 bytes.  If a downstream
   router has link MTU of 1,280 then the packet will be dropped since
   its length exceeds the link MTU.  Since the host sent a minimum MTU
   sized packet, it cannot fallback to a smaller MTU using PLMTUD hence
   there is no recovery.  Note the encapsulation is being done when
   packet egress a domain there is no expectation that all the potential
   paths outside of the domain have a large enough MTU to accommodate
   encapsulation.

Herbert                 Expires 23 February 2024                [Page 7]
Internet-Draft             Inflight-EH-Removal               August 2023

   Sending encapsulated packets into the Internet requires that they can
   successfully transit the Internet.  IPinIP encapsulation number could
   be filtered by some networks (similar to how networks can block
   packets with Hop-by-Hop Options header.  Using a UDP encapsulation,
   such as VXLAN, might have better success than IPinIP.

   All potential receivers would need to do decapsulation.  This could
   be modeled as an anonymous encapsulation.  Currently, this is not
   enabled on commodity host stacks, and would be a major change in
   deployment.

   Packets to a destination may undergo network address translation such
   that the outer addresses might not match the inner addresses of an
   encapsulation.  If a flow contains a mix of encapsulated and non-
   encapsulated packets then the destination may view that a packet in
   different flows.  In order to prevent this, a router could
   encapsulate all packets, but that would be very costly for what is
   currently a narrow use case.

3.  Considerations

3.1.  Reflection of Hop-by-Hop Options

   Some Hop-by-Hop options are designed to be reflected by a remote host
   back to the sender.  IOAM Loopback [RFC9332] is used to report
   measurements on the forward path of a sender, the Minimum Path MTU
   Hop-by-Hop Option [RFC9268] returns the path MTU in the forward path
   to a sender, and FAST [I-D.herbert-fast] allows tickets to be
   reflected to affect packet processing in the return path of a flow.
   Note that Hop-by-Hop Options reflection is not guaranteed and hence
   is an opportunistic mechanism; it cannot be assumed that options will
   always be reflected.

   In the case that an intermediate node removes Hop-by-Hop Options,
   reflection won't happen since the destination host does not see the
   Hop-by-Hop option to be reflected.  In order to preserve the benefits
   of reflection, intermediate nodes should only remove Hop-by-Hops that
   might include options to be reflected as a last resort to prevent the
   packets being dropped by a downstream node.

3.2.  End host processing of Routing Headers

   Per [RFC8200], "If Segments Left is zero, the node must ignore the
   Routing header and proceed to process the next header in the packet".
   Effectively, this means once the last segment has been processed and
   the final destination is set then the routing header carries no
   useful information to any downstream nodes, so removal of the
   extension header doesn't affect the how the packet is processed.

Herbert                 Expires 23 February 2024                [Page 8]
Internet-Draft             Inflight-EH-Removal               August 2023

   A possible exception is that the destination host may elect to
   validate the Routing Header.  For instance, the end host may validate
   the HMAC TLV in a Segment Routing Header.  Since routing header are
   most likely used only in limited domains, which is an explicit
   requirement in Segment Routing, the network nodes processing the
   routing header should know if the final destination participates is
   required to validate the routing header-- if it's not then the header
   can be removed.

3.3.  ICMP errors

   When an ICMP error message is sent for a packet with removed
   extension headers, the packet headers in the ICMP data will be
   different then what the host sent.  Operationally, this should not be
   an issue since a sender doesn't normally need to correlate packet
   with Hop-by-Hop options that were originally sent and the host stack
   doesn't usually maintain sufficient state to make a precise
   correlation.

   It is possible that a packet may be dropped because it does not have
   an expected Hop-by-Hop Options, such as a firewall ticket.  In this
   case, the ICMP error does contain relevant information that can be
   logged and used for debugging.

4.  Requirements

   An intermediate node MAY remove a Hop-by-Hop Options extension header
   from a packet if the following conditions are met:

   *  The packet does not contain an Authentication Header.  If the
      packet contains and Authentication Header then the Hop-by-Hop
      Options Extension Header MUST NOT be removed

   *  The Payload Length of the packet is non-zero and the Hop-by-Hop
      options does not include a Jumbo Payload Option (if the packet
      contains a Jumbo Payload option then the Payload Length should be
      zero)

   An intermediate node MAY remove a Routing Header extension header
   from a packet if the following conditions are met:

   *  The Destination Address has been set to the address of the final
      destination and the Segments Left field is zero

   *  The packet does not contain an Authentication Header

   *  There are no extension headers the precede the Routing Header in

Herbert                 Expires 23 February 2024                [Page 9]
Internet-Draft             Inflight-EH-Removal               August 2023

      the packet.  An exception is if the Routing Header immediately
      follow a Hop-by-Hop Options extension header that is also being
      removed

   *  The final destination is not required to process or validate the
      Routing Header

   *  The routing header does not contain options (segment routing TLVs
      for instance), or the destination host doesn't need to process or
      validate the options.

5.  Procedures

   This section describes the procedures for removing a Hop-by-Hop
   Options Header, removing a Routing Header, and removing a Hop-by-Hop
   Options Header and Routing Header at the same time.

5.1.  Removing a Hop-by-Hop Options Header

   The procedures for removing a Hop-by-Hop Options Header are:

   1.  Save the value in the Next Header field of the Hop-by-Hop Options
       extension header in a temporary variable

   2.  Determine the length of the Hop-by-Hop header and save in a
       temporary variable.  This is equal to the value of the Hdr Ext
       Len field time eight plus eight

   3.  Determine the offset of the first byte in the following the Hop-
       by-Hop Options Header.  This is equal to the forty plus the
       length of the Hop-by-Hop Options Header derived in step 2

   4.  Copy the IPv6 header with length, forty bytes, to the offset
       derived in set 3 minus forty.  Reset the starting offset of the
       packet to be the offset of the copied IPv6 header

   5.  Set the Next Header field in the copied IPv6 header to the value
       saved in step 1

   6.  Subtract the length of the Hop-by-Hop Options Header, determined
       in step 2, from the Payload Length in the copied IPv6 header.
       Set the result as the Payload Length in the copied IPv6 header

   An example of removing Hop-by-Hop Options Header is shown in the
   diagrams below.

Herbert                 Expires 23 February 2024               [Page 10]
Internet-Draft             Inflight-EH-Removal               August 2023

   The diagram below illustrates shows an example TCP/IPv6 packet with a
   Hop-by-Hop Options Header; the Payload Length is 1200 bytes and the
   length of the Hop-by-Hop Options Header is sixty-four bytes.

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |Version| Traffic Class |           Flow Label                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |    Payload Length = 1200      |  Next Hdr = 0 |   Hop Limit   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                         Source Address                        +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                      Destination Address                      +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  Next Hdr = 6 |   EH Len = 7  |                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
      |                                                               |
      .                                                               .
      .                            Options                            .
      .                                                               .
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      .                                                               .
      .                    TCP packet and payload                     .
      .                                                               .
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   The diagram below illustrates the packet after the Hop-by-Hop Options
   Header has been removed.  Note that the Payload Length is now 1,136
   bytes which is the original payload length minus the length of the
   Hop-by-Hop Options Header that was removed.

Herbert                 Expires 23 February 2024               [Page 11]
Internet-Draft             Inflight-EH-Removal               August 2023

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |Version| Traffic Class |           Flow Label                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |    Payload Length = 1136      |  Next Hdr = 6 |   Hop Limit   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                         Source Address                        +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                      Destination Address                      +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      .                                                               .
      .                    TCP packet and payload                     .
      .                                                               .
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

5.2.  Removing a Routing Header

   The procedures for removing a Routing Header are:

   1.  Save the value in the Next Header field of the Routing Header in
       a temporary variable

   2.  Determine the length of the Routing Header and save in a
       temporary variable.  This is equal to the value of the Hdr Ext
       Len field time eight plus eight

   3.  Determine the offset of the first byte in the following the
       Routing Header.  This is equal to the forty plus the length of
       the Hop-by-Hop Options header derived in step 2

   4.  Copy the IPv6 header with length, forty bytes, to the offset
       derived in set 3 minus forty.  Reset the starting offset of the
       packet to be the offset of the copied IPv6 header

Herbert                 Expires 23 February 2024               [Page 12]
Internet-Draft             Inflight-EH-Removal               August 2023

   5.  Set the Next Header field in the copied IPv6 header to the value
       saved in step 1

   6.  Subtract the length of the Routing Header, determined in step 2,
       from the Payload Length in the copied IPv6 header.  Set the
       result as the Payload Length in the copied IPv6 header

   An example of removing Routing Header is shown in the diagrams below.

   The diagram below illustrates shows an example TCP/IPv6 packet with a
   Routing Header; the Payload Length is 1400 bytes and the length of
   the Routing Header is 160 bytes.  The Segments Left field is set to
   zero so that the Routing Header may be removed.

Herbert                 Expires 23 February 2024               [Page 13]
Internet-Draft             Inflight-EH-Removal               August 2023

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |Version| Traffic Class |           Flow Label                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |    Payload Length = 1400      |  Next Hdr = 43|   Hop Limit   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                         Source Address                        +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                      Destination Address                      +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  Next Hdr = 6 |  EH Len = 19  |  Routing Type | Segs Left = 0 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      .                                                               .
      .                       type-specific data                      .
      .                                                               .
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      .                                                               .
      .                    TCP packet and payload                     .
      .                                                               .
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   The diagram below illustrates the packet after the Routing Header has
   been removed.  Note that the Payload Length is now 1,240 bytes which
   is the original payload length minus the length of the Routing Header
   that was removed.

Herbert                 Expires 23 February 2024               [Page 14]
Internet-Draft             Inflight-EH-Removal               August 2023

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |Version| Traffic Class |           Flow Label                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |    Payload Length = 1240      |  Next Hdr = 6 |   Hop Limit   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                         Source Address                        +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                      Destination Address                      +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      .                                                               .
      .                    TCP packet and payload                     .
      .                                                               .
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

5.3.  Removing both Hop-by-Hop Options and a Routing Headers

   The procedures for removing both a Hop-by-Hop Options Header and a
   Routing Header are:

   1.  Save the value in the Next Header field of the Routing Header
       extension header in a temporary variable

   2.  Determine the length of the Hop-by-Hop Options Header and save in
       a temporary variable.  This is equal to the value of the Hdr Ext
       Len field time eight plus eight

   3.  Determine the length of the Routing Header and save in a
       temporary variable.  This is equal to the value of the Hdr Ext
       Len field time eight plus eight

   4.  Determine the offset of the first byte in the packet following
       the Routing Header.  This is equal to the forty plus the length
       of the Hop-by-Hop Options Header derived in step 2 plus the
       length of the Routing Header derived in step 3

Herbert                 Expires 23 February 2024               [Page 15]
Internet-Draft             Inflight-EH-Removal               August 2023

   5.  Copy the IPv6 header with length, forty bytes, to the offset
       derived in set 3 minus forty.  Reset the starting offset of the
       packet to be the offset of the copied IPv6 header

   6.  Set the Next Header field in the copied IPv6 header to the value
       saved in step 1

   7.  Subtract the length of the Hop-by-Hop Options Header plus the
       length of the Routing Header (values determined in step 2 and
       step 3) from the Payload Length in the copied IPv6 header.  Set
       the result as the Payload Length in the copied IPv6 header

   An example of removing a Hop-by-Hop Options Header a Routing Header
   is shown in the diagrams below.

   The diagram below illustrates an example TCP/IPv6 packet with both a
   Hop-by-Hop Options Header and a Routing Header; the Payload Length is
   1,300 bytes, the length of the Hop-by-Hop Options Header is sixty-
   four bytes, the length of the Routing Header is 160 bytes.  The
   Segments Left field is set to zero so that the Routing Header may be
   removed.

Herbert                 Expires 23 February 2024               [Page 16]
Internet-Draft             Inflight-EH-Removal               August 2023

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |Version| Traffic Class |           Flow Label                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |    Payload Length = 1300      |  Next Hdr = 0 |   Hop Limit   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                         Source Address                        +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                      Destination Address                      +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  Next Hdr = 43 |   EH Len = 7 |                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
      |                                                               |
      .                                                               .
      .                            Options                            .
      .                                                               .
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |  Next Hdr = 6 |  EH Len = 19  |  Routing Type | Segs Left = 0 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      .                                                               .
      .                       type-specific data                      .
      .                                                               .
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      .                                                               .
      .                    TCP packet and payload                     .
      .                                                               .
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Herbert                 Expires 23 February 2024               [Page 17]
Internet-Draft             Inflight-EH-Removal               August 2023

   The diagram below illustrates the packet after the Hop-by-Hop Options
   Header and the Routing Header have been removed.  Note that the
   Payload Length is now 1,076 bytes which is the original payload
   length minus the length of the Hop-by-Hop Options Header and the
   Routing Header that were removed.

      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |Version| Traffic Class |           Flow Label                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |    Payload Length = 1076      |  Next Hdr = 6 |   Hop Limit   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                         Source Address                        +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                      Destination Address                      +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      .                                                               .
      .                    TCP packet and payload                     .
      .                                                               .
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

6.  Implementation Considerations

   Removal of extension headers must be efficient and considered a "fast
   path" operation in a router [I-D.ietf-6man-hbh-processing].  The most
   computationally complex part of removing extension headers is moving
   the IPv6 header.  There are two methods to move the bits of the IPv6
   header: memory copy and scatter/gather.

Herbert                 Expires 23 February 2024               [Page 18]
Internet-Draft             Inflight-EH-Removal               August 2023

6.1.  Copying the IPv6 Header

   Extension header removal can be accomplished by performing a data
   copy of the IPv4 header (forty bytes) to the offset after the
   extension header being removed minus forty bytes.  Since the number
   of bytes being moved is relatively small and fits within a typical
   cacheline, the data copy is amenable to efficient implementation in
   hardware or software.  Once the copy completes, the pointer to the
   packet is advanced by the length of data removed.  Note that an
   implemenation may choose to move the link layer header as well.

6.2.  Scatter/gather

   Scatter/gather allows a packet to be constructed from a list of
   memory buffers where each buffer has a data pointer and length.  To
   use scatter/gather for externsion header removal, a receiver might
   employ header/data split to store the packet as two buffers in
   memory: the first buffer contains the link layer and IPv6 headers,
   and the second buffer contains the data following the IPv6 header.
   Removing an extension headers entails advancing the pointer to the
   second buffer by the length of the extension header being removed.

7.  Security Considerations

   This specification does not introduce any new security concerns,

8.  IANA Considerations

   There are no IANA considerations in this specification.

9.  References

9.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC8200]  Deering, S. and R. Hinden, "Internet Protocol, Version 6
              (IPv6) Specification", STD 86, RFC 8200,
              DOI 10.17487/RFC8200, July 2017,
              <https://www.rfc-editor.org/info/rfc8200>.

9.2.  Informative References

Herbert                 Expires 23 February 2024               [Page 19]
Internet-Draft             Inflight-EH-Removal               August 2023

   [APNIC-EH] Huston, G., "IPv6 extension headers revisited", October
              2022, <https://blog.apnic.net/2022/10/13/ipv6-extension-
              headers-revisited>.

   [I-D.herbert-fast]
              Herbert, T., "Firewall and Service Tickets (FAST)", Work
              in Progress, Internet-Draft, draft-herbert-fast-06, 4
              August 2023, <https://datatracker.ietf.org/doc/html/draft-
              herbert-fast-06>.

   [I-D.ietf-6man-hbh-processing]
              Hinden, R. M. and G. Fairhurst, "IPv6 Hop-by-Hop Options
              Processing Procedures", Work in Progress, Internet-Draft,
              draft-ietf-6man-hbh-processing-09, 4 July 2023,
              <https://datatracker.ietf.org/doc/html/draft-ietf-6man-
              hbh-processing-09>.

   [RFC2473]  Conta, A. and S. Deering, "Generic Packet Tunneling in
              IPv6 Specification", RFC 2473, DOI 10.17487/RFC2473,
              December 1998, <https://www.rfc-editor.org/info/rfc2473>.

   [RFC6192]  Dugal, D., Pignataro, C., and R. Dunn, "Protecting the
              Router Control Plane", RFC 6192, DOI 10.17487/RFC6192,
              March 2011, <https://www.rfc-editor.org/info/rfc6192>.

   [RFC7872]  Gont, F., Linkova, J., Chown, T., and W. Liu,
              "Observations on the Dropping of Packets with IPv6
              Extension Headers in the Real World", RFC 7872,
              DOI 10.17487/RFC7872, June 2016,
              <https://www.rfc-editor.org/info/rfc7872>.

   [RFC8799]  Carpenter, B. and B. Liu, "Limited Domains and Internet
              Protocols", RFC 8799, DOI 10.17487/RFC8799, July 2020,
              <https://www.rfc-editor.org/info/rfc8799>.

   [RFC9098]  Gont, F., Hilliard, N., Doering, G., Kumari, W., Huston,
              G., and W. Liu, "Operational Implications of IPv6 Packets
              with Extension Headers", RFC 9098, DOI 10.17487/RFC9098,
              September 2021, <https://www.rfc-editor.org/info/rfc9098>.

   [RFC9268]  Hinden, R. and G. Fairhurst, "IPv6 Minimum Path MTU Hop-
              by-Hop Option", RFC 9268, DOI 10.17487/RFC9268, August
              2022, <https://www.rfc-editor.org/info/rfc9268>.

Herbert                 Expires 23 February 2024               [Page 20]
Internet-Draft             Inflight-EH-Removal               August 2023

   [RFC9332]  De Schepper, K., Briscoe, B., Ed., and G. White, "Dual-
              Queue Coupled Active Queue Management (AQM) for Low
              Latency, Low Loss, and Scalable Throughput (L4S)",
              RFC 9332, DOI 10.17487/RFC9332, January 2023,
              <https://www.rfc-editor.org/info/rfc9332>.

Author's Address

   Tom Herbert
   SiPanda
   Santa Clara, CA,
   United States of America
   Email: tom@herbertland.com

Herbert                 Expires 23 February 2024               [Page 21]