Skip to main content

Operational Issues with Processing of the Hop-by-Hop Options Header

Document Type Active Internet-Draft (v6ops WG)
Authors Shuping Peng , Zhenbin Li , Chongfeng Xie , Zhuangzhuang Qin , Gyan Mishra
Last updated 2024-02-26 (Latest revision 2024-02-16)
Replaces draft-peng-v6ops-hbh
RFC stream Internet Engineering Task Force (IETF)
Intended RFC status (None)
Additional resources Mailing list discussion
Stream WG state In WG Last Call
Document shepherd Tom Herbert
Shepherd write-up Show Last changed 2024-02-26
IESG IESG state I-D Exists
Consensus boilerplate Unknown
Telechat date (None)
Responsible AD (None)
Send notices to
Network Working Group                                            S. Peng
Internet-Draft                                                     Z. Li
Intended status: Informational                       Huawei Technologies
Expires: 20 August 2024                                           C. Xie
                                                           China Telecom
                                                                  Z. Qin
                                                            China Unicom
                                                               G. Mishra
                                                            Verizon Inc.
                                                        17 February 2024

  Operational Issues with Processing of the Hop-by-Hop Options Header


   This document describes the processing of the Hop-by-Hop Options
   Header (HBH) in current routers in the aspects of standards
   specification, common implementations, and default operations.  This
   document outlines reasons why the Hop-by-Hop Options Header is rarely
   utilized in current networks.  In addition, this document describes
   how HBH Options Header could be used as a powerful mechanism allowing
   deployment and operations of new services requiring a more optimized
   way to leverage network resources of an infrastructure.  The purpose
   of this draft is to document reasons why HBH Options Header is rarely
   used within networks.  It motivates the benefits and requirements
   needed to enable wider use of HBH Options.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   document are to be interpreted as described in [RFC2119] [RFC8174]
   when, and only when, they appear in all capitals, as shown here.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at

Peng, et al.             Expires 20 August 2024                 [Page 1]
Internet-Draft           Processing HBH Opt Hdr            February 2024

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 20 August 2024.

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   4
   3.  New Services  . . . . . . . . . . . . . . . . . . . . . . . .   4
   4.  Modern Router Architecture  . . . . . . . . . . . . . . . . .   5
   5.  Specification of RFC 8200 . . . . . . . . . . . . . . . . . .   6
   6.  Common Implementations  . . . . . . . . . . . . . . . . . . .   7
   7.  Typical Processing  . . . . . . . . . . . . . . . . . . . . .   8
   8.  Design Principles . . . . . . . . . . . . . . . . . . . . . .   8
   9.  Migration Strategies  . . . . . . . . . . . . . . . . . . . .   9
   10. Security Considerations . . . . . . . . . . . . . . . . . . .   9
   11. IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
   12. Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   9
   13. References  . . . . . . . . . . . . . . . . . . . . . . . . .  10
     13.1.  Normative References . . . . . . . . . . . . . . . . . .  10
     13.2.  Informative References . . . . . . . . . . . . . . . . .  11
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  12

1.  Introduction

   When IPv6 was first implemented on high-speed routers, Hop-by-Hop
   Options header (HBH) options were not yet well-understood and ASICs
   were not as capable as they are today.  Due to these historical
   reasons, as well as limited IPv6 deployments and few service
   requirements, the early common IPv6 implementation was that the node
   sends the IPv6 packets with the HBH Options Header to the control

Peng, et al.             Expires 20 August 2024                 [Page 2]
Internet-Draft           Processing HBH Opt Hdr            February 2024

   plane of the node [RFC9098].  The option type of each option carried
   within the HBH Options Header is not even examined before the packet
   is sent to the control plane [RFC7045].  Such processing behavior is
   often the default configuration or, even worse, is the only behavior
   of the IPv6 implementation of the node.

   Sending the HBH Options Header to the control plane for processing
   could result in Denial of Service (DoS) attack on the router control
   plane [RFC9288] and inconsistent packet drops.  This is due to rate-
   limiting on the interface between the router control plane and
   forwarding plane, which impacts packet forwarding.  This has in turn
   led to configuration of measures (such as infrastructure Access
   Control Lists (ACLs)) to mitigate this DoS vector.

   This actually introduced a circular problem:

   -> Implementations that send packets to the control plane for
   processing are at risk for Denial of Service attack.

   -> To mitigate the risk of DoS attack, many network operators
   configured their nodes to discard all packets containing HBH Options

   -> Because of network operators' configurations that discard packets
   containing HBH Options Header, protocol designers stopped defining
   new HBH Options.

   -> Because protocol designers stopped defining new HBH Options, the
   community was not motivated to change the conventional implementation
   that caused use of a HBH Option Header to become a DoS vector.

   Driven by the wide deployments of IPv6 and ever-emerging new services
   in recent years, the HBH Options Header is a valuable container for
   carrying the information to facilitate these new services.

   The purpose of this work is to:

   *  Break the endless cycle that resulted in HBH being a DOS vector.

   *  Enable the HBH options header to be utilized in a safe and secure
      way without impacting the control plane.

   *  Ease the deployments of the new network services that utilize the
      HBH Option Header in a multi-vendor scenario that can now be
      deployed without operational impact.

Peng, et al.             Expires 20 August 2024                 [Page 3]
Internet-Draft           Processing HBH Opt Hdr            February 2024

2.  Terminology

   The terms Forwarding Plane and Control Plane used in this draft refer
   to the terminologies as defined in [I-D.ietf-6man-hbh-processing].

3.  New Services

   As IPv6 is being rapidly and widely deployed worldwide, more and more
   applications and network services are migrating to or directly
   adopting IPv6.  New services that require hop-by-hop processing are
   emerging and the HBH Options header is going to be utilized by the
   new services in various scenarios.

   In-situ OAM (IOAM) with IPv6 encapsulation [RFC9486] is used to
   enhance diagnostics of IPv6 networks and complements other
   mechanisms, such as the IPv6 Performance and Diagnostic Metrics
   Destination Option described in [RFC8250].  The IOAM data fields are
   encapsulated in "option data" fields of the Hop-by-Hop Options

   The Alternate Marking Option is defined as a new IPv6 Hop-by-Hop
   option to encode an alternate marking technique [RFC9343].

   The Minimum Path MTU Hop-by-Hop Option is defined in [RFC9268] to
   record the minimum Path MTU along the forward path between a source
   host to a destination host.  This Hop-by-Hop option is intended to be
   used in environments like Data Centers and on paths between Data
   Centers as well as other environments including the general Internet.
   It provides a useful tool to better take advantage of paths capable
   of supporting a large Path MTU.

   A Virtual Transport Network (VTN) Option is used to carry the VTN
   related information, which could used to identify the set of network
   resources allocated to a VTN and the rules for packet processing.

   As more services start utilizing the HBH Options header, more packets
   containing HBH Options are going to be injected into the networks.  A
   currently common configuration in deployed networks is for routers to
   send all the packets of the new services to the control plane of the
   nodes, with the possible consequence of resulting in a DoS
   vulnerability on the control plane.  The packets will be dropped and
   the normal IP forwarding is severely impacted.  The deployment of new
   network services involving multi-vendor interoperability is greatly

Peng, et al.             Expires 20 August 2024                 [Page 4]
Internet-Draft           Processing HBH Opt Hdr            February 2024

4.  Modern Router Architecture

   The architecture of modern routers deployed by Internet operators
   maintains a strict separation of the router control plane and its
   forwarding plane [RFC6192], as shown in Figure 1.  Either the control
   plane or the forwarding plane is composed of both software and
   hardware, but each plane is responsible for different functions.  In
   this document, we focus on only hardware routers following the
   architecture as shown in Figure 1 which are typical of core routers
   or datacenter routers, as opposed to software-based routers such as
   home routers or hotspot routers in a smartphone.

                             | Router Control |
                             |     Plane      |
                                    | |
                                 Interface Z
                                    | |
                             |   Forwarding   |
               Interface X ==[     Plane      ]== Interface Y

Figure 1. IP router architecture splitting the control and forwarding planes

   Many of the packet-processing devices employed in current router
   designs are fixed-function ASICs that handle common functions,
   however there is a trend towards programmable device datapaths such
   as P4 [P4].  While these devices can be very efficient for the set of
   functions they are designed for, they can be very inflexible.  There
   is a tradeoff of price, performance and flexibility when vendors make
   a choice to use a fixed function ASIC as opposed to Network
   Processing Unit (NPU).  Due to the inflexibility of the fixed
   function ASIC, tasks that require additional processing such as IPv6
   HBH header processing must be punted to the control plane.

   This problem is still a challenge today and is the reason why
   operators drop or ignore HBH options to protect against control plane
   DOS attack.  Semiconductor chip technology has advanced significantly
   in the last decade.  An industry trend is for intelligent multi-core
   CPU hardware using modern NPUs for forwarding packets at line rate
   while still being able to perform other complex tasks such as HBH
   forwarding options processing without having to punt to the control
   plane increasing the viability of ubiquitous HBH use cases.

Peng, et al.             Expires 20 August 2024                 [Page 5]
Internet-Draft           Processing HBH Opt Hdr            February 2024

   Parsing buffers with a limited size are commonly used in router
   implementations.  The parsing buffer contains the headers of a packet
   that can be processed in the fast path and typically have sizes like
   128, 256, 284 bytes.  A source node is permitted to encode hundreds
   of options in 2,048 bytes [I-D.ietf-6man-eh-limits].  With today's
   technology it would be cost prohibitive to be able to process
   hundreds of options with either NPU or proprietary fixed function
   ASIC [I-D.ietf-6man-eh-limits].

   IPv6 Extended Header limitations that need to be addressed to make
   HBH processing more efficient and viable in the forwarding plane.

   [I-D.ietf-6man-eh-limits] defines the IPv6 router requirements and
   how to protect a router from excessive header chain and excessive
   header options with various limitations that can be defined on a
   router.  Per [RFC8200] HBH options must be processed serially.
   However, an implementation of options processing can be done with
   more parallelism in serial processing by grouping similar options to
   be processed in parallel.

   Each Option is encoded in a TLV and so processing of a long list of
   TLVs is expensive.  Zero data length encoded options TLVs are a valid
   option with a minimum length of two bytes.  A DOS vector could be
   easily generated by encoding around 700 HBH options (Zero data
   length) in a standard 1500 MTU packet.  Limiting the number of
   options that are processed could avoid an arbitrary large amount of
   processing for a single EH.

5.  Specification of RFC 8200

   As specified in [RFC8200], the HBH Options header is used to carry
   optional information that will be examined and processed by every
   node along a packet's delivery path, and it is identified by a Next
   Header value of zero in the IPv6 header.

   As per [RFC8200], it is now expected that nodes along a packet's
   delivery path only examine and process the Hop-by-Hop Options header
   if explicitly configured to do so.  This means that the HBH
   processing behavior in a node depends on its configuration.

   However, in the current [RFC8200], there is no explicit specification
   of the possible configurations leading to likely uncertain processing
   behaviors, that is, the nodes may be configured to ignore the HBH
   Options Header, drop packets containing a HBH Options Header, or
   assign packets containing a HBH Options Header to the control plane
   [RFC8200].  This specification is updated in

Peng, et al.             Expires 20 August 2024                 [Page 6]
Internet-Draft           Processing HBH Opt Hdr            February 2024

6.  Common Implementations

   Current routers deployed by operators usually process an IPv6 packet
   that has a Next Header field set to 0 by directly forwarding it to
   the control plane of the node.  With such implementations, the value
   of the Next Header field in the IPv6 header is the only trigger for
   the default processing behavior.  The option type of each option
   carried within the HBH Options Header is not even examined before the
   packet is sent to the control plane.

   Very often, such processing behavior is the default configuration on
   the node, which is embedded in the implementation and cannot be
   changed or reconfigured.

   Another critical component of IPv6 HBH processing, in some cases
   overlooked, is that an operator core network can be designed to use
   the global Internet routing table for internet traffic and in other
   cases use an overlay MPLS VPN to carry Internet traffic.

   In the global Internet routing table scenario where only an underlay
   global routing table exists, and no VPN overlay carrying customer
   Internet traffic, the IPv6 HBH options could be used as a DOS attack
   vector for both the operator nodes, adjacent inter-AS peer nodes, as
   well as customer nodes along a path.

   In a case where the Internet routing table is carried in an MPLS VPN
   overlay payload, the HBH options header does not impact the operator
   underlay framework and only impacts the VPN overlay payload.  Thus
   the operator underlay top most label global table routing FEC LSP
   instantiation is not impacted as the operator underlay is within the
   operators closed domain.

   However, the HBH options DOS attack vector in the VPN overlay can
   still impact the customer end nodes as well as other adjacent inter-
   AS operators that only use the underlay global Internet routing
   table.  In an operator domain where an MPLS VPN overlay is utilized
   to carry internet traffic, the operator has full control of the
   underlay and IPv6 Extension Header chain length as well as the number
   of HBH options encoded [I-D.ietf-6man-eh-limits].

   In the global routing table scenario for Internet traffic there is no
   way to control the IPv6 Extended header chain length as well as the
   number of HBH options encoded.

   When IPv6 was first implemented on high-speed routers, HBH options
   were not yet well-understood and ASICs were not as capable as they
   are today.  So, early IPv6 implementations dispatched all packets
   that contain HBH options to their control plane.  Such implementation

Peng, et al.             Expires 20 August 2024                 [Page 7]
Internet-Draft           Processing HBH Opt Hdr            February 2024

   introduces a risk of a DoS attack on the control plane of the node,
   and a large flow of IPv6 packets could congest the control plane,
   causing other critical functions (including routing and network
   management) that are executed on the control plane to fail.  Rate
   limiting mechanisms will cause inconsistent packet drops and impact
   the normal IP forwarding of packets.

7.  Typical Processing

   To mitigate this DoS vulnerability, many operators have deployed
   Access Control Lists (ACLs) that discard all packets containing HBH
   Options [RFC9288].

   [RFC6564] shows the Reports from the field indicating that some IP
   routers deployed within the global Internet are configured either to
   ignore or to drop packets having a hop-by-hop header.  As stated in
   [RFC7872], many network operators perceive HBH Options to be a breach
   of the separation between the forwarding and control planes.
   Therefore, several network operators configured their nodes so as to
   discard all packets containing the HBH Options Extension Header,
   while others configured nodes to forward the packet but to ignore the
   HBH Options.  [RFC7045] also states that hop-by-hop options are not
   handled by many high-speed routers or are processed only on a control
   plane.  [I-D.vyncke-v6ops-james] shows that the HBH options header
   cannot reliably traverse the global Internet; only small headers with
   'skippable' options have some chances.

   Due to such behaviors observed and described in these specifications,
   [RFC8200] did not recommend new HBH Options, which limited the
   usability of HBH Options.

   Besides service providers' networks, other sectors such as industrial
   IoT networks are slowly replacing a dozen of semi-proprietary
   protocols in industrial automation into IP.  The proper processing of
   the HBH options header is also required

8.  Design Principles

   Here are some design principles for HBH Options Header and its
   carrying Options.

   *  The HBH Options Header should not become a possible DDoS Vector.
      Therefore, the control plane needs to be preserved from unwanted
      incoming traffic due to HBH header present in the packet.

Peng, et al.             Expires 20 August 2024                 [Page 8]
Internet-Draft           Processing HBH Opt Hdr            February 2024

   *  HBH Options need to be designed in a manner so that they don't
      reduce the probability of packet delivery, especially in length

   *  HBH processing implementations need to perform well at a
      reasonable cost without endangering the security of the router.

   *  The Router Alert Option needs not to impact the processing of
      other HBH Options that should be processed more quickly

   *  HBH Options can influence how a packet is forwarded.  However,
      with the exception of the Router Alert Option, an HBH Option must
      not cause control plane state to be created, modified or destroyed
      on the processing node.  As per [RFC6398], protocol developers
      SHOULD avoid future use of the Router Alert Option.

   *  The limits specified in [I-D.ietf-6man-eh-limits] need to be
      configurable to manage HBH processing so it doesn't become a

   *  MLDv2 [RFC3810] should not be forbidden.

9.  Migration Strategies

   In order to make the HBH Options Header usable and facilitate the
   ever-emerging new services to be deployed across multiple vendors'
   devices, the new specifications related to the processing of HBH
   Options Header need to be designed to allow a smooth migration from
   old to new behavior without disruption time.

10.  Security Considerations

   The security considerations can refer to
   [I-D.ietf-6man-hbh-processing], [RFC7045], [RFC7112], [RFC8200],
   [RFC4302], and [RFC4303].

11.  IANA Considerations

   This document does not include an IANA request.

12.  Acknowledgements

   The authors would like to acknowledge Ron Bonica, Fred Baker, Bob
   Hinden, Stefano Previdi, and Donald Eastlake, Gorry Fairhurst, Nick
   Hilliard, Tom Herbert, Haisheng Yu, Nalini Elkins, and Alexandre
   Petrescu for their valuable review and comments.

Peng, et al.             Expires 20 August 2024                 [Page 9]
Internet-Draft           Processing HBH Opt Hdr            February 2024

13.  References

13.1.  Normative References

              Herbert, T., "Limits on Sending and Processing IPv6
              Extension Headers", Work in Progress, Internet-Draft,
              draft-ietf-6man-eh-limits-12, 18 December 2023,

              Hinden, R. M. and G. Fairhurst, "IPv6 Hop-by-Hop Options
              Processing Procedures", Work in Progress, Internet-Draft,
              draft-ietf-6man-hbh-processing-12, 21 November 2023,

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,

   [RFC2711]  Partridge, C. and A. Jackson, "IPv6 Router Alert Option",
              RFC 2711, DOI 10.17487/RFC2711, October 1999,

   [RFC3810]  Vida, R., Ed. and L. Costa, Ed., "Multicast Listener
              Discovery Version 2 (MLDv2) for IPv6", RFC 3810,
              DOI 10.17487/RFC3810, June 2004,

   [RFC4302]  Kent, S., "IP Authentication Header", RFC 4302,
              DOI 10.17487/RFC4302, December 2005,

   [RFC4303]  Kent, S., "IP Encapsulating Security Payload (ESP)",
              RFC 4303, DOI 10.17487/RFC4303, December 2005,

   [RFC6564]  Krishnan, S., Woodyatt, J., Kline, E., Hoagland, J., and
              M. Bhatia, "A Uniform Format for IPv6 Extension Headers",
              RFC 6564, DOI 10.17487/RFC6564, April 2012,

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <>.

Peng, et al.             Expires 20 August 2024                [Page 10]
Internet-Draft           Processing HBH Opt Hdr            February 2024

   [RFC8200]  Deering, S. and R. Hinden, "Internet Protocol, Version 6
              (IPv6) Specification", STD 86, RFC 8200,
              DOI 10.17487/RFC8200, July 2017,

13.2.  Informative References

              Dong, J., Li, Z., Xie, C., Ma, C., and G. S. Mishra,
              "Carrying Virtual Transport Network (VTN) Information in
              IPv6 Extension Header", Work in Progress, Internet-Draft,
              draft-ietf-6man-enhanced-vpn-vtn-id-05, 6 July 2023,

              Thubert, P. and F. Yang, "IPv6 Options for DetNet", Work
              in Progress, Internet-Draft, draft-pthubert-detnet-ipv6-
              hbh-07, 22 February 2022,

              Vyncke, E., Léas, R., and J. Iurman, "Just Another
              Measurement of Extension header Survivability (JAMES)",
              Work in Progress, Internet-Draft, draft-vyncke-v6ops-
              james-03, 9 January 2023,

   [P4], "P4".

   [RFC6192]  Dugal, D., Pignataro, C., and R. Dunn, "Protecting the
              Router Control Plane", RFC 6192, DOI 10.17487/RFC6192,
              March 2011, <>.

   [RFC6398]  Le Faucheur, F., Ed., "IP Router Alert Considerations and
              Usage", BCP 168, RFC 6398, DOI 10.17487/RFC6398, October
              2011, <>.

   [RFC7045]  Carpenter, B. and S. Jiang, "Transmission and Processing
              of IPv6 Extension Headers", RFC 7045,
              DOI 10.17487/RFC7045, December 2013,

Peng, et al.             Expires 20 August 2024                [Page 11]
Internet-Draft           Processing HBH Opt Hdr            February 2024

   [RFC7112]  Gont, F., Manral, V., and R. Bonica, "Implications of
              Oversized IPv6 Header Chains", RFC 7112,
              DOI 10.17487/RFC7112, January 2014,

   [RFC7872]  Gont, F., Linkova, J., Chown, T., and W. Liu,
              "Observations on the Dropping of Packets with IPv6
              Extension Headers in the Real World", RFC 7872,
              DOI 10.17487/RFC7872, June 2016,

   [RFC8250]  Elkins, N., Hamilton, R., and M. Ackermann, "IPv6
              Performance and Diagnostic Metrics (PDM) Destination
              Option", RFC 8250, DOI 10.17487/RFC8250, September 2017,

   [RFC9098]  Gont, F., Hilliard, N., Doering, G., Kumari, W., Huston,
              G., and W. Liu, "Operational Implications of IPv6 Packets
              with Extension Headers", RFC 9098, DOI 10.17487/RFC9098,
              September 2021, <>.

   [RFC9268]  Hinden, R. and G. Fairhurst, "IPv6 Minimum Path MTU Hop-
              by-Hop Option", RFC 9268, DOI 10.17487/RFC9268, August
              2022, <>.

   [RFC9288]  Gont, F. and W. Liu, "Recommendations on the Filtering of
              IPv6 Packets Containing IPv6 Extension Headers at Transit
              Routers", RFC 9288, DOI 10.17487/RFC9288, August 2022,

   [RFC9343]  Fioccola, G., Zhou, T., Cociglio, M., Qin, F., and R.
              Pang, "IPv6 Application of the Alternate-Marking Method",
              RFC 9343, DOI 10.17487/RFC9343, December 2022,

   [RFC9486]  Bhandari, S., Ed. and F. Brockners, Ed., "IPv6 Options for
              In Situ Operations, Administration, and Maintenance
              (IOAM)", RFC 9486, DOI 10.17487/RFC9486, September 2023,

Authors' Addresses

   Shuping Peng
   Huawei Technologies

Peng, et al.             Expires 20 August 2024                [Page 12]
Internet-Draft           Processing HBH Opt Hdr            February 2024

   Zhenbin Li
   Huawei Technologies

   Chongfeng Xie
   China Telecom

   Zhuangzhuang Qin
   China Unicom

   Gyan Mishra
   Verizon Inc.
   United States of America

Peng, et al.             Expires 20 August 2024                [Page 13]