Network Working Group K. Majumdar
Internet Draft Upscale AI
Intended status: Standards Track L. Dunbar
Expires: February 24, 2027 Futurewei
V.Kasiviswanathan
NextHop AI
A. Ramchandra
Google
A. Choudhary
Cisco
August 24, 2026
Multi-segment SD-WAN via Cloud Backbone
draft-ietf-rtgwg-multisegment-sdwan-16
Abstract
This document describes a method for seamlessly
interconnecting geographically separated SD-WAN segments via
a Cloud Backbone without requiring Cloud Gateways (GWs) to
decrypt and re-encrypt traffic. By encapsulating IPsec-
encrypted payloads within GENEVE headers (RFC 8926), the
approach enables Cloud GWs to forward encrypted traffic
directly between distant Customer Premises Equipment (CPEs).
This reduces processing overhead, improves scalability, and
preserves the confidentiality of enterprise data while
ensuring secure and efficient multi-segment SD-WAN
connectivity.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet
Engineering Task Force (IETF), its areas, and its working
groups. Note that other groups may also distribute working
documents as Internet-Drafts.
xxx, et al. Expires February 24, 2027 [Page 1]
Internet-Draft Multi-segment SD-WAN
Internet-Drafts are draft documents valid for a maximum of
six months and may be updated, replaced, or obsoleted by
other documents at any time. It is inappropriate to use
Internet-Drafts as reference material or to cite them other
than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed
at http://www.ietf.org/shadow.html
This Internet-Draft will expire on February 21, 2027 .
Copyright Notice
Copyright (c) 2026 IETF Trust and the persons identified as
the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date
of publication of this document. Please review these
documents carefully, as they describe your rights and
restrictions with respect to this document. Code Components
extracted from this document must include Simplified BSD
License text as described in Section 4.e of the Trust Legal
Provisions and are provided without warranty as described in
the Simplified BSD License.
Table of Contents
1. Introduction..............................................3
2. Conventions and Definitions...............................5
3. Use Cases.................................................6
3.1. Multi-segment SD-WAN via a Single Cloud GW...........6
3.2. Multi-segment SD-WAN via Cloud Backbone..............8
3.3. Traffic Steering Challenges in Multi-Segment SD-WAN..9
4. Data Plane encoding for SD-WAN Transit...................10
4.1. Multi-Segment SD-WAN Option Class...................11
4.2. SD-WAN Tunnel Endpoint Sub-TLV......................12
4.3. SD-WAN Tunnel Originator Sub-TLV....................14
Dunbar, et al. Expires Dec 24, 2027 [Page 2]
Internet-Draft Multi-segment SD-WAN
4.4. Egress GW Sub-TLV...................................15
4.5. Permitted Regions Sub-TLV...........................16
4.6. Excluded Regions Sub-TLV............................18
4.7. HMAC Authentication Value Sub-TLV...................19
5. Packet Header Processing.................................21
6. Error Handling...........................................22
7. Control Plane considerations.............................23
7.1. Control Plane for CPEs..............................23
7.2. Control Plane between CPEs and Cloud GWs............24
8. Observability Consideration..............................24
9. Security Considerations..................................25
9.1. Threat Analysis.....................................25
9.2. HMAC-based Integrity and Authentication.............26
9.3. Additional Integrity and Authentication.............28
10. Manageability Considerations............................29
11. Applicability Considerations............................30
12. IANA Considerations.....................................31
13. References..............................................32
13.1. Normative References...............................32
13.2. Informative References.............................33
14. Acknowledgments.........................................35
Appendix A: Illustration of Packets through Cloud GWs.......35
A.1 Single Hop Cloud GW.....................................35
A.2 Multi-hop Transit GWs...................................38
1. Introduction
Enterprises are increasingly turning to Software-Defined Wide
Area Network (SD-WAN) to connect Customer Premises Equipment
(CPE)s with cloud services, as discussed in detail in
[Net2Cloud]. Each SD-WAN segment typically connects a CPE to
its nearest Cloud Gateway (GW). Some of this traffic
terminates at the cloud services and must be decrypted by the
Cloud GW. Other traffic is destined for remote CPEs located
in different geographic regions and only require forwarding
across a Cloud Backbone, without decryption.
Figures 1 and 2 in Sections 3.1 and 3.2 illustrate the CPEs,
Cloud GWs, Cloud Backbone, and the connectivity considered in
this document.
Multi-segment SD-WAN refers to the architecture in which two
or more SD-WAN segments are interconnected via a Cloud
Backbone. This model enables traffic that originates in one
Dunbar, et al. Expires Dec 24, 2027 [Page 3]
Internet-Draft Multi-segment SD-WAN
SD-WAN segment to reach a distant CPE through transit Cloud
GWs without decryption. It supports hybrid traffic handling:
local cloud-bound traffic is decrypted by the Cloud GW, while
CPE-to-CPE traffic is forwarded securely across the backbone.
Interconnecting these SD-WAN segments via a Cloud Backbone
provides several key benefits:
a) Seamless connectivity - Enterprises can integrate
geographically dispersed SD-WAN segments into a unified
network without complex manual configurations.
b) Scalability - The Cloud Backbone's elasticity accommodates
increased traffic demands without requiring extensive on-
premises infrastructure.
c) Simplified operations - Centralized orchestration
streamlines policy enforcement and network management
across all segments.
The challenges and motivations for this architecture are
further detailed in [Net2Cloud], which outlines issues
enterprises face when interconnecting branch sites with
dynamic workloads in third-party Cloud Datacenters (DC),
particularly when leveraging existing VPN infrastructure.
A key requirement for Cloud Backbone interconnecting
(stitching) multiple SD-WAN segments is the ability to forward
encrypted traffic across the Cloud Backbone without requiring
decryption at Cloud GWs. Since IPsec Security Associations
(SAs) are established end-to-end between CPEs, Cloud GWs
cannot access the payload for routing. Introducing an
additional IPsec tunnel layer between CPE and Cloud GW just
for routing purposes is inefficient-it adds processing
overhead, increases latency due to decryption and re-
encryption, and imposes scalability limits due to cloud
provider restrictions on IPsec capacity per GW instance.
This document defines a GENEVE-based method that avoids these
inefficiencies. SD-WAN CPEs encapsulate IPsec-encrypted
packets with GENEVE headers [RFC8926] that include Sub-TLVs to
signal when traffic should transit the Cloud Backbone without
decryption. This enables Cloud GWs to forward encrypted
Dunbar, et al. Expires Dec 24, 2027 [Page 4]
Internet-Draft Multi-segment SD-WAN
traffic efficiently to remote CPEs, without accessing the
payload. The result is secure, low-latency, and scalable
interconnection of geographically distributed SD-WAN segments
using the Cloud Backbone.
2. Conventions and Definitions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT
RECOMMENDED", "MAY", and "OPTIONAL" in this document are to
be interpreted as described in BCP 14 [RFC2119] [RFC8174]
when, and only when, they appear in all capitals, as shown
here.
The IP addresses used in this document are for illustrative
purposes only. IPv4 examples use addresses from the
documentation blocks defined in [RFC5737] (192.0.2.0/24 and
198.51.100.0/24). All mechanisms described apply equally to
IPv6, where equivalent examples would use addresses from the
documentation prefix defined in [RFC3849] (2001:db8::/32).
The following acronyms and terms are used in this document:
Cloud Backbone: The global, private network infrastructure
operated by a cloud provider that interconnects
its regions, zones, and points of presence.
Cloud DC: Off-Premises Data Center, managed by the third
party, that hosts applications, services, and
workload for different organizations or tenants.
CPE: Customer (Edge) Premises Equipment.
Ingress Cloud GW: The Cloud GW through which traffic enters
the Cloud Backbone.
Egress Cloud GW: The Cloud GW through which traffic exits the
Cloud Backbone toward the destination CPE.
OnPrem: On Premises data centers and branch offices.
RR Route Reflector.
Dunbar, et al. Expires Dec 24, 2027 [Page 5]
Internet-Draft Multi-segment SD-WAN
Region: A geographic or administrative area defined by the
Cloud Backbone provider and used for placement,
routing, or policy enforcement.
SA IPsec Security Association
SD-WAN Software-Defined Wide Area Network. An overlay
connectivity service that optimizes transport of
IP Packets over one or more Underlay Connectivity
Services and determining forwarding behavior by
applying Policies to them. [MEF-70.1]
Transit Node: A node within the Cloud Backbone through which
traffic may transit between the Ingress and
Egress Cloud GWs.
VPN Virtual Private Network.
Zone: A subdivision of a Region representing a distinct
deployment or failure domain within the Cloud
Backbone.
3. Use Cases
3.1. Multi-segment SD-WAN via a Single Cloud GW
Enterprise branches, such as CPE1 and CPE2 in Figure 1, with
established SD-WAN paths to a Cloud GW for accessing cloud
services can also use the Cloud GW to interconnect with one
another. For customers that have already chosen SD-WAN,
stitching SD-WAN segments through a Cloud Gateway provides a
way to extend policy enforcement and traffic control across
branches using their existing SD-WAN-to-cloud connectivity,
while avoiding the need to establish separate direct branch
to branch SD-WAN paths. This approach is beneficial for
several reasons:
Dunbar, et al. Expires Dec 24, 2027 [Page 6]
Internet-Draft Multi-segment SD-WAN
- The public Internet path from a branch to its nearby Cloud
GW is typically shorter and more controlled than a public
Internet path between branches, which can reduce exposure
to unpredictable performance, limited bandwidth, and
security risks.
- Centralized enforcement of enterprise security policies can
be enabled through cloud-hosted services. Traffic destined
to cloud-resident applications can be decrypted for full
inspection (e.g., firewall, threat detection), while CPE-
to-CPE traffic that remains IPsec-encrypted can still
benefit from header- or flow-based functions-such as DDoS
mitigation, rate limiting, anomaly detection, and SLA/usage
analytics-especially when the same CPE also sends traffic
terminating in the cloud.
- Cloud platforms often offer enhanced monitoring,
proprietary threat detection tools, and analytics services
that can inspect and respond to suspicious traffic crossing
segments.
Dunbar, et al. Expires Dec 24, 2027 [Page 7]
Internet-Draft Multi-segment SD-WAN
+-----------------+
| Cloud |
| +----+ +----+ |
+--------|Edge+--+ GW | |
Direct | +-----+----+ +-/\-+ |
Connect | | | / \ |
| | +---------/----\--+
| | / \SD-WAN Path CPE<->GW
PrivateLines / \
| +----------/-----+ IPsec Tunnel
+-------+ / | \
| / | \
++--/+ | +-\--+
|CPE1| +----+CPE2|
+----+ +----+
Figure 1 Multi-Segment SD-WAN stitching via a Cloud GW
Note: For clarity, each line in this figure represents
connectivity that may consist of multiple parallel paths.
Multiple paths are not shown to avoid excessive complexity in
the illustration.
3.2. Multi-segment SD-WAN via Cloud Backbone
For geographically distant enterprise branches that have
established SD-WAN paths to their respective Cloud GWs for
accessing cloud services, the Cloud Backbone provides an
efficient way to interconnect these branches, as shown in
Figure 2. As outlined in the Introduction section, this
approach enhances network integration, supports dynamic
scaling, and simplifies overall management, making it well-
suited for multi-segment SD-WAN deployments across different
regions.
Dunbar, et al. Expires Dec 24, 2027 [Page 8]
Internet-Draft Multi-segment SD-WAN
+---------------------------------------+
| Cloud Backbone |
+-------+-----+ +----+ +-----+ |
| +-----| Edge+--+ GW1|============| GW2 | |
Direct| | | +-----+ +|--|+ +--|--+ |
Connect| | | | | | |
| | +------------|--|----------------|------+
| | IPsec +-----+
Private Lines / \ |CPE10|
| +-------------/--+ \ +-----+
| / | \
| +----+ | +----+
+---------+CPE1| +--+CPE2|
+----+ +----+
Figure 2 Multi-Segment SD-WAN Stitching via Cloud Backbone
3.3. Traffic Steering Challenges in Multi-Segment SD-WAN
Traffic engineering mechanisms, such as SR Policy [RFC9256]
and RSVP-TE [RFC3209], can be used to steer traffic through
specific network nodes.
However, in typical SD-WAN deployments, CPE-to-CPE traffic is
carried as best-effort over the public Internet or other
shared transport. Forwarding in the underlay is destination-
based, and the on-premises CPEs cannot directly control the
specific path that packets take. This limits the ability to
enforce precise traffic engineering (TE) to reach destination
CPEs.
This lack of predictable routing makes traffic steering
between branch offices highly challenging. Unlike private
MPLS networks or provider-controlled backbones, SD-WAN cannot
inherently dictate the intermediate paths for branch-to-
branch traffic. As a result, policies intended to optimize
performance, enforce security, or ensure compliance can be
difficult to implement.
Dunbar, et al. Expires Dec 24, 2027 [Page 9]
Internet-Draft Multi-segment SD-WAN
To address this issue, this document describes a method where
Cloud GWs explicitly interconnect SD-WAN segments, ensuring
that branch-to-branch traffic is steered through the Cloud
Backbone rather than taking unpredictable Internet routes.
This approach provides greater control over traffic flows,
improving reliability, security, and policy enforcement.
Note: The mechanism described in this document does not alter
the forwarding behavior of the underlay network. Traffic from
the source CPE to the ingress Cloud GW and from the egress
Cloud GW to the destination CPE continues to follow normal
underlay forwarding. Since these are typically short hops,
the more useful traffic engineering (TE) occurs across the
longer-range Cloud Backbone. In this model, the overlay
steering defined here enables predictable selection of
ingress and egress Cloud GWs, while TE within the backbone is
offloaded to the Cloud Backbone provider.
4. Data Plane encoding for SD-WAN Transit
To enable Cloud GWs to distinguish between packets requiring
decryption for internal cloud services and transit packets
that should be forwarded to destination CPEs, proper packet
marking is essential. Many encapsulation methods, such as
VLAN Tags [IEEE802.1Q], IP-in-IP [RFC2003], GRE [RFC2784],
etc., can be used to steer traffic from a CPE to its nearest
(or chosen) Cloud GW. However, GENEVE encapsulation [RFC8926]
offers significant advantages, including flexible option Sub-
TLVs that can signal routing and policy preferences, such as
Permitted Regions, Excluded Regions, preferred egress Cloud
GWs, and other service specific requirements. In addition,
GENEVE Encapsulation [RFC8926] is widely supported by major
Cloud Service Providers, which allows Cloud GWs to
efficiently steer IPsec-encrypted packets between CPEs via
Cloud Backbone without decryption, reducing processing
overhead and improving performance while maintaining end-to-
end encryption.
Dunbar, et al. Expires Dec 24, 2027 [Page 10]
Internet-Draft Multi-segment SD-WAN
4.1. Multi-Segment SD-WAN Option Class
The Multi-Segment SD-WAN Option uses the GENEVE tunnel option
format defined in Section 3.5 of [RFC8926]. This document
uses the GENEVE Option Class value 0x0163, which has been
assigned by IANA to identify the Multi-Segment SD-WAN Option.
This enables Cloud GWs to interpret and process SD-WAN
transit packets efficiently without decrypting the
encapsulated SD-WAN traffic. Figure 3 reproduces the GENEVE
option header for context and illustrates how the Multi-
Segment SD-WAN Sub-TLVs are carried in the option data. The
GENEVE option header format and the semantics of its fields
are unchanged from [RFC8926]. The Sub-TLVs MAY appear in any
order, and other optional Sub-TLVs defined for this option
MAY also be included.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Multi-seg-SD-WAN Option Class |C| Type |R|R|R| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
~ Optional SD-WAN Tunnel Endpoint Sub-TLV ~
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
~ Optional SD-WAN Tunnel Originator Sub-TLV ~
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
~ Optional Egress GW Sub-TLV ~
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// //
// Optional Type Length Value objects (variable) //
// //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3 Multi Segment SD-WAN Option Class
- Multi-seg-SD-WAN Option Class: value 0x0163 (assigned by
IANA).
- C-bit: MUST be set to ensure that a receiving node drops
the packet if it does not recognize the option, as per
[RFC8926].
Dunbar, et al. Expires Dec 24, 2027 [Page 11]
Internet-Draft Multi-segment SD-WAN
- Type (7 bits): Specifies the multi-segment SD-WAN
forwarding model. Values are assigned from the "Multi-
Segment SD-WAN GENEVE Option Types" registry defined in
Section 12:
Type = 1: Single-hop transit SD-WAN
Type = 2: Multi-Hop transit SD-WAN with an explicitly
specified egress Cloud GW (via Egress GW Sub-TLV).
Type = 3: Multi-hop transit SD-WAN without an explicitly
specified egress Cloud GW.
- R-bits (3 bits): Reserved for future use. These bits MUST
be set to zero on transmission and ignored on receipt.
- Length (5 bits): Indicates the length of the option data,
containing the Sub-TLVs defined below, in 4-byte units,
excluding the 4-byte option header [RFC8926]. A value of
zero indicates that no Sub-TLVs are present. If the total
encoded length of the included Sub-TLVs is not a multiple
of 4 octets, zero-valued padding octets MUST be appended at
the end of the option data to achieve 4-octet alignment.
The padding is not part of any Sub-TLV. The option data is
limited to 124 octets. Therefore, the total length of all
included Sub-TLVs and any padding MUST NOT exceed 124
octets. An encapsulator MUST NOT include a Sub-TLV that
would cause this limit to be exceeded.
As specified in [RFC8926], the GENEVE Protocol Type is the
EtherType of the encapsulated PDU and MUST be set to 0x0800
for IPv4 or 0x86DD for IPv6. For ESP, the IPv4 Protocol or
IPv6 Next Header field is set to 50.
4.2. SD-WAN Tunnel Endpoint Sub-TLV
The SD-WAN Endpoint sub-TLV indicates the destination CPE,
which is the endpoint of the IPsec Tunnel between branch
CPEs. This Sub-TLV is used by the Cloud Backbone to determine
the optimal egress Cloud GW for forwarding the encrypted
traffic. The SD-WAN Endpoint Sub-TLV is OPTIONAL. When it is
absent, the destination CPE is determined from the
destination address of the encapsulated IP packet.
Dunbar, et al. Expires Dec 24, 2027 [Page 12]
Internet-Draft Multi-segment SD-WAN
For example, in an SD-WAN deployment where CPE1 establishes
an IPsec SA with CPE2 (as shown in Figure 1), this Sub-TLV
within the GENEVE header contains CPE2's IP address, ensuring
that encrypted traffic is correctly routed to the terminating
CPE of the IPsec tunnel while enabling the Cloud Backbone to
steer the packet to the most suitable egress Cloud GW.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|SD-WAN Endpoint| length | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SD-WAN Endpoint Addr Family | Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ (variable) +
~ ~
| SD-WAN Endpoint Address
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4 SD-WAN Endpoint Sub-TLV
- SD-WAN Endpoint (8 bits): Identifies the SD-WAN Tunnel
Endpoint Sub-TLV with a Type value of 1.
- Length (8 bits): Specifies the total length of the value
field in octets.
- Reserved (16 bits): Reserved for future use. These bits
MUST be set to zero on transmission and ignored on
receipt.
- SD-WAN Endpoint Addr Family (16 bits): Identifies the
address family of the SD-WAN Endpoint Address. Values
are taken from the IANA "Address Family Numbers"
registry [IANA-AF]. A value of 1 indicates an IPv4
address and a value of 2 indicates an IPv6 address. Other
Address Family values are not supported by this
specification.
- SD-WAN Endpoint Address: Contains the destination
endpoint address. The field is 4 octets for Address
Family 1 (IPv4) and 16 octets for Address Family 2
(IPv6). If the Address Family is unsupported or the Sub-
TLV Length is inconsistent with the address length
indicated by the Address Family, the packet MUST be
discarded as malformed.
Dunbar, et al. Expires Dec 24, 2027 [Page 13]
Internet-Draft Multi-segment SD-WAN
4.3. SD-WAN Tunnel Originator Sub-TLV
The SD-WAN Tunnel Originator Sub-TLV is an optional Sub-TLV
within the multi-seg-SD-WAN Option Class to indicate the
originating CPE of the IPsec Tunnel.
For example, in an SD-WAN deployment where CPE1 establishes
an IPsec SA with CPE2 (as shown in Figure 1), this Sub-TLV
within the GENEVE header carries CPE1's address, allowing
transit nodes and Cloud GWs to recognize the source of the
encrypted traffic.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|SDWAN Origin | Length | reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SD-WAN Org Addr Family | Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ (variable) +
~ ~
|SD-WAN Tunnel Originator Addr
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5 SD-WAN Tunnel Originator Sub-TLV
- SDWAN Origin (8 bits): Identifies the SDWAN Tunnel
Originator Sub-TLV with a Type value of 2.
- Length (8 bits): Specifies the total length of the value
field in octets, excluding the first 4 bytes, which
include the SD-WAN Origin (1 byte), Length (1 byte), and
Reserved (2 bytes) fields.
- Reserved (16 bits): Reserved for future. These bits MUST
be set to zero on transmission and ignored on receipt.
- SD-WAN Org Addr Family (16 bits): Identifies the address
family of the originator. Values are taken from the IANA
"Address Family Numbers" registry [IANA-AF]. A value of
1 indicates an IPv4 address and a value of 2 indicates
an IPv6 address. Other Address Family values are not
supported by this specification.
- SD-WAN Tunnel Originator Addr: Contains the originator
address. The field is 4 octets for Address Family 1
(IPv4) and 16 octets for Address Family 2 (IPv6). If the
Address Family is unsupported or the Sub-TLV Length is
Dunbar, et al. Expires Dec 24, 2027 [Page 14]
Internet-Draft Multi-segment SD-WAN
inconsistent with the address length indicated by the
Address Family, the packet MUST be discarded as
malformed.
This Sub-TLV allows Cloud GWs and transit nodes to identify
the packet's source, allowing them to apply source-specific
policies for forwarding. Such policies may include traffic
engineering rules specific to the originating CPE, security
enforcement tailored to the source, or path selection
constraints based on the origin. Any such policy, however,
MUST be applied only after the ingress Cloud GW authenticates
the sender and validates the Originator value against
authorized policy state. A false, unauthorized, or
unverifiable Originator value MUST NOT be used to obtain
preferred treatment and such packets MUST be discarded.
4.4. Egress GW Sub-TLV
In a multi-segment SD-WAN deployment over the Cloud Backbone,
the originating CPE can use the Egress GW Sub-TLV to
explicitly specify the egress Cloud GW responsible for
forwarding traffic to the destination CPE. This ensures
predictable routing behavior and enables policy-driven packet
delivery across the Cloud Backbone.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|SDWAN EgressGW | length | reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Egress GW Addr Family | Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ (variable) +
~ ~
| Egress GW Address
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6 SD-WAN Egress GW Sub-TLV
- SDWAN EgressGW (8 bits): Identifies Egress GW Sub-TLV
with a Type value of 3.
- Length (8 bits): Specifies the total length of the value
field in octets, excluding the first 4 bytes, which
include the SD-WAN EgressGW Sub-TLV Type (1 byte),
Length (1 byte), and Reserved (2 bytes) fields.
- Reserved (16 bits): Reserved for future. MUST set to 0.
MUST be ignored by recipients.
Dunbar, et al. Expires Dec 24, 2027 [Page 15]
Internet-Draft Multi-segment SD-WAN
- Egress GW Addr Family: Identifies the address family of
the Egress GW. A value of 1 indicates an IPv4 address
and a value of 2 indicates an IPv6 address.
- Egress GW Address: Contains the address of the egress
Cloud GW. The field is 4 octets for Address Family 1
(IPv4) and 16 octets for Address Family 2 (IPv6). If the
Address Family is unsupported or the Sub-TLV Length is
inconsistent with the address length indicated by the
Address Family, the packet MUST be discarded as
malformed.
The Egress GW Sub-TLV allows the originating CPE to specify
the Egress Cloud GW responsible for forwarding traffic to the
destination CPE. This Egress GW address can be either
preconfigured or dynamically discovered through a control
plane protocol exchange with the destination CPE. By
explicitly defining the egress GW, this Sub-TLV ensures
predictable traffic steering, reducing reliance on
destination-based routing and optimizing packet delivery
across the Cloud Backbone. The details of the control plane
protocol used for GW discovery are beyond the scope of this
document.
4.5. Permitted Regions Sub-TLV
Some enterprises may require that traffic across the Cloud
Backbone is strictly confined to a specific set of regions.
This Sub-TLV allows the ingress SD-WAN CPE to express such
restrictions as part of the encapsulation metadata.
Traffic MUST be discarded if the Ingress Gateway or Egress
Gateway belongs to a region not listed in this Sub-TLV. For
intermediate transit nodes or regions, enforcement is based
on the Cloud Backbone provider's knowledge of its topology
and region boundaries and the policy agreed with the
enterprise.
This restriction is commonly used to enforce regulatory,
security, or latency-based geographic constraints, where data
must remain confined to specified regions.
Format of the Permitted Regions Sub-TLV:
Dunbar, et al. Expires Dec 24, 2027 [Page 16]
Internet-Draft Multi-segment SD-WAN
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|PermittedReg | Length | Reserved (16 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Region Len | UTF-8 encoding of Region Name or ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Region Len | UTF-8 encoding of Region Name or ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7 Permitted Regions Sub-TLV
- PermittedReg (8 bits): Identifies the Permitted Regions
Sub-TLV with a Type value of 4.
- Length (8 bits): Total length of the Value field
(everything after the Type and Length fields), in
octets.
- Reserved (16 bits): Reserved for future use. MUST be set
to zero and ignored on receipt.
- Region Len (8 bits per region entry): Length of the UTF-
8 encoding of the Region Name or identifier, in octets.
- UTF-8 encoding of the Region Name (e.g., "us-west", "eu-
central"). Region Names are compared case-insensitively.
Multiple regions MAY be present, each starting with its own Region
Len field.
Processing notes:
- The receiving Cloud GW MUST apply the region checks
specified above. Enforcement of restrictions on internal
Cloud Backbone transit regions depends on the Cloud
Backbone provider's knowledge of its topology and region
boundaries and is outside the scope of this document.
- If any component of the path falls outside the listed
regions, the packet MUST be discarded.
- Region Names and their interpretation are based on prior
agreement between the enterprise and the Cloud Backbone
provider and may use commonly recognized or provider-
specific naming conventions. Region Names are compared
case-insensitively
Dunbar, et al. Expires Dec 24, 2027 [Page 17]
Internet-Draft Multi-segment SD-WAN
Note:
It is beyond the scope of this document to specify how the
Cloud Backbone enforces this restriction. Mechanisms for
identifying region boundaries, enforcing region-based
constraints, and generating alerts or alarm notifications
when traffic violates region restrictions are subject to
implementation decisions and based on prior agreement
between the Cloud Backbone provider and the enterprise.
4.6. Excluded Regions Sub-TLV
Excluded Regions Sub-TLV is an optional field used to specify
a list of Cloud Availability Regions or Zones that must be
avoided when forwarding packets across the Cloud Backbone.
This can be used for:
- Regulatory compliance, ensuring traffic does not traverse
non-compliant regions.
- Risk mitigation, preventing traffic from passing through
regions with known security, performance, or geopolitical
concerns.
Multiple region entries MAY be specified in a single Sub-TLV.
Each region is identified by a variable length UTF-8 encoded
name or numeric ID, preceded by a length field. This Sub-TLV
expresses explicit exclusions and supports both soft and hard
enforcement.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|ExcludedRegion | length |E| Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Region Len | UTF-8 encoding of Region Name or ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Region Len | UTF-8 encoding of Region Name or ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 8 Excluded Regions Sub-TLV
- ExcludedRegion (8 bits): identifies the Excluded Regions
Sub-TLV with a Type value of 5.
Dunbar, et al. Expires Dec 24, 2027 [Page 18]
Internet-Draft Multi-segment SD-WAN
- Length (8 bits): Total length of the Value field in
octets (everything after the first 2 bytes).
- E-bit (1 bit) - Exclusion severity indicator:
o 0: Soft exclusion - The listed region SHOULD be
avoided when feasible.
o 1: Hard exclusion - The listed region MUST NOT be
traversed. If no path satisfying the exclusion is
available, the packet MUST be discarded, and an
alert or alarm MAY be generated.
- Reserved (15 bits): MUST be set to zero on transmission
and ignored on receipt.
- Region Len (8 bits per region entry): Length of the UTF-
8 encoding of the Region Name or identifier, in octets.
- UTF-8 encoding of the Region Name (e.g., "us-west", "eu-
central"). Region Names, based on prior agreement
between the enterprise and the Cloud Backbone provider,
are compared case-insensitively.
Multiple region entries MAY be listed, each beginning with a
Region Len byte.
Processing Notes:
The E-bit determines how strictly the exclusions are
enforced. A value of 1 (hard exclusion) requires that the
listed region MUST NOT be traversed. If no compliant path
is available, the packet MUST be discarded and an alert or
alarm MAY be generated. A value of 0 (soft exclusion)
indicates that the Cloud Backbone SHOULD avoid the listed
region when feasible. If avoidance is not feasible, the
packet MAY traverse the listed region and no alert or alarm
is required. The meaning and granularity of region
identifiers MUST be agreed upon between the enterprise and
the Cloud Backbone provider. It is beyond the scope of this
document to define how enforcement or alerting is
implemented. These are subject to operator policies and
implementation specifics.
4.7. HMAC Authentication Value Sub-TLV
To support integrity protection and authentication of the
Multi-Segment SD-WAN GENEVE metadata, this document defines
Dunbar, et al. Expires Dec 24, 2027 [Page 19]
Internet-Draft Multi-segment SD-WAN
the HMAC (Hash-based Message Authentication Code)
Authentication Value (HMAC-Auth-Val) Sub-TLV. The Sub-TLV
carries the HMAC authentication value computed over the
GENEVE header, including the Multi-Segment SD-WAN Option
Class, allowing a receiving Cloud GW to verify that the
steering metadata has not been modified in transit.
The selection of the HMAC algorithm, authentication value
length, key establishment, and verification procedures are
described in Section 9.2.
The HMAC Authentication Value Sub-TLV has the following
format:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| HMAC-Auth-Val | length | reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ ~
| HMAC Authentication Value |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 9 Multi Segment SD-WAN HMAC Sub-TLV
- HMAC-Auth-Val (8 bits): HMAC Authentication Value Sub-
TLV Type = 6 (Assigned by this document).
- Length (8 bits): Total length of the value field, which
is the length of the HMAC Authentication Value in bytes
plus 2 reserved bytes.
- Reserved (16 bits): Reserved for future use. These bits
MUST be set to zero on transmission and ignored on
receipt.
- HMAC Authentication Value (variable): Contains the HMAC
computed over the GENEVE header, including the Multi-
Segment SD-WAN Option Class and all included Sub-TLVs,
excluding the HMAC-Auth-Val Sub-TLV itself (i.e., its
Type, Length, Reserved, and Authentication Value
fields). The HMAC algorithm, keying method, and
authentication value length are specified in Section
9.2.
Dunbar, et al. Expires Dec 24, 2027 [Page 20]
Internet-Draft Multi-segment SD-WAN
5. Packet Header Processing
The procedures described in this section apply only to
packets that carry the SD-WAN Option Class in the GENEVE
header. Packets without this option are processed using
default forwarding behavior.
As illustrated in Figure 1, when a Cloud GW receives a
GENEVE-encapsulated packet (i.e., Dst Port = 6081 (GENEVE);
MultiSeg-SDWAN Option Class; encapsulated IPv4 Protocol or
IPv6 Next Header = 50 (ESP)), it processes the packet as
follows:
Processing at the Ingress Cloud GW:
- Authenticate the packet or its originating CPE, and
validate the GENEVE metadata, using the authentication
mechanism configured for the deployment.
- Process the Multi-Segment SD-WAN Option Type:
o For Type = 1, the Cloud GW performs the applicable
policy, security, and analytics processing and
forwards the packet toward the destination CPE. An
Egress GW Sub-TLV MUST NOT be present.
o For Type = 2, an Egress GW Sub-TLV MUST be present.
The Cloud Backbone uses the Sub-TLV to identify the
egress Cloud GW.
o For Type = 3, an Egress GW Sub-TLV MUST NOT be
present. The Cloud Backbone determines the egress
Cloud GW based on the destination CPE address.
o A packet containing a Type value inconsistent with
the presence or absence of the Egress GW Sub-TLV
MUST be discarded as malformed.
- For Type = 1, the Cloud GW forwards the packet toward
the destination CPE after performing the applicable
policy, security, and analytics processing. For Type =
2 or Type = 3, update the outer IP header for
forwarding across the Cloud Backbone by setting the
source address to the ingress Cloud GW and the
destination address to the egress Cloud GW identified
by the Egress GW Sub-TLV (Type = 2) or selected by the
Cloud Backbone (Type = 3). The ingress Cloud GW MUST
Dunbar, et al. Expires Dec 24, 2027 [Page 21]
Internet-Draft Multi-segment SD-WAN
update the IPv4 header checksum, when applicable, and
recompute the UDP checksum as required by [RFC8926].
- Forward the packet to the egress Cloud GW.
To prevent unauthorized access, Cloud GW MUST drop any
packets containing unrecognized source addresses or invalid
values in the GENEVE Sub-TLVs, ensuring that only
registered entities can utilize Cloud services.
Processing at the Egress Cloud GW:
- Decapsulate the GENEVE header to extract the IPsec-
encrypted payload.
- Validate that the SD-WAN Tunnel Endpoint Sub-TLV
corresponds to a registered destination CPE.
- Ensure the source Cloud GW is an authorized forwarding
node to prevent unauthorized traffic injection.
- Forward the IPsec-encrypted payload to the destination
CPE, preserving the end-to-end encryption.
- Drop any packet that lacks a valid destination CPE or
originates from an untrusted source.
By enforcing these processing steps at both the ingress and
egress Cloud GWs, the system ensures secure, efficient, and
policy-compliant forwarding of SD-WAN traffic across the
Cloud Backbone.
6. Error Handling
To ensure secure and efficient traffic forwarding through the
Cloud Backbone, Cloud GW MUST enforce the following error
handling measures:
- Drop packets with unregistered or invalid
source/destination addresses to prevent unauthorized
access.
- Validate the SD-WAN Endpoint Sub-TLV, when present, and
drop packets if the destination CPE is unauthorized,
unreachable, or mismatched.
Dunbar, et al. Expires Dec 24, 2027 [Page 22]
Internet-Draft Multi-segment SD-WAN
- Discard packets with incorrect GENEVE headers, invalid
Sub-TLV formats, or authentication failures. The
receiver MUST verify that each Sub-TLV fits within the
remaining option data and that the total encoded length
of all Sub-TLVs, together with any trailing padding,
does not exceed the length of the enclosing Multi-
Segment SD-WAN Option. A packet that fails either check
MUST be discarded as malformed. An unrecognized Sub-TLV
Type MUST be skipped using its Length field, and
processing MUST continue with the next Sub-TLV.
- Reject misrouted packets if the Cloud Backbone cannot
determine an optimal egress Cloud GW or if the
specified egress GW is unreachable.
- Enforce rate limits on excessive traffic from a single
source to prevent congestion and abuse.
- Verify compliance with region restrictions specified by
the Permitted Regions and Excluded Regions Sub-TLVs.
Service subscription validation, rate limiting, and other
provider-specific admission and traffic-management policies
are deployment-specific and are outside the scope of this
document.
By implementing these error handling mechanisms, Cloud GWs
ensure network stability, security, and efficient resource
utilization while preventing misconfigurations and forwarding
errors.
7. Control Plane considerations
7.1. Control Plane for CPEs
The control plane enables SD-WAN CPEs to discover their
network attributes, establish connectivity, and exchange
routing information. In an SD-WAN deployment, on-premises
CPEs and virtual CPEs (vCPEs) in Cloud DCs may be managed
under a common iBGP administrative domain, facilitating route
propagation and policy enforcement.
Mechanisms such as BGP-based SD-WAN Edge Discovery
[SD-WAN-Edge-Discovery] allow CPEs to dynamically discover
each other's properties, improving automation and reducing
manual configurations. Additionally, IPsec SA parameters
Dunbar, et al. Expires Dec 24, 2027 [Page 23]
Internet-Draft Multi-segment SD-WAN
between CPEs can be exchanged through the iBGP control plane
using a RR to simplify security policy management.
The IPsec SAs themselves are established separately, for
example via IKEv2 or out-of-band management systems.
The iBGP sessions among CPEs described in this section are a
logical control plane relationship among enterprise managed
nodes, using the enterprise's own management/control
connectivity. They are independent of the eBGP sessions
between a CPE and its attached Cloud GW described in Section
7.2.
7.2. Control Plane between CPEs and Cloud GWs
This section describes the control plane relationship between
a CPE and its attached Cloud GW, which is distinct from the
iBGP based control plane among CPEs described in Section 7.1.
There are typically eBGP sessions between a CPE and a Cloud
GW for exchanging routing information related to services
that terminate within the cloud. This allows the CPE to learn
routes to cloud-hosted resources and enables the Cloud GW to
learn routes to the CPE's on-premises networks. This control-
plane relationship is separate from the CPE-to-CPE encrypted
traffic that transits the Cloud Backbone, which remains end-
to-end encrypted and is not decrypted at the Cloud GWs.
When the connection between a CPE and a Cloud GW traverses a
public or otherwise untrusted network, an IPsec tunnel may
also be established to secure that traffic. In such cases,
the IPsec Security Association (SA) parameters between the
CPE and its corresponding Cloud GW are established out-of-
band (e.g., via management or automation systems) or
negotiated dynamically using IKEv2.
Control plane mechanisms MUST ensure that Cloud GWs can
identify and authenticate SD-WAN CPEs, validate SD-WAN
metadata, and apply appropriate routing policies based on
dynamic network conditions. This ensures that route exchanges
are trustworthy, policy-compliant, and adaptive to changing
operational requirements.
8. Observability Consideration
Observability considerations encompass monitoring, analysis,
and reporting mechanisms to gain insights into the behavior
Dunbar, et al. Expires Dec 24, 2027 [Page 24]
Internet-Draft Multi-segment SD-WAN
and performance of the multi-segment SD-WAN infrastructure.
Key observability aspects include:
- Performance Metrics:
Monitor and collect performance metrics related to link
utilization, latency, and packet loss across the SD-WAN
segments and Cloud DC backbone. This data provides insights
into the overall health and efficiency of the network. IP
Flow Information Export (IPFIX) [RFC7011] is one of the
standardized methods to expose traffic flow over the
network.
- Global Network Topology Visualization:
Utilize visualization tools to depict the global network
topology, showcasing the interconnections and traffic flows
between different SD-WAN segments and Cloud DCs.
- Control Plane Monitoring:
Monitor the control plane for both CPEs and the
communication between CPEs and Cloud GWs. This includes
tracking route discovery, path selection, and any changes
in network state to ensure proper functioning of the SD-WAN
control plane.
- Security Event Logging:
The security event logging is to capture and analyze
security-related events, including threat detection,
authentication failures, and any unauthorized access
attempts. Syslog [RFC5424] is a valuable tool for security
monitoring and auditing.
These considerations contribute to the overall success of the
multi-segment SD-WAN deployment connecting edge devices via a
Cloud DC backbone.
9. Security Considerations
9.1. Threat Analysis
The GENEVE header used for steering is not encrypted, making
it susceptible to on-path attacks between CPEs and Cloud GWs.
Key risks include:
a) Eavesdropping: Attackers can learn branch and Cloud GW
locations, though payload remains protected by IPsec.
Dunbar, et al. Expires Dec 24, 2027 [Page 25]
Internet-Draft Multi-segment SD-WAN
b) Header Manipulation: Altered Sub-TLVs may cause misrouting
or packet drops.
c) Bandwidth Theft: A malicious or misconfigured CPE could
spoof SD-WAN metadata to use Cloud Backbone resources
without authorization.
Mitigation the above risks requires authenticating and
validating SD-WAN metadata to ensure it originates from
authorized CPEs.
9.2. HMAC-based Integrity and Authentication
The HMAC-Auth-Val Sub-TLV defined in Section 4.7 provides
integrity protection and source authentication for the Multi-
Segment SD-WAN steering metadata carried in the GENEVE
header. It allows Cloud GW to verify that the steering
metadata originated from an authorized SD-WAN CPE and has not
been modified while in transit.
HMAC is a widely deployed cryptographic mechanism that
provides message integrity and data origin authentication by
combining a shared secret key with a cryptographic hash
function. Upon receipt of a packet, the receiving Cloud GW
computes the HMAC over the received GENEVE header using the
shared key and compares the result with the authentication
value carried in the HMAC-Auth-Val Sub-TLV. A successful
comparison confirms that the protected metadata has not been
altered and was generated by an entity possessing the shared
key.
This mechanism authenticates the GENEVE metadata exchanged
between SD-WAN CPEs and the Cloud GWs. The HMAC key is a
dedicated symmetric key shared between the CPE and its
attached Cloud GW. The key MAY be provisioned by the SD-WAN
controller or another secure management mechanism. When the
key is distributed over an existing IPsec-protected CPE-to-GW
channel, no additional IKEv2 exchange is required solely for
protecting the GENEVE metadata.
The HMAC authentication mechanism defined in this document
provides integrity and source authentication of the GENEVE
steering metadata but does not by itself provide replay
protection. Replay protection, when required by a deployment,
is provided by mechanisms outside the scope of this document.
Dunbar, et al. Expires Dec 24, 2027 [Page 26]
Internet-Draft Multi-segment SD-WAN
The anti-replay mechanism of the encapsulated IPsec ESP SA
protects the ESP traffic according to the applicable IPsec
processing but does not constitute replay protection for the
GENEVE metadata defined in this document.
Unless otherwise configured, HMAC-SHA-256 [RFC4868] is the
default algorithm for computing the authentication value.
Other HMAC algorithms MAY be used when explicitly configured
by the SD-WAN controller for both the CPE and its attached
Cloud GW, together with the corresponding keying parameters.
For HMAC-SHA-256, a 16-octet (128-bit) truncated
authentication value, consistent with [RFC4868], is
RECOMMENDED. The Multi-Segment SD-WAN GENEVE Option has a
maximum option-data length of 124 octets, which MUST
accommodate all included Sub-TLVs. The HMAC protects the
GENEVE steering metadata exchanged between a CPE and its
attached Cloud GW, while the encapsulated payload is
independently protected by IPsec. Deployments in which the
CPE-to-Cloud-GW path is trusted or otherwise protected MAY
configure a shorter authentication value, or MAY omit the
HMAC Authentication Value Sub-TLV. Deployments using shorter
authentication values MUST consider the resulting reduction
in forgery resistance.
The HMAC defined in this section provides the following
security properties:
- Data Integrity: Detects unauthorized modification of
the Multi-Segment SD-WAN GENEVE metadata.
- Source Authentication: Allows Cloud GWs to verify that
the steering metadata originated from an authorized
SD-WAN CPE possessing the shared authentication key.
- Low Overhead: Supports truncated authentication values
to minimize packet overhead while providing adequate
protection for steering metadata.
- Algorithm Agility: Supports different HMAC algorithms
through controller configuration.
- Operational Simplicity: Reuses existing IPsec trust
relationships and key management, avoiding additional
Security Associations between Cloud GWs and CPEs.
Dunbar, et al. Expires Dec 24, 2027 [Page 27]
Internet-Draft Multi-segment SD-WAN
Although truncating the HMAC authentication value reduces the
security margin against forgery compared to the full HMAC
output, the HMAC protects only the GENEVE steering metadata,
while the payload remains independently protected by IPsec.
An attacker without the shared secret key may attempt to
forge an authentication value by guessing; the probability of
successful forgery increases as the authentication value is
shortened. Deployments using truncated authentication values
MUST therefore select a length appropriate to their threat
model and operational environment.
9.3. Additional Integrity and Authentication
Some deployments may require stronger or more comprehensive
integrity protection than a truncated HMAC, such as when
mandated by security policy, regulatory compliance, or risk
management practices. In these cases, an additional integrity
layer can be applied using ESP with NULL encryption (ESP-
NULL) [RFC2410] [RFC6071] or Authentication Header (AH)
[RFC4302] on top of the existing IPsec encryption between
CPEs. Consistent with [RFC8221], ESP-NULL is preferred over
AH for authentication-only protection because of its
compatibility with NAT traversal.
AH and ESP-NULL provide cryptographic integrity for the
entire IP packet, not just the GENEVE metadata. All
approaches (including the HMAC) require cryptographic keys.
The operational difference is that AH/ESP-NULL require
dedicated IPsec SAs and IKE state between each Cloud GW and
CPE, increasing per-peer state and processing. By contrast,
the HMAC Sub-TLV (Type = 6, defined in this document) can use
controller-distributed symmetric keys (e.g., per-tenant or
per-CPE) without establishing additional IPsec SAs between
Cloud GWs and CPEs.
NAT Considerations: AH is not compatible with NAT traversal
because it authenticates the outer IP header, and any address
change will cause verification to fail. ESP-NULL avoids this
issue but still incurs additional per-packet processing.
When AH or ESP-NULL is used, the required IPsec SAs and
associated keying material SHOULD be established using IKEv2
[RFC7296]. Deployments MAY use manual or controller based key
provisioning when supported by the operational environment,
but such provisioning MUST provide equivalent authentication,
Dunbar, et al. Expires Dec 24, 2027 [Page 28]
Internet-Draft Multi-segment SD-WAN
authorization, replay protection, key freshness, and key
rollover properties.
10. Manageability Considerations
In multi-segment SD-WAN deployments where the Cloud GW and
CPEs belong to different administrative domains,
manageability must address the challenges of secure,
interoperable, and policy-compliant operation across
organizational boundaries, consistent with the service
framework defined in MEF 70.1 [MEF-70.1]. Key considerations
include:
- Cross-Domain Authentication and Authorization:
Ensure that CPEs connecting to the Cloud GW are
authenticated using mutually agreed methods, and that
authorization policies are enforced to prevent
unauthorized use of Cloud Backbone resources.
- Metadata Validation and Policy Enforcement:
Cloud GWs MUST validate SD-WAN metadata (e.g., GENEVE
Sub-TLVs) against the registered information for each
CPE. This prevents spoofing, misrouting, and cross-
tenant traffic leakage.
- Operational Coordination and Fault Handling:
Define inter-organization procedures for troubleshooting
and incident response. This should include point-of-
contact directories, escalation processes, and shared
logging formats for event correlation.
- Coordination of Configuration Changes:
Coordinate configuration changes-such as policy updates,
region restrictions, or authentication parameters-so
that both the Cloud GW and CPEs apply them consistently,
avoiding mismatches that disrupt traffic.
- Policy Automation Using I2NSF Principles ([RFC8192]):
Where feasible, leverage I2NSF concepts to automate
policy configuration, exchange, and enforcement between
domains, reducing manual coordination and improving
operational consistency.
Dunbar, et al. Expires Dec 24, 2027 [Page 29]
Internet-Draft Multi-segment SD-WAN
11. Applicability Considerations
The GENEVE encapsulation defined in this document is used to
carry Multi-Segment SD-WAN steering metadata between a CPE
and its attached Cloud GW and across the Cloud Backbone. The
CPE-to-Cloud-GW path may traverse the public Internet or
other shared underlay networks. In such cases, the public
Internet serves only as the underlay for reaching the Cloud
GW; Internet transit nodes do not interpret or process the
Multi-Segment SD-WAN GENEVE metadata. Traffic steering based
on this metadata is performed by the Cloud GWs across the
controlled Cloud Backbone.
When GENEVE is carried over the general Internet,
implementations MUST follow the UDP usage and congestion
considerations specified in Section 4.1 of [RFC8926]. GENEVE
traffic sent over the general Internet MUST either carry
congestion-controlled traffic or use an appropriate tunnel-
level congestion-control mechanism. Within a controlled Cloud
Backbone, traffic engineering, capacity management, and
congestion handling are provided by the Cloud Backbone
operator.
The encapsulator MUST account for the maximum encapsulation
overhead that it can add, including the outer IP header, UDP
header, GENEVE header, and Multi-Segment SD-WAN option data.
As specified in Section 4.4.1 of [RFC8926], the maximum
possible GENEVE option length MUST be included when
calculating the tunnel MTU. As specified in Section 4.1 of
this document, the Multi-Segment SD-WAN Option data,
including all Sub-TLVs and any trailing padding, is limited
to 124 octets. An encapsulator MUST NOT include a set of Sub-
TLVs whose total encoded length, including any trailing
padding, exceeds this limit.
An encapsulator MUST NOT emit an encapsulated packet that
exceeds the effective tunnel MTU. If an inner packet is too
large to be encapsulated without exceeding the effective
tunnel MTU, the encapsulator MUST drop the packet unless IPv4
fragmentation is permitted and performed. For IPv4, if
fragmentation is not permitted, the encapsulator SHOULD
return an ICMP Destination Unreachable, Fragmentation Needed
message toward the source when possible. For IPv6, routers do
not fragment packets; therefore, the encapsulator SHOULD
return an ICMPv6 Packet Too Big message toward the source
when possible.
Dunbar, et al. Expires Dec 24, 2027 [Page 30]
Internet-Draft Multi-segment SD-WAN
Implementations SHOULD use Path MTU Discovery (PMTUD) and
SHOULD avoid fragmentation where possible. Upon receiving a
valid ICMP Destination Unreachable, Fragmentation Needed
message for IPv4, or an ICMPv6 Packet Too Big message for
IPv6, the encapsulator SHOULD reduce the effective tunnel MTU
as appropriate.
MTU-related ICMP or ICMPv6 messages generated on the Cloud-
GW-to-Cloud-GW path are delivered to the ingress Cloud GW
rather than directly to the originating CPE. Deployments
SHOULD therefore configure the effective tunnel MTU to
account for the complete encapsulation overhead and the MTU
of the Cloud Backbone path.
12. IANA Considerations
IANA has assigned the following GENEVE Option Class from the
First Come First Served range of the "Geneve Option Class"
registry:
Option
Class Description Assignee/Contact Reference
------ ------------------- ------------- -----------
0x0163 Multi Segment SD-WAN IETF [this document]
IANA is requested to create a new registry in the "Network
Virtualization Overlays (NVO3)" registry group for Sub-TLV
types used specifically by the Multi-Segment SD-WAN GENEVE
Option Class (0x0163):
Registry Name: Multi Segment SD-WAN Sub-TLVs
Applicable Option Class: 0x0163 (Multi-Segment SD-WAN)
Registration Procedure: IETF Review
Change Controller: IETF
Reference: [this document]
Sub-TLV Type Description Reference
------------ ---------------------- ---------------
0 Reserved
1 SD-WAN Endpoint [Section 4.2]
2 SD-WAN Originator [Section 4.3]
3 SD-WAN Egress GW [Section 4.4]
4 Permitted Regions [Section 4.5]
5 Excluded Regions [Section 4.6]
6 Multi SD-WAN-HMAC [Section 4.7]
Dunbar, et al. Expires Dec 24, 2027 [Page 31]
Internet-Draft Multi-segment SD-WAN
7-254 Unassigned
255 Reserved
IANA is requested to create a new registry in the "Network
Virtualization Overlays (NVO3)" registry group for the 7-bit
Type field of the Multi-Segment SD-WAN GENEVE Option Class
(0x0163).
Registry Name: Multi-Segment SD-WAN GENEVE Option Types
Applicable Option Class: 0x0163 (Multi-Segment SD-WAN)
Registration Procedure: IETF Review
Change Controller: IETF
Reference: [this document]
Type Description Reference
---- ------------ --------------
0 Reserved [Section 4.1]
1 Single-Hop Transit SD-WAN [Section 4.1]
2 Multi-Hop Transit SD-WAN
with Explicit Egress Cloud GW [Section 4.1]
3 Multi-Hop Transit SD-WAN
without Explicit Egress Cloud GW [Section 4.1]
4-127 Unassigned
The values in this registry are applicable only to the
Multi-Segment SD-WAN GENEVE Option Class (0x0163).
13. References
13.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC4302] Kent, S., "IP Authentication Header", RFC 4302, DOI
10.17487/RFC4302, December 2005.
[RFC4868] Kelly, S. and S. Frankel, "Using HMAC-SHA-256,
HMAC-SHA-384, and HMAC-SHA-512 with IPsec", RFC
4868, May 2007.
Dunbar, et al. Expires Dec 24, 2027 [Page 32]
Internet-Draft Multi-segment SD-WAN
[RFC7296] Kaufman, C., Hoffman, P., Nir, Y., Eronen, P., and
T. Kivinen, "Internet Key Exchange Protocol Version
2 (IKEv2)", RFC 7296, October 2014.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in
RFC 2119 Key Words", BCP 14, RFC 8174, DOI
10.17487/RFC8174, May 2017, <https://www.rfc-
editor.org/info/rfc8174>.
[RFC8221] Migault, D. and T. Guggemos, "Cryptographic
Algorithm Implementation Requirements and Usage
Guidance for Encapsulating Security Payload (ESP)
and Authentication Header (AH)", RFC 8221, DOI
10.17487/RFC8221, October 2017.
[RFC8926] J. Gross, et al., "GENEVE: Generic Network
Virtualization Encapsulation", RFC8926, Nov 2020.
[IANA-AF] IANA, "Address Family Numbers", Internet Assigned
Numbers Authority,
https://www.iana.org/assignments/address-family-
numbers/
13.2. Informative References
[RFC2003] Perkins, C., "IP Encapsulation within IP", RFC
2003, October 1996.
[RFC2410] R. Glenn and S. Kent, "The NULL encryption
Algorithm and Its Use with IPsec", RFC2410, Nov.
1998.
[RFC2784] D. Farinacci, T. Li, S. Hanks, D. Meyer, and P.
Traina, "Generic Routing Encapsulation (GRE)",
RFC2784, March 2000.
[RFC3209] Awduche, D., Berger, L., Gan, D., Li, T.,
Srinivasan, V., and G. Swallow, "RSVP-TE:
Extensions to RSVP for LSP Tunnels", RFC 3209, DOI
10.17487/RFC3209, December 2001.
Dunbar, et al. Expires Dec 24, 2027 [Page 33]
Internet-Draft Multi-segment SD-WAN
[RFC3849] S. Huston, M. Thomson, S. Kent, and J. Seo, "IPv6
Address Prefix Reserved for Documentation",
RFC3849, July 2004.
[RFC5424] R. Gerhards, "The Syslog Protocol", RFC5424, March
2009.
[RFC5737] J. Arkko, M. Cotton, and L. Vegoda, "IPv4 Address
Blocks Reserved for Documentation", RFC5737,
January 2010.
[RFC6071] S. Frankel and S. Krishnan, "IP Security (IPsec)
and Internet Key Exchange (IKE) Document Roadmap",
Feb. 2011.
[RFC7011] B. Claise, B. Trammell, and P. Aitken,
"Specification of the IP Flow Information Export
(IPFIX) Protocol for the Exchange of Flow
Information", RFC7011, Sept 2013.
[RFC8192] S. Hares, et al., "Interface to Network Security
Functions (I2NSF) Problem Statement and Use Cases",
July 2017
[RFC9256] Filsfils, C., Talaulikar, K., Voyer, D., Bogdanov,
A., and P. Mattes, "Segment Routing Policy
Architecture", RFC 9256, DOI 10.17487/RFC9256, July
2022.
[MEF-70.1] MEF 70.1 SD-WAN Service Attributes and Service
Framework. Nov. 2021.
[Net2Cloud] L. Dunbar and A. Malis, "Dynamic Networks to
Hybrid Cloud DCs Problem Statement", draft-ietf-
rtgwg-net2cloud-problem-statement-43, Jan, 2025.
[SD-WAN-Edge-Discovery] L. Dunbar, et al., "BGP UPDATE for
SD-WAN Edge Discovery", draft-ietf-idr-sdwan-edge-
discovery-29, July. 2026.
Dunbar, et al. Expires Dec 24, 2027 [Page 34]
Internet-Draft Multi-segment SD-WAN
[IEEE802.1Q] IEEE, "IEEE Standard for Local and Metropolitan
Area Networks--Bridges and Bridged Networks", IEEE
Std 802.1Q.
14. Acknowledgments
The authors would like to thank Adrian Farrel, Joel Halpern,
Donald Eastlake, Stephen Farrell, Ajeet Gill, Gunter Van De
Velde, Mike Bishop, Roman Danyliw, for their extensive review
and suggestions.
This document was prepared using 2-Word-v2.0.template.dot.
Appendix A: Illustration of Packets through Cloud GWs
This appendix illustrates the encapsulation of traffic
between SD-WAN CPEs as it traverses one or more Cloud GWs.
The Cloud GWs act as intermediate forwarding points that
stitch together the SD-WAN segments based on the Multi-
Segment SD-WAN metadata; they do not terminate the enterprise
traffic or the end-to-end IPsec protection between the
communicating CPEs.
A.1 Single Hop Cloud GW
For the single-hop Cloud GW scenario shown in Figure 1,
assume the following:
- There is a bidirectional IPsec tunnel between CPE1 and
Cloud GW; with IPsec SA1 for the traffic from the CPE1
to the Cloud-GW; and IPsec SA2 for the traffic from
the Cloud-GW to the CPE1. This tunnel can carry CPE1
traffic to and from cloud services. The existing
IPsec-protected channel can also be used to securely
exchange configuration, control, and keying
information, such as the HMAC key used to protect the
GENEVE metadata. This tunnel is separate from the end-
to-end CPE1-to-CPE2 IPsec tunnel described below.
- There is a bidirectional IPsec tunnel between CPE2 and
Cloud GW; with IPsec SA3 for the traffic from the CPE2
to the Cloud-GW; and IPsec SA4 for the traffic from
the Cloud-GW to the CPE2. This tunnel can similarly
Dunbar, et al. Expires Dec 24, 2027 [Page 35]
Internet-Draft Multi-segment SD-WAN
carry CPE2 traffic to and from cloud services and can
also be used to securely exchange configuration,
control, and keying information. It is separate from
the end-to-end CPE1-to-CPE2 IPsec tunnel.
- All the CPEs are under a common administrative control
and participate in a common iBGP control plane, with
routing information exchanged through a Route
Reflector (RR). The CPEs can exchange information
about their corresponding Cloud GW addresses through
the control plane; the mechanism used to distribute
this information is outside the scope of this
document.
When CPE1 (192.0.2.0/26) and CPE2 (192.0.2.64/26) need to
communicate with each other, they establish a separate
bidirectional end-to-end IPsec Tunnel, with SA5 for the
traffic from CPE1 to CPE2 and SA6 for the traffic from CPE2
to CPE1. The Cloud GW forwards this IPsec-protected traffic
without terminating or decrypting the CPE1-to-CPE2 IPsec
tunnel. Assume the IPsec ESP Tunnel Mode is used. A packet
from 192.0.2.1 to 192.0.2.65 has the following outer
header:
Dunbar, et al. Expires Dec 24, 2027 [Page 36]
Internet-Draft Multi-segment SD-WAN
+---------------------------------------------------+
| Outer IP Header |
| Protocol = 17 (UDP) |
| Src IP = CPE1 (underlay address) |
| Dst IP = Cloud GW (underlay address) |
+---------------------------------------------------+
| UDP Header |
| Src Port = xxxx (ephemeral) |
| Dst Port = 6081 (GENEVE) |
+===================================================+
| GENEVE Header |
| Protocol Type = 0x0800 (IPv4) or 0x86DD (IPv6) |
| [Indicates the payload is an IP packet] |
+---------------------------------------------------+
| MultiSeg-SDWAN Option Class |
| SD-WAN EndPt Sub-TLV (CPE2 address) |
| [Optional other SD-WAN Sub-TLVs] |
| HMAC-Auth-Val Sub-TLV (GENEVE Hdr Authentication)|
| (validated by GW) |
+===================================================+
| ESP Outer IP Header (Tunnel Mode) |
| Src IP = CPE1 (tunnel IP) |
| Dst IP = CPE2 (tunnel IP) |
| Protocol = 50 (ESP) |
+---------------------------------------------------+
| ESP Header |
| SPI (Security Parameters Index) |
| Sequence Number |
+---------------------------------------------------+
| Encrypted Payload |
| Inner IP Header |
| Src = 192.0.2.1 (host behind CPE1) |
| Dst = 192.0.2.65 (host behind CPE2) |
| Protocol = TCP |
| TCP Header |
| Application Payload |
| Padding |
| Pad Length |
| Next Header |
+---------------------------------------------------+
| Integrity Check Value (ICV) |
| (Generated by CPE1, validated by CPE2) |
+---------------------------------------------------+
Figure 10 Packet Encapsulation through a Single Cloud GW
Dunbar, et al. Expires Dec 24, 2027 [Page 37]
Internet-Draft Multi-segment SD-WAN
A.2 Multi-hop Transit GWs
Traffic between geographically separated CPEs can traverse
multiple Cloud DCs over the Cloud backbone.
Using Figure 2 as an example, assume the following:
- There is a bidirectional IPsec tunnel between CPE1 and
the Cloud GW1; with IPsec SA1 for the traffic from the
CPE1 to the Cloud-GW1; and IPsec SA2 for the traffic
from the Cloud-GW1 to the CPE1. This tunnel can carry
CPE1 traffic to and from cloud services. The existing
IPsec-protected channel can also be used to securely
exchange configuration, control, and keying
information, such as the HMAC key used to protect the
GENEVE metadata. This tunnel is separate from the end-
to-end CPE1-to-CPE2 IPsec tunnel described below.
- There is a bidirectional IPsec tunnel between CPE10
and the Cloud GW2; with IPsec SA3 for the traffic from
the CPE10 to the Cloud-GW2; and IPsec SA4 for the
traffic from the Cloud-GW2 to the CPE10. These SAs can
carry CPE10 traffic to and from cloud services and can
also be used to securely exchange configuration,
control, and keying information. It is separate from
the end-to-end CPE1-to-CPE10 IPsec tunnel.
- All the CPEs are under a common administrative control
and exchange routing information through an iBGP Route
Reflector (RR). The CPEs can exchange information
about their corresponding Cloud GW addresses through
the control plane; the mechanism used to distribute
this information is outside the scope of this
document.
When CPE1(192.0.2.0/26) and CPE10(192.0.2.128/25) need to
communicate with each other, CPE1 and CPE10 establish a
bidirectional IPsec Tunnel, with SA5 for the traffic from
CPE1 to CPE10 and SA6 for the traffic from CPE10 to CPE1.
Assuming the IPsec ESP Tunnel Mode is used, a packet from
192.0.2.1 to 192.0.2.129 has the following encapsulation:
Dunbar, et al. Expires Dec 24, 2027 [Page 38]
Internet-Draft Multi-segment SD-WAN
+---------------------------------------------------+
| Outer IP Header |
| Protocol = 17 (UDP) |
| Src IP = CPE1 (underlay address) |
| Dst IP = Cloud GW1 (underlay address) |
+---------------------------------------------------+
| UDP Header |
| Src Port = xxxx (ephemeral) |
| Dst Port = 6081 (GENEVE) |
+===================================================+
| GENEVE Header |
| Protocol Type = 0x0800 (IPv4) or 0x86DD (IPv6) |
| [Indicates payload is an IP packet] |
+---------------------------------------------------+
| MultiSeg-SDWAN Option Class |
| SD-WAN EndPt Sub-TLV (CPE10 address) |
| EgressGW Sub-TLV (explicit egress Cloud GW2) |
| HMAC-Auth-Val Sub-TLV (GENEVE Hdr Authentication) |
| (validated by GW) |
+===================================================+
| ESP Outer IP Header (Tunnel Mode) |
| Src IP = CPE1 (tunnel IP) |
| Dst IP = CPE10 (tunnel IP) |
| Protocol = 50 (ESP) |
+---------------------------------------------------+
| ESP Header |
| SPI (Security Parameters Index) |
| Sequence Number |
+---------------------------------------------------+
| Encrypted Payload |
| Inner IP Header |
| Src = 192.0.2.1 (host behind CPE1) |
| Dst = 192.0.2.129 (host behind CPE10) |
| Protocol = TCP |
| TCP Header |
| Application Payload |
| Padding |
| Pad Length |
| Next Header |
+---------------------------------------------------+
| Integrity Check Value (ICV) |
| <- validated by CPE10 |
+---------------------------------------------------+
Figure 11: Packet Encapsulation through Multiple Cloud GWs
Dunbar, et al. Expires Dec 24, 2027 [Page 39]
Internet-Draft Multi-segment SD-WAN
Authors' Addresses
Kausik Majumdar
Upscale AI
Email: kmajumdar@upscaleai.com
Linda Dunbar
Futurewei
Email: ldunbar@futurewei.com
Venkit Kasiviswanathan
Nexthop AI
Email: venkit@nexthop.ai
Ashok Ramchandra
Google
Email: archiashok@gmail.com
Dunbar, et al. Expires Dec 24, 2027 [Page 40]
Internet-Draft Multi-segment SD-WAN
Aseem Choudhary
Cisco
Email: asechoud@cisco.com
Contributors' Addresses
Ajeet Pal Singh Gill
Microsoft Azure
Email: ajeetgill@microsoft.com
Dunbar, et al. Expires Dec 24, 2027 [Page 41]
