Skip to main content

BGP UPDATE for SD-WAN Edge Discovery
draft-ietf-idr-sdwan-edge-discovery-17

The information below is for an old version of the document.
Document Type
This is an older version of an Internet-Draft whose latest revision state is "Active".
Authors Linda Dunbar , Kausik Majumdar , Susan Hares , Robert Raszuk , Venkit Kasiviswanathan
Last updated 2024-10-14 (Latest revision 2024-09-10)
Replaces draft-dunbar-idr-sdwan-edge-discovery
RFC stream Internet Engineering Task Force (IETF)
Formats
Reviews
Additional resources Mailing list discussion
Stream WG state WG Consensus: Waiting for Write-Up
Document shepherd Keyur Patel
Shepherd write-up Show Last changed 2023-09-08
IESG IESG state I-D Exists
Consensus boilerplate Yes
Telechat date (None)
Responsible AD (None)
Send notices to keyur@arrcus.com
draft-ietf-idr-sdwan-edge-discovery-17
Network Working Group                                          L. Dunbar
Internet-Draft                                                 Futurewei
Intended status: Standards Track                             K. Majumdar
Expires: 17 April 2025                                   Microsoft Azure
                                                                S. Hares
                                                 Hickory Hill Consulting
                                                               R. Raszuk
                                                                  Arrcus
                                                      V. Kasiviswanathan
                                                                  Arista
                                                         14 October 2024

                  BGP UPDATE for SD-WAN Edge Discovery
                 draft-ietf-idr-sdwan-edge-discovery-17

Abstract

   The document describes the encoding of BGP UPDATE messages for the
   SD-WAN edge node property discovery.

   In the context of this document, BGP Route Reflector (RR) is the
   component of the SD-WAN Controller that receives the BGP UPDATE from
   SD-WAN edges and in turns propagates the information to the intended
   peers that are authorized to communicate via the SD-WAN overlay
   network.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119] [RFC8174]
   when, and only when, they appear in all capitals, as shown here.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

Dunbar, et al.            Expires 17 April 2025                 [Page 1]
Internet-Draft            SDWAN Edge Discovery              October 2024

   This Internet-Draft will expire on 17 April 2025.

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Conventions used in this document . . . . . . . . . . . . . .   3
   3.  Framework of SD-WAN Edge Discovery  . . . . . . . . . . . . .   4
     3.1.  The Objectives of SD-WAN Edge Discovery . . . . . . . . .   5
     3.2.  Comparing with Pure IPsec VPN . . . . . . . . . . . . . .   5
     3.3.  Client Routes and SDWAN UPDATE  . . . . . . . . . . . . .   7
       3.3.1.  Client Routes . . . . . . . . . . . . . . . . . . . .   7
       3.3.2.  SD-WAN Underlay UPDATE  . . . . . . . . . . . . . . .   9
     3.4.  Edge Node Discovery . . . . . . . . . . . . . . . . . . .  11
   4.  Constrained propagation of BGP UPDATE . . . . . . . . . . . .  12
     4.1.  SD-WAN Segmentation, Virtual Topology and Client VPN  . .  12
     4.2.  Constrained Propagation of Edge Capability  . . . . . . .  13
   5.  Client Route UPDATE Procedures  . . . . . . . . . . . . . . .  14
     5.1.  Tunnel Form 1: Encapsulation Extended Community
           (Encaps-EC) . . . . . . . . . . . . . . . . . . . . . . .  14
     5.2.  Tunnel Form 2: Tunnel Encapsulation Path Attribute
           (TEA) . . . . . . . . . . . . . . . . . . . . . . . . . .  14
     5.3.  Multiple tunnels attached to One Route  . . . . . . . . .  15
     5.4.  SD-WAN VPN ID in Client Route Update  . . . . . . . . . .  15
     5.5.  SD-WAN VPN ID in Data Plane . . . . . . . . . . . . . . .  15
   6.  SD-WAN Underlay UPDATE  . . . . . . . . . . . . . . . . . . .  15
     6.1.  NLRI for SD-WAN Underlay Tunnel Update  . . . . . . . . .  16
     6.2.  SD-WAN-Hybrid Tunnel Encoding . . . . . . . . . . . . . .  17
   7.  Extended Port Attribute Sub-TLV . . . . . . . . . . . . . . .  18
     7.1.  Extended SubSub-TLV . . . . . . . . . . . . . . . . . . .  21
       7.1.1.  Underlay Network Transport SubSub-TLV . . . . . . . .  21
   8.  IPsec SA Property Sub-TLVs  . . . . . . . . . . . . . . . . .  22
     8.1.  IPsec-SA-ID Sub-TLV . . . . . . . . . . . . . . . . . . .  23
     8.2.  IPsec SA Rekey Counter Sub-TLV  . . . . . . . . . . . . .  24
     8.3.  IPsec Public Key Sub-TLV  . . . . . . . . . . . . . . . .  26

Dunbar, et al.            Expires 17 April 2025                 [Page 2]
Internet-Draft            SDWAN Edge Discovery              October 2024

     8.4.  IPsec SA Proposal Sub-TLV . . . . . . . . . . . . . . . .  26
     8.5.  Simplified IPsec SA sub-TLV . . . . . . . . . . . . . . .  27
   9.  Error handling  . . . . . . . . . . . . . . . . . . . . . . .  29
     9.1.  Error handling for the Tunnel Encapsulation Signaling . .  29
     9.2.  Error Handling for NLRI . . . . . . . . . . . . . . . . .  31
     9.3.  SDWAN NLRI and Tunnel Encapsulation Attribute . . . . . .  32
   10. Manageability Considerations  . . . . . . . . . . . . . . . .  32
     10.1.  Detecting Misaligned Tunnels . . . . . . . . . . . . . .  32
     10.2.  IPsec Attributes Mismatch  . . . . . . . . . . . . . . .  32
       10.2.1.  SD-WAN Hybrid Tunnel Mechanisms for Passing IPSec
               Security Association Info . . . . . . . . . . . . . .  33
       10.2.2.  Example creation of IPsec Security Association over
               SD-WAN Hybrid tunnel  . . . . . . . . . . . . . . . .  34
   11. Security Considerations . . . . . . . . . . . . . . . . . . .  35
   12. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  36
     12.1.  Hybrid (SD-WAN) Overlay SAFI . . . . . . . . . . . . . .  36
     12.2.  Tunnel Encapsulation Attribute Type  . . . . . . . . . .  36
     12.3.  Tunnel Encapsulation Attribute Sub-TLV Types . . . . . .  36
   13. References  . . . . . . . . . . . . . . . . . . . . . . . . .  37
     13.1.  Normative References . . . . . . . . . . . . . . . . . .  37
     13.2.  Informative References . . . . . . . . . . . . . . . . .  38
   Appendix A.  Acknowledgments  . . . . . . . . . . . . . . . . . .  39
   Contributors  . . . . . . . . . . . . . . . . . . . . . . . . . .  39
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  39

1.  Introduction

   BGP [RFC4271] can be used as a control plane for a SD-WAN network.
   SD-WAN network refers to a policy-driven network over multiple
   heterogeneous underlay networks to get better WAN bandwidth
   management, visibility, and control.

   The document describes BGP UPDATE messages for an SD-WAN edge node to
   advertise its properties to its RR which then propagates that
   information to the authorized peers.

2.  Conventions used in this document

   The following acronyms and terms are used in this document:

   Cloud DC:  Off-Premises Data Centers that usually host applications
      and workload owned by different organizations or tenants.

   Color-EC:  Color Extended Community defined in [RFC9012].

   Controller:  Used interchangeably with SD-WAN controller to manage
      SD-WAN overlay path creation/deletion and monitor the path
      conditions between sites.

Dunbar, et al.            Expires 17 April 2025                 [Page 3]
Internet-Draft            SDWAN Edge Discovery              October 2024

   CPE:  Customer (Edge) Premises Equipment

   CPE-Based VPN:  Virtual Private Secure network formed among CPEs.
      This is to differentiate such VPNs from most commonly used PE-
      based VPNs discussed in [RFC4364].

   Encaps-EC:  Encapsulation Extended Community defined in [RFC9012].

   MP-NLRI:  Multi-Protocol Network Layer Reachability Information
      [MP_REACH_NLRI] Path Attribute defined in [RFC4760].

   RR:  Route Reflector

   SD-WAN:  An overlay connectivity service that optimizes transport of
      IP Packets over one or more Underlay Connectivity Services by
      recognizing applications (Application Flows) and determining
      forwarding behavior by applying Policies to them.  [MEF-70.1]

   SD-WAN Endpoint:  can be the SD-WAN edge node address, a WAN port
      address (logical or physical) of a SD-WAN edge node, or a client
      port address.

   SD-WAN Hybrid tunnel:  A single logical tunnel that combines several
      links of different encapsulation iinto a single tunnel.

   RT-EC:  Route Target Extended Community [RFC4360]

   TEA:  Tunnel Encapsulation Path Attribute [RFC9012]

   VPN:  Virtual Private Network

   VRF:  VPN Routing and Forwarding instance

   WAN:  Wide Area Network

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC8174] when, and only when, they appear in all capitals, as
   shown here.

3.  Framework of SD-WAN Edge Discovery

Dunbar, et al.            Expires 17 April 2025                 [Page 4]
Internet-Draft            SDWAN Edge Discovery              October 2024

3.1.  The Objectives of SD-WAN Edge Discovery

   The objectives of SD-WAN edge discovery for an SD-WAN edge node are
   to discover its authorized BGP peers and each peer's associated
   properities in order to establish secure overlay tunnels [Net2Cloud].
   The attributes to be propagated include:

   *  the SD-WAN (client) VPNs information,

   *  the attached routes under the SD-WAN VPNs, and

   *  the properties of the underlay networks over which the client
      routes can be carried.

   Some SD-WAN peers are connected by both trusted VPNs and untrusted
   public networks.  Some SD-WAN peers are connected only by untrusted
   public networks.  For the traffic over untrusted networks, IPsec
   Security Associations (IPsec SA) must be established and maintained.
   For the trusted VPNs, IPsec Security associations may not be set-up.
   If an edge node has network ports behind a NAT, the NAT properties
   need to be discovered by the authorized SD-WAN peers.

   Like any VPN networks, the attached client routes belonging to
   specific SD-WAN VPNs can only be exchanged with the SD-WAN peer nodes
   authorized to communicate.

3.2.  Comparing with Pure IPsec VPN

   A pure IPsec VPN has IPsec tunnels connecting all edge nodes over
   public networks.  Therefore, it requires stringent authentication and
   authorization (i.e., IKE Phase 1) before other properties of IPsec SA
   can be exchanged.  The IPsec Security Association (SA) between two
   untrusted nodes typically requires the following configurations and
   message exchanges:

      IPsec IKEV2:  messages sent to authenticate with each other.

      Establish IPsec SA  : requires the following set-up

         -  Local key configuration,

         -  Setting the Remote Peer address,

         -  Information from IKEv2 Proposal directly sent to peer
            (Encryption method, Integrity sha512, etc.), and

         -  Transform set.

Dunbar, et al.            Expires 17 April 2025                 [Page 5]
Internet-Draft            SDWAN Edge Discovery              October 2024

      Attached client prefixes discovery acchieved by:
         -  Running routing protocol within each IPsec SA.

         -  If multiple IPsec SAs between two peer nodes are established
            to achieve load sharing, each IPsec tunnel needs to run its
            own routing protocol to exchange client routes attached to
            the edges.

      Set-up Access List or Traffic Selector:  such as Permit Local-IP1,
         Remote-IP2, and other parameters.

   In a BGP-controlled SD-WAN network over hybrid MPLS VPN and public
   internet underlay networks, all edge nodes and RRs are already
   connected by private secure paths.  The RRs have the policies to
   manage the authentication of all peer nodes.  More importantly, when
   an edge node needs to establish multiple IPsec tunnels to many edge
   nodes, all the management information can be multiplexed into the
   secure management tunnel between RR and the edge node operating as a
   BGP peer.  Therefore, the amount of authentication in a BGP-
   Controlled SD-WAN network can be significantly reduced.

   Client VPNs are configured via VRFs, just like the configuration of
   the existing MPLS VPN.  The IPsec equivalent traffic selectors for
   local and remote routes are achieved by importing/exporting VPN Route
   Targets.  The binding of client routes to IPsec SA is dictated by
   policies.  As a result, the IPsec configuration for a BGP controlled
   SD-WAN (with mixed MPLS VPN) can be simplified in the following
   manner:

   *  The SD-WAN controller has the authority to authenticate edges and
      peers so the Remote Peer association is controlled by the SD-WAN
      Controller (RR).

   *  The IKEv2 proposals (including the IPsec Transform set) can be
      sent directly to peers, or incorporated in a BGP UPDATE.

   *  The BGP UPDATE announces the client route reachability through the
      SDWAN hybrid tunnels.  A SDWAN hybrid tunnel combines several
      other tunnels into a single logical tunnel.  The SD-WAN Hybrid
      tunnel implementations insure that all tunnels within are either
      running over secure network links or secured by IPsec.

   *  Importing/exporting Route Targets under each client VPN (VRF)
      achieves the traffic selection (or permission) among clients'
      routes attached to multiple edge nodes.

Dunbar, et al.            Expires 17 April 2025                 [Page 6]
Internet-Draft            SDWAN Edge Discovery              October 2024

   Note: The web of SDWAN hybrid tunnels in a network is denoted in this
   document as an SD-WAN underlay.  BGP passes information about the
   SDWAN hybrid tunnels between BGP peers by passing an SD-WAN Underlay
   NLRI paired with the tunnel encapsulation attribute (TEA) with an
   SDWAN tunnel type SD-WAN-Hybrid TLV.

   Also, note that with this method there is no need to run multiple
   routing protocols in each IPsec tunnel.

3.3.  Client Routes and SDWAN UPDATE

   There are two different sequences of BGP UPDATE messages are used for
   SD-WAN Edge Discovery.  The first associates Client routes BGP
   Updates with a SD-WAN Hybrid Tunnel.  The second passes information
   regarding the SD-WAN web of Hybrid tunnels (denoted as the SD-WAN
   Underlay) between Egress routers and Ingress routers to aid set-up of
   SD-WAN Hybrid tunnels in a network./

3.3.1.  Client Routes

   This section describes how client routes in BGP Updates are
   associated with the SD-WAN Hybrid Tunnel.

   Client routes BGP UPDATE:

         This BGP UPDATE message contains the client routes (NLRI), Next
         Hop, and the attributes which identify the Hybrid SD-WAN tunnel
         toward the Next Hop. The SD-WAN-Hybrid Tunnel BGP attributes
         are either passed as either:

         *  Encapsulation Extended Community [Encap-EC] which identifies
            the SD-WAN-hybrid tunnel with a Tunnel Egress End Point as
            NextHop in BGP Update [per RFC9012],

         *  Tunnel Encapsulation Attribute (TEA) which identifies the
            SD-WAN-Hybrid tunnel and uses the Tunnel Egress Endpoint
            SubTLV to identify the egress endpoint (see [RFC9012]
            section 3.1)

         Ordering: If both the TEA and the Extended Community for tunnel
         information exists, the Extended Community is preferred (i.e.
         takes precedence.)

   Sample Topology:  For a Hybrid SD-WAN Tunnel

Dunbar, et al.            Expires 17 April 2025                 [Page 7]
Internet-Draft            SDWAN Edge Discovery              October 2024

               Site 15.0.0.2
                                     +---+
                      +--------------|RR |----------+
                     /  Untrusted    +-+-+           \
                    /                                 \
                   /                                   \
           +---------+  MPLS Path                      +-------+
   11.1.1.x| C-PE1   A1-------------------------------B1 C-PE2 |10.1.1.x
           |         |                                 |       |
   21.1.1.x|         A2(192.10.0.10)------( 192.0.0.1)B2       |20.1.1.x
           |         |                                 |       |
           | Addr    A3(160.0.0.1) --------(170.0.0.1)B3 Addr  |
           | 1.1.1.1 |                                 |2.2.2.2|
           +---------+                                 +-------+

                        Figure 1: Hybrid SD-WAN

   Example Client Routes:  In figure 1, four overlay paths between C-PE1
      and C-PE2 are established for illustration purpose.  More overlay
      paths are possible.  One physical port on C-PE2 can terminate
      multiple overlay paths from different ports on C-PE1.

      a)  MPLS-in-GRE path;

      b)  node-based IPsec tunnel [2.2.2.2 - 1.1.1.1].  As C-PE2 has two
         public internet facing WAN ports, either of those two WAN port
         IP addresses can be the outer destination address of the IPsec
         encapsulated data packets;

      c)  port-based IPsec tunnel [192.10.0.10 - 192.0.0.1]; and

      d)  port-based IPsec tunnel [160.0.0.1 - 172.0.0.1].

   C-PE2 advertises the attached client routes  in one of two forms
      below:

         Client Route UPDATE - [RFC9012] "Barebones" (option 1)

            NLRIs: AFI=IPv4 (1) and SAFI = VPN (128)
              Prefix: 10.1.1.x; 20.1.1.x
              NextHop: 2.2.2.2 (C-PE2)
            Attributes:
              Extended community RT: (RT-EC) for SD-WAN VPN 1
              Encapsulation Extended Community (Encaps-EC)
                    tunnel-type=SD-WAN-Hybrid
                  Color Extended Communit (Color-EC) [optional]

Dunbar, et al.            Expires 17 April 2025                 [Page 8]
Internet-Draft            SDWAN Edge Discovery              October 2024

                      Figure 2: Client Routes Update Message

         Client Route UPDATE [RFC9012] Tunnel Encaps Attribute (TEA)
         (option 2)

            NLRIs: AFI=IPv4 (1) and SAFI = VPN (128)
              Prefix: 10.1.1.x; 20.1.1.x
                  NextHop: 2.2.2.2 (C-PE2)
            Attributes:
              Extended community RT: (RT-EC) for SD-WAN VPN 1
              Tunnel Encapsulation Attribute (TEA)
                Tunnel Egress Endpoint SubTLV: 2.2.2.2 (C-PE2)
                Color SubTLV [optional]

                      Figure 3: Client Routes Update Message

3.3.2.  SD-WAN Underlay UPDATE

   Edges nodes use this BGP UPDATE to advertise the properties of
   directly attached underlay networks and IPsec SA attributes
   associated with an SD-WAN-Hybrid tunnel.  The properties of underlay
   networks include encapsulation, NAT and underlay physical properties.
   The IPsec SA attributes passed include keys, nonce, and encryption
   algorithms, and other IP SEC attributes.

   The security attributes are likely to change more rapidly than the
   physical attributes of links within the Hybrid SD-Wan Tunnel.
   Typically the attributes of the links are passed during initial set-
   up of Hybrid SD-WAN tunnels in the network.

   Given the topology in figure 1, C-PE2 can send the SD-WAN NLRI in a
   BGP Update messages to advertise the properties of Internet facing
   ports 192.0.0.1 and 170.0.0.1, and their associated IPsec SA related
   parameters.  One associated paramater for the SDWAN underlay is the
   SD-WAN Color.  This SD-WAN color (see section 6.1) indicates a group
   of tunnels.  The Client routes may signal the desire to use this
   group of tunnels by optionally attaching a Color Extended Community
   (Color-EC) or by local policy.  Local network policy determines the
   mapping between Color-EC and the tunnel groupings identified by the
   SD-WAN-Color.

   Example 1 below provides sample BGP Updates per port.  This type of
   UPDATE packing provides poor packing of UPDATES, but it may occur.
   Example 2 provides a single BGP Update which passes the initial
   information in one update.  In the update, Color-EC is Color Extended
   Community [RFC4360].

Dunbar, et al.            Expires 17 April 2025                 [Page 9]
Internet-Draft            SDWAN Edge Discovery              October 2024

3.3.2.1.  Example #1 SD-WAN Underlay Update Messages Used in Set-Up

   Example #1 - BGP Updates per Port

   # Update #1 for Port B2 on C-PE2
     SD-WAN NLRI (AFI=1/SAFI=SD-WAN)
       Port-id = Local port ID for WAN Port 192.0.0.1
       SD-WAN Color = Color indicates Group of underlay tunnels
       SD-WAN Node ID = 2.2.2.2 (C-PE2)
     Tunnel Encaps Attribute (TEA)
       Tunnel TLV: (type: Hybrid-SD-WAN)
         Tunnel Egress Endpoint SubTLV: 2.2.2.2
         Extended-Port Attribute SubTLV: Port B2 (192.0.0.1)
         IPsec SA Identifier SubTLV:  SA-1, SA-2

   # Update #2 for Port B3 (IPsec) in Hybrid tunnel on PE2
     SD-WAN NLRI (AFI=1/SAFI=SD-WAN)
       Port-id = Local port ID for WAN Port 170.0.0.1
       SD-WAN Color = Color indicates Group of Underlay tunnels
       SD-WAN Node ID = 2.2.2.2 (C-PE2)
     Tunnel Encaps Attribute (TEA)
       Tunnel TLV: (type: SD-Wan-Hybrid)
         Tunnel Egress Endpoint SubTLV (SubTLV=6) = 2.2.2.2
             Extended Port Attribute SubTLV: Port B3 (170.0.0.1)
             Simplified IPSec SD-WAN SubTLV: SA-3, SA-4

   See section 6.1 for SD-WAN NLRI definition.
   See section 7.1 for Extended Port Attribute SubTLV definition.
   See section 8.1 for IPsec SA Identifier SubTLV.
   See section 8.5 for Simplified IPsec SD-WAN SubTlv.

                    Figure 4: SDWAN NLRI Update per Port

3.3.2.2.  Example #2 IPsec terminated at Node with Hybrid Tunnel

   Example #2 - IP Sec Terminated at Node C-PE2

Dunbar, et al.            Expires 17 April 2025                [Page 10]
Internet-Draft            SDWAN Edge Discovery              October 2024

   # Ports B1 (MPLS), B2 (IPsec), B3 (IPsec)
   # Update for C-PE2 (IPSec)
     SD-WAN NLRI (AFI=1 / SAFI = SD-WAN)
       SD-WAN Color = Match to Color-EC of Client routes
           Port ID = 0
           SD-WAN Node ID = 2.2.2.2
     Tunnel Encaps Attribute (TEA)
           Tunnel TLV (type: SD-Wan-Hybrid)
         Egress Endpoint = 2.2.2.2
         IPsec-SA-ID: SA-1, SA-2, SA-3, SA-4

                    Figure 5: SDWAN NLRI Update 3 ports

3.4.  Edge Node Discovery

   The basic scheme of SD-WAN edge node discovery using BGP consists of
   the following:

   *  Secure connection to a SD-WAN controller (BGP RR in this context):

      -  For an SD-WAN edge with both MPLS and IPsec paths, the edge
         node should already have a secure connection to its controller
         (RR in this context).  For an SD-WAN edge that is only
         accessible via Internet, the SD-WAN edge upon power-up
         establishes a secure tunnel (such as TLS or SSL) with the SD-
         WAN central controller whose address is preconfigured on the
         edge node.  The central controller informs the edge node of its
         local RR.  The edge node then establishes a transport layer
         secure session with the RR (such as TLS or SSL).

   *  The BGP Peer Edge node will advertise the properties of its Hybrid
      SD-WAN Tunnel to its designated RR via the secure connection.

   *  The RR propagates the received information to the authorized BGP
      peers.

   *  The authorized BGP peers can establish the secure data channels
      via Hybrid SD-WAN tunnel and exchange more information among each
      other.

   For an SD-WAN deployment with multiple RRs, it is assumed that there
   are secure connections among those RRs.  How secure connections are
   established among those RRs is out of the scope of this document.
   The existing BGP UPDATE propagation mechanisms control the edge
   properties propagation among the RRs.

Dunbar, et al.            Expires 17 April 2025                [Page 11]
Internet-Draft            SDWAN Edge Discovery              October 2024

   For some environments where the communication to RR is highly
   secured, [RFC9016] IKE-less can be deployed to simplify IPsec SA
   establishment among edge nodes.

4.  Constrained propagation of BGP UPDATE

4.1.  SD-WAN Segmentation, Virtual Topology and Client VPN

   In SD-WAN deployment, SD-WAN Segmentation is a frequently used term
   which refers to partitioning a network into multiple subnetworks,
   just like MPLS VPNs.  SD-WAN Segmentation is achieved by creating SD-
   WAN virtual topologies and SD-WAN VPNs.  An SD-WAN virtual topology
   consists of a set of edge nodes and the tunnels (a.k.a. underlay
   paths) interconnecting those edge nodes.  These tunnels forming the
   underlay paths can be IPsec tunnels, or MPLS VPN tunnels, or other
   tunnels.

   An SD-WAN VPN is configured in the same way as the VRFs of an MPLS
   VPN.  One SD-WAN client VPN can be mapped to multiple SD-WAN virtual
   topologies.  SD-WAN Controller governs the policies of mapping a
   client VPN to SD-WAN virtual topologies.

   Each SD-WAN edge node may need to support multiple VPNs.  Route
   Target is used to differentiate the SD-WAN VPNs.  For example, in the
   picture below, the Payment-Flow on C-PE2 is only mapped to the
   virtual topology of C-PEs to/from Payment Gateway, whereas other
   flows can be mapped to a multipoint-to-multipoint virtual topology.

Dunbar, et al.            Expires 17 April 2025                [Page 12]
Internet-Draft            SDWAN Edge Discovery              October 2024

                                     +---+
                      +--------------|RR |----------+
                     /  Untrusted    +-+-+           \
                    /                                 \
                   /                                   \
           +---------+  MPLS Path                      +-------+
   11.1.1.x| C-PE1   A1-------------------------------B1 C-PE2 |10.1.1.x
           |         |                                 |       |
   21.1.1.x|         A2(192.10.0.10)------( 192.0.0.1)B2       |20.1.1.x
           |         |                                 |       |
           | Addr    A3(160.0.0.1) --------(170.0.0.1)B3 Addr  |11.2.2.x
           | 1.1.1.1 |                              /  |2.2.2.2|
           +---------+                             /   +-------+
                      \                           /
                       \                         /PaymentFlow
                        \                       /
                         \                +----+----+
                          +---------------| payment |
                                          | Gateway |
                                          +---------+

                 Figure 6: SD-WAN Virtual Topology and VPN

4.2.  Constrained Propagation of Edge Capability

   BGP Route Reflectors [RFC4456] may be configured to constrain the
   distribution of BGP informtion to specific BGP clients.

          NLRI={SD-WAN#1, SD-WAN#2}       NLRI={SD-WAN#1, SD-WAN#3}
               ----->                 +---+      <-----------
                 +--------------------|RR1|------------------+
                 | Outbound Filter    +---+  Outbound Filter |
                 | Permit SD-WAN#1,#2      Permit SD-WAN#1,#3|
                 |   <-------                --------->      |
                 |                                           |
           +-----+---+  MPLS Path                      +-----+-+
   11.1.1.x| C-PE1   A1-------------------------------B1 C-PE2 |10.1.1.x
           |         |                                 |       |
   21.1.1.x|         A2(192.10.0.10)------( 192.0.0.1)B2       |20.1.1.x
           |         |                                 |       |
           | Addr    A3(160.0.0.1) --------(170.0.0.1)B3 Addr  |
           | 1.1.1.1 |                                 |2.2.2.2|
           +---------+                                 +-------+
   SD-WAN VPN #1                                          SD-WAN VPN #1
   SD-WAN VPN #2                                          SD-WAN VPN #3

             Figure 7: Constraint propagation of Edge Property

Dunbar, et al.            Expires 17 April 2025                [Page 13]
Internet-Draft            SDWAN Edge Discovery              October 2024

   The RR is configured to speak to the BGP clients (CE-PE1 and CE-PE2)
   over secure virtual links (IPsec), and send only certain routes.  The
   configuration on the RR and the BGP Peers sending SD-WAN routes forms
   a "walled garden" for the SD-WAN information.

   It is out of the scope of this document on how RR is configured with
   the policies to filter out unauthorized nodes for specific SD-WAN
   VPNs.

5.  Client Route UPDATE Procedures

   The Tunnel Encapsulation Attribute for the SD-WAN Hybrid Tunnel Type
   may be associated with BGP UPDATE messages with NLRI with AFI/SAFI
   IPv4 Unicast (1/1), IPv4 with MPLS labels (1/4), IPVPN-IPv4 Label
   Unicast (1/128), IPv6 Unicast (2/1), IPv6 with MPLS labels (2/4),
   VPN-IPv6 Label Unicast (2/128), and EVPN (25/70).

   When associated with any NLRI in this set, these routes are described
   as "Client Routes" in this document.  Based on [RFC9012], there are
   two forms a Tunnel Encapsulation Attribute (TEA) can take:
   "Barebones" using the Encapsulation Extended Community (Encaps-EC)
   and a normal Tunnel Encapsulation form.

5.1.  Tunnel Form 1: Encapsulation Extended Community (Encaps-EC)

   The SD-WAN Client Route UPDATE message uses the Encapsulation
   Extended Community (Encap-EC) to identify the Hybrid SD-WAN tunnel
   and the Tunnel Egress Endpoint.  Per [RFC9012], the Encapsulation
   Extended Community uses the NextHop Field in the BGP UPDATE as the
   Tunnel Egress EndPoint.  The validation for the Tunnel Egress
   Endpoint uses the validation in section 6, 8, and 13 applied to the
   NextHop.

   A Color Extended Community (Color-EC) or local policy applied to the
   client route directs the traffic for the client route to across
   appropriate interface within the Hybrid SD-WAN Tunnel to the Tunnel
   Egress Endpoint.

5.2.  Tunnel Form 2: Tunnel Encapsulation Path Attribute (TEA)

   The Client route with the Tunnel Encapsulation Path Attribute (TEA)
   with the Hybrid SD-WAN route TLV must have the Tunnel Egress Endpoint
   (SubTLV=6) and any of the SubTLVs found in [RFC9012].  The validation
   for the Tunnel Egress Endpoint uses the validation in section 6, 8,
   and 13 from [RFC9012].

Dunbar, et al.            Expires 17 April 2025                [Page 14]
Internet-Draft            SDWAN Edge Discovery              October 2024

5.3.  Multiple tunnels attached to One Route

   A single SD-WAN client route may be attached to multiple SD-WAN
   Hybrid tunnels.  An Update with an SD-WAN client route may express
   these tunnels as an Encap-EC or a TEA.  Each of these tunnel
   descriptions is treated as a unique Hybrid SD-WAN tunnel with a
   unique Egress Endpoint.  Local Policy on the BGP Peer determines
   which tunnel the client data traffic will use.

5.4.  SD-WAN VPN ID in Client Route Update

   An SD-WAN VPN ID is same as a client VPN in a BGP controlled SD-WAN
   network.  The Route Target Extended Community should be included in a
   Client Route UPDATE message to differentiate the client routes from
   routes belonging to other VPNs.  Route Target value is taken as the
   VPN ID (for 1/1 and 2/1).  For 1/128 and 2/128, the RD from the NLRI
   identifies the VPN ID.  For EVPN, picking up the VPN-ID from EVPN
   SAFI.

5.5.  SD-WAN VPN ID in Data Plane

   SD-WAN edge node which can be reached by either an MPLS path or an
   IPsec path within the hybrid SD-WAN tunnel.  If client packets are
   sent via a secure MPLS network within the Hybrid SD-WAN tunnel, then
   the data packets will have MPLS headers with the MPLS Labels based on
   the scheme specified by [RFC8277].  It is assumed the secure MPLS
   network assures the security outer MPLS Label header.

   If the packets are sent via a link with IPsec outer encryption across
   a public network, the payload is still encrypted with GRE or VXLAN
   encryption.  For GRE Encapsulation within an IPsec tunnel, the GRE
   key field can be used to carry the SD-WAN VPN ID.  For network
   virtual overlay (VxLAN, GENEVE, etc.) encapsulation within the IPsec
   tunnel, the Virtual Network Identifier (VNI) field is used to carry
   the SD-WAN VPN ID.

6.  SD-WAN Underlay UPDATE

   The hybrid SD-WAN underlay tunnel UPDATE is to advertise the detailed
   properties associated with the public facing WAN ports and IPsec
   tunnels.  The Edge BGP Peer will advertise the SD-WAN properties to
   its designated RR via the secure connection.  Each BGP Update message
   with a the SD-WAN Underlay NLRI MUST contain a Tunnel Encapsulation
   Attribute (TEA) for a Hybrid Tunnel type.  The TEA can include with
   SubTLVs for Extended Port attribute (see section 7) or IP Sec
   information (see section 8).  The IPsec information subTLVs include:
   IPsec-SA-ID, IPsec SA Nonce, IPsec Public Key, IPsec SA Proposal, and
   Simplified IPsec SA.

Dunbar, et al.            Expires 17 April 2025                [Page 15]
Internet-Draft            SDWAN Edge Discovery              October 2024

6.1.  NLRI for SD-WAN Underlay Tunnel Update

   A new NLRI SAFI (SD-WAN SAFI=74) is introduced within the
   MP_REACH_NLRI Path Attribute of [RFC4760] for advertising the
   detailed properties of the SD-WAN tunnels terminated at the edge
   node:

   +------------------+
   |    Route Type    | 2 octet
   +------------------+
   |     Length       | 2 Octet
   +------------------+
   |   Type Specific  |
   ~ Value (Variable) ~
   |                  |
   +------------------+

                           Figure 8: SD-WAN NLRI

   Where

   Route (NLRI) Type:  2 octet value to define the encoding of the rest
      of the SD-WAN the NLRI.

   Length:  2 octets of length expressed in bits as defined in
      [RFC4760].

   This document defines the following SD-WAN Route type:

   NLRI Route-Type = 1:  For advertising the detailed properties of the
      SD-WAN tunnels terminated at the edge, where the transport network
      port can be uniquely identified by a tuple of three values (Port-
      Local-ID, SD-WAN-Color, SD-WAN-Node-ID).  The SD-WAN NLRI Route-
      Type =1 has the following encoding:

Dunbar, et al.            Expires 17 April 2025                [Page 16]
Internet-Draft            SDWAN Edge Discovery              October 2024

           +------------------+
           |  Route Type = 1  | 2 octet
           +------------------+
           |     Length       | 2 Octet
           +------------------+
           |   Port-Local-ID  | 4 octets
           +------------------+
           |   SD-WAN-Color   | 4 octets
           +------------------+
           |  SD-WAN-Node-ID  | 4 or 16 octets
           +------------------+

                      Figure 9: SD-WAN NLRI Route Type 1

   Port-local-ID:  SD-WAN edge node Port identifier, which is locally
      significant.  If the SD-WAN NLRI applies to multiple WAN ports,
      this field is zero.

   SD-WAN-Color:  represents a group of tunnels, which correlate with
      the Color-Extended-community included in the client routes UPDATE.
      When a client route can be reached by multiple SD-WAN edges co-
      located at one site, the SD-WAN-Color can represent a group of
      tunnels terminated at those SD-WAN edges co-located at the site.
      If an SD-WAN-Color represents all the tunnels at a site, then the
      SD-WAN-Color effectively represents the site.

   SD-WAN Node ID:  The node's IPv4 or IPv6 address.

   Route Type values outside of 1 are out of scope for this document.

6.2.  SD-WAN-Hybrid Tunnel Encoding

   A new BGP Tunnel-Type SD-WAN-Hybrid (code point 25) indicates hybrid
   underlay tunnels.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Tunnel-Type=25(SD-WAN-Hybrid )| Length (2 Octets)             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Sub-TLVs                            |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                    Figure 10: SD-WAN Hybrid Value Field

Dunbar, et al.            Expires 17 April 2025                [Page 17]
Internet-Draft            SDWAN Edge Discovery              October 2024

   The valid SubTLVs for the Hybrid Tunnel type include subTLVs
   specified in [RFC9012] and the following SubTLVs specified in this
   document:

   *  Extended Port Attributes SubTLV

   *  IPsec SA-ID SubTLV

   *  IPSec SA Rekey SubTLV

   *  IPsec Public Key SubTLV

   *  IPsec SA Proposal SubTLV

   *  Simplified IPsec SA SubTLV

   The Extended Port Attributes are described in section 7, and the
   IPsec related SubTLVs are described in section 8.

7.  Extended Port Attribute Sub-TLV

   The SD-WAN Underlay NLRI is sent with a Tunnel Encapsulation
   Attribute with the Extended Port Attribute Sub-TLV advertises the
   properties associated with a public Internet-facing WAN port that
   might be behind NAT.  An SD-WAN edge node can query a STUN Server
   (Session Traversal of UDP through Network address translation
   [RFC8489]) to get the NAT properties, including the public IP address
   and the Public Port number, to pass to its peers.

   The location of a NAT device can be:

   *  Only the initiator is behind a NAT device.  Multiple initiators
      can be behind separate NAT devices.  Initiators can also connect
      to the responder through multiple NAT devices.

   *  Only the responder is behind a NAT device.

   *  Both the initiator and the responder are behind a NAT device.

   The initiator's address and/or responder's address can be dynamically
   assigned by an ISP or when their connection crosses a dynamic NAT
   device that allocates addresses from a dynamic address pool.

   As one SD-WAN edge can connect to multiple peers, the pair-wise NAT
   exchange as IPsec's IKE[RFC7296] is not efficient.  In the BGP
   Controlled SD-WAN, NAT properties for a WAN port are encoded in the
   Extended Port Attribute sub-TLV, which the following format:

Dunbar, et al.            Expires 17 April 2025                [Page 18]
Internet-Draft            SDWAN Edge Discovery              October 2024

        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |Type=65(extPort| ExtPort Length| Reserved      |I|O|R|R|R|R|R|R|
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | NAT Type      |  Encap-Type   |Trans networkID|     RD ID     |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                  Local  IP Address                            |
               32-bits for IPv4, 128-bits for Ipv6
                       ~~~~~~~~~~~~
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                  Local  Port                                  |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                Public IP                                      |
               32-bits for IPv4, 128-bits for Ipv6
                       ~~~~~~~~~~~~
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                Public Port                                    |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |               Extended SubSub-TLV                             |
       ~                                                               ~
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                 Figure 11: Extended Port Attribute Sub-TLV

   Where:

   *  Extended Port Attribute Type (=65): indicating it is the Extended
      Port Attribute SubTLV

   *  ExtPort Length: the length of the subTLV in octets (variable).

   *  Flags:

      -  I bit (CPE port address or Inner address scheme):

         o  If set to 0, indicate the inner (private) address is IPv4.

         o  If set to 1, indicates the inner address is IPv6.

      -  O bit (Outer address scheme):

         o  If set to 0, indicate the inner (private) address is IPv4.

         o  If set to 1, indicates the inner address is IPv6.

      -  R bits: reserved for future use.  Must be set to 0, and ignored
         upon reception.

Dunbar, et al.            Expires 17 April 2025                [Page 19]
Internet-Draft            SDWAN Edge Discovery              October 2024

   *  NAT Type: the NAT type can be one of the following values:

      -  1: without NAT ;

      -  2: 1-to-1 static NAT;

      -  3: Full Cone;

      -  4: Restricted Cone;

      -  5: Port Restricted Cone;

      -  6: Symmetric; or

      -  7: Unknown (i.e. no response from the STUN server).

      NAT type values outside of 1-7 are invalid for this SubTLV.

   *  Encap-Type: the supported encapsulation types for the port.

      -  Encap-Type=1: GRE;

      -  Encap-Type=2: VxLAN;

      Notes:

      -  The Encap-Type inside the Extended Port Attribute Sub-TLV is
         different from the RFC9012's BGP-Tunnel-Encapsulation type.
         The port can indicate the specific encapsulations, such as:

         o  If the IPsec-SA-ID subTLV or the IPsec SA detailed subTLVs
            (Nonce/publicKey/Proposal) are included in the SD-WAN-Hybrid
            tunnel, the Encap-Type indicates the encapsulation type
            within the IPsec payload.

         o  If the IPsec SA subTLVs are not included in the SD-WAN-
            Hybrid Tunnel, the Encap-Type indicates the encapsulation of
            the payload without IPsec encryption.

      -  Encapsulation types outside of GRE and VxLAN are outside of the
         scope of this specification.

   *  Transport Network ID: Central Controller assigns a global unique
      ID to each transport network.  Any value in this octet is valid

   *  RD ID: Routing Domain ID, need to be globally unique.  Any value
      in this octet is valid.

Dunbar, et al.            Expires 17 April 2025                [Page 20]
Internet-Draft            SDWAN Edge Discovery              October 2024

      -  Some SD-WAN deployment might have multiple levels, zones, or
         regions that are represented as logical domains.  Policies can
         govern if tunnels can be established across domains.  For
         example, a hub node can establish tunnels with different
         logical domains but the spoke nodes cannot establish tunnels
         with nodes in different domains.

   *  Local IP: The local (or private) IP address of the WAN port.

   *  Local Port: used by Remote SD-WAN edge node for establishing IPsec
      to this specific port.

   *  Public IP: The IP address after the NAT.  If NAT is not used, this
      field is set to all-zeros

   *  Public Port: The Port after the NAT.  If NAT is not used, this
      field is set to all-zeros.

   *  Extended SubSub-TLV: for carrying additional information about the
      underlay networks.

7.1.  Extended SubSub-TLV

   One Extended SubSub-TLVs is specified in this document: Underlay
   Network Transport SubSub-TLV

7.1.1.  Underlay Network Transport SubSub-TLV

   The Underlay Network Transport SubSub-TLV is an optional Sub-TLV to
   carry the WAN port connection types and bandwidth, such as LTE, DSL,
   Ethernet, etc.

   The format of this Sub-TLV is as follows:

        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | UnderlayType  |      Length   |      Reserved                 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |Connection Type|   Port Type   |        Port Speed             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   Figure 12: Underlay Network SubSub-TLV

   Where:

   Underlay Network Properties:  sub Type=66

Dunbar, et al.            Expires 17 April 2025                [Page 21]
Internet-Draft            SDWAN Edge Discovery              October 2024

   Length:  always 6 bytes

   Reserved:  2 octet of reserved bits.  It SHOULD be set to zero on
      transmission and MUST be ignored on receipt.

   Connection Type:  are listed below as:

      *  1 = Wired

      *  2 = WIFI

      *  3 = LTE

      *  4 = 5G

      *  Any value outside of 1-4 is outside the scope of this
         specification.

   Port Type:  Port type define as follows:

      *  1 = Ethernet

      *  2 = Fiber Cable

      *  3 = Coax Cable

      *  4 = Cellular

      *  Any value outside of values 1-4 are outside the scope of this
         specification.

   Port Speed:  The port seed is defined as 2 octet value.  The values
      are defined as Gigabit speed.  For example, a value of 1 would
      mean 1 gigabit.  The port speed of "0" is not valid.

   The connection types of equipment and port types will continue to
   grow with technology change.  Future specifications may specify
   additional connection types or port types.

8.  IPsec SA Property Sub-TLVs

   This section describes the SubTLVs that pass data regarding IPsec
   parameters for the Hybrid SD-WAN tunnel.  During set-up of the Hybrid
   SD-WAN tunnels, the IPsec parameters need to be securely passed to
   set-up secure association.  For hybrid SD-WAN tunnels, the IPsec
   security association for IPsec links may change to different security
   associations over time.

Dunbar, et al.            Expires 17 April 2025                [Page 22]
Internet-Draft            SDWAN Edge Discovery              October 2024

   The IPSec subTLVs supported by the Hybrid Tunnel type are: IPSec-SA-
   ID, IPsec SA Nounce, IPsec Public Key, IPsec Proposal, and Simplified
   IPSec SA.  The IPSec-SA-ID SubTLV provides a way to indicate the
   IPsec SA Identifiers (section 8.1) for pre-configured security
   association.  The other four SubTLVs provide different ways to pass
   details regarding IPsec security associations.  The IPsec SA Nounce
   passes Nounce and rekey counters for a Secure Association identified
   by IPsec SA Identifier (see section 8.2).  The IPSec Public Key
   SubTLV passes IPsec Public Key data with a time duration (see section
   8.3).  The IPsec Proposal SubTLV provides Transform attributes and
   Transform IDs (see section 8.4).  The Simplified IP SEC SA passes the
   information that identifies configuration for 2 keys (see section
   8.5).

   For a quick rotation between security associations, the SDWAN NLRI
   (port-id, color, node) can quickly distribute a switch to a set of
   new security association using the BGP Update message.  In this case,
   the BGP UPDATE message would like figure 10

       SDWAN NLRI
              Route-type: 1
              length: 12
              port-id - 0.0.0.0
              SD-WAN-Color - 0.0.0.1
              node-id - 2.2.2.2
       TEA:
        Tunnel TLV: (type: SD-WAN Hybrid)
              Tunnel Egress Endpoint SubTLV: 2.2.2.2
              IPsec-SA-ID SubTLV: 20, 30

              Figure 13: SD-WAN NLRI IPsec rotation in attack

8.1.  IPsec-SA-ID Sub-TLV

   IPsec-SA-ID Sub-TLV within the Hybrid Underlay Tunnel UPDATE
   indicates one or more pre-established IPsec SAs by using their
   identifiers, instead of listing all the detailed attributes of the
   IPsec SAs.

   Using an IPsec-SA-ID Sub-TLV not only greatly reduces the size of BGP
   UPDATE messages, but also allows the pairwise IPsec rekeying process
   to be performed independently.

   The following is the structure of the IPsec-SA-ID sub-TLV

Dunbar, et al.            Expires 17 April 2025                [Page 23]
Internet-Draft            SDWAN Edge Discovery              October 2024

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |Type=64 (IPsec-SA-ID subTLV)   |  Length (2 Octets)            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      IPsec SA Identifier #1                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      IPsec SA Identifier #2                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   ~                                                               ~
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      IPsec SA Identifier #n                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                       Figure 14: IPsec-SA-ID Sub-TLV

   where:

      length is the length of the subTLV in octets (must be on 4 octet
      boundary)

      The length is followed by a sequence of IPsec SA Identifiers of
      length 4 octets.

      -  IPSec SA Identifier #1 - is a 4 octet identifier for a IP
         Security association.

      -  IPSec SA Identifier #2 - is a 4 octet identifier for a IP
         Security association.

      -  IPSec SA Identifier #n - is a 4 octet identifier for a IP
         Security association.

8.2.  IPsec SA Rekey Counter Sub-TLV

   The IPsec SA Rekey Counter Sub-TLv provides the rekey counter for a
   security association (identified by IPsec SA Identifier).

   The format of this Sub-TLV is as follows:

Dunbar, et al.            Expires 17 April 2025                [Page 24]
Internet-Draft            SDWAN Edge Discovery              October 2024

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |   ID Length   |       Nonce Length            |I|   Flags     |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                             Rekey                             |
       |                            Counter                            |
       +---------------------------------------------------------------+
       |                IPsec SA Identifier                            |
       +---------------------------------------------------------------+
       |                                                               |
       ~                          Nonce Data                           ~
       |                                                               |
       +---------------------------------------------------------------+

              Figure 15: IPsec SA Rekey Counter Sub-TLV

   where:

      ID Length - is a 1 octet value indicating the length of SA-
      Identifer length.  This length should be 4 octets.

      Nonce length - is a 2 octet value indicate the length in octets of
      the Nonce Data.

      Flags: - is 1 octet field with the following form [I-R-R-R-R-R-R]

      -  I - indicates status of initial contact (1 = initial, 0 =
         follow-on) in the left most bit.

      -  R - Reserved bits - which are ignored upon reception and set to
         zero upon transmission.

      IPsec SA Identifier (IPSec-SA-ID) - identifies a specific IPsec
      SAs terminated at the edge.  The length is 4 octets.

      Nonce Data - a random or pseudo-random number for preventing
      replay attacks.  Its length is a multiple of 32 bits[RFC7296].

   Note:

   *  The IPsec-SA-ID may also refer to the values carried in the same
      TEA in the same Tunnel TLV (type SD-WAN Hybrid) as the IPsec SA
      Rekey SubTLV in either the IPsec Public Key SubTLV or the IPsec SA
      Proposal SubTLV.  The IPsec SA Rekey Counter, IPsec Public Key,
      and IPsec SA Proposal SubTLVs work together to create security
      associations.

Dunbar, et al.            Expires 17 April 2025                [Page 25]
Internet-Draft            SDWAN Edge Discovery              October 2024

   *  The IPsec-SA-ID may refer to information in another Tunnel TLV in
      the same TEA associated wiht the same BGP UPDATE message as the
      IPsec SA Rekey Counter sub-TLV.

   *  The IPsec-SA-ID can be used in the IPsec-SA-ID subTLV of a
      different BGP UPDATE message.

8.3.  IPsec Public Key Sub-TLV

   The format of this Sub-TLV is as follows:

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |   Diffie-Hellman Group Num    |          Reserved             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                                                               |
       ~                       Key Exchange Data                       ~
       |                                                               |
       +---------------------------------------------------------------+
       |                            Duration                           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                Figure 16: IPsec SA Public Key Sub-TLV

   where:

      Diffie-Hellman Group Num - is 2 octets long.  It identifies the
      Diffie-Hellman group in the Key Exchange Data was computed.
      Diffie-Hellman group numbers are discussed in IKEv2 [RFC7296]
      Appendix B and [RFC5114].

      The Key Exchange data - is constructed by copying one's Diffie-
      Hellman public value into the "Key Exchange Data" portion of the
      payload.  The length of the Diffie-Hellman public value is
      described for MOPD groups in [RFC7296] and for ECP groups in
      [RFC5903].

      Duration - is a 4-octet value specifying the life span of the
      Diffie-Hellman key usage.  The units of the life span depends on
      the Diffie-Hellman group.

8.4.  IPsec SA Proposal Sub-TLV

   The IPsec SA Proposal Sub-TLV is to indicate the number of Transform
   Sub-TLVs.

Dunbar, et al.            Expires 17 April 2025                [Page 26]
Internet-Draft            SDWAN Edge Discovery              October 2024

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    Transform Attr Length      |Transform Type |    Reserved.  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Transform ID              |          Reserved             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   ~                        Transform Attributes                   ~
   |                                                               |
   +---------------------------------------------------------------+

                    Figure 17: IPsec SA Proposal Sub-TLV

   where:

      Transform Attr Length is a 2 octet field indicating the length of
      the Transform Attributes field.

      Transform Type is a 1 octet field.  The Transform Type values are
      taken from is from Section 3.3.2 of [RFC7296] and [IKEV2IANA].
      Only the values ENCR, INTEG, and ESN are allowed.

      Reserved - is a 1 octet field reserved for future use.  The
      Reserved os ignored upon receipt and set to zero upon
      transmission.

      Transform ID - is a 2 octet identifer for the transform described
      by the transform attributes.

      Transform Attributes Sub-SubTLV are taken from the section 3.3.5
      of RFC7296.

8.5.  Simplified IPsec SA sub-TLV

   For a simple SD-WAN network with edge nodes supporting only a few
   pre-defined encryption algorithms, a simple IPsec sub-TLV can be used
   to encode the pre-defined algorithms, as below:

Dunbar, et al.            Expires 17 April 2025                [Page 27]
Internet-Draft            SDWAN Edge Discovery              October 2024

        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |IPsec-simType  |IPsecSA Length                 | Flag          |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Transform     | Mode          | AH algorithms |ESP algorithms |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |         ReKey Counter (SPI)                                   |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | key1 length   |         Key 1                                 ~
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | key2 length   |         Key 2                                 ~
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | nonce length  |         Nonce                                 ~
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |        Duration                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                   Figure 18: Siplified IPsec SA Sub-TLV

   Where:

   IPsec-SimType=70:  indicate the simplified IPsec SA attributes.

   IPsec-SA subTLV Length (2 Byte):  variable (25 or longer).

   Flags:  1 octet of flags.  None are defined at this stage.  Flags
      SHOULD be set to zero on transmission and MUST be ignored on
      receipt.

   Transform (1 Byte):
      *  Transform = 1 means AH,

      *  Transform = 2 means ESP, or

      *  Transform = 3 means AH+ESP.

      All other transform values are outside the scope of this document.

   IPsec Mode (1 byte):
      *  Mode = 1 indicates that the Tunnel Mode is used.

      *  Mode = 2 indicates that the Transport mode is used.

      All mode values besides 1 and 2 are outside the scope of this
      document.

   AH algorithms (1 byte):  AH authentication algorithms supported.  The

Dunbar, et al.            Expires 17 April 2025                [Page 28]
Internet-Draft            SDWAN Edge Discovery              October 2024

      values are specified by [IANA-AH].  Each SD-WAN edge node can
      support multiple authentication algorithms; send to its peers to
      negotiate the strongest one.

   ESP algorithms (1 byte):  the ESP algorithms supported.  Its values
      are specified by [IANA-ESP].  One SD-WAN edge node can support
      multiple ESP algorithms and send them to its peers to negotiate
      the strongest one.  The default algorithm is AES-256.

         When a node supports multiple authentication algorithms, the
         initial UPDATE needs to include the "Transform Sub-TLV"

   Rekey Counter (Security Parameter Index)):  4 octet which indicates
      the count for rekeying.

   key1 length:  is a 1 octet value indicating the IPsec public key 1
      length

   Public Key 1:  IPsec public key 1

   key2 length:  1 octet value indicating the IPsec public key 2 length

   Public Key 2:  IPsec public key 2

   none-length:  1 octet value indicating the Nonce key length

   Nonce:  IPsec Nonce

   Duration:  is a 4 octet value specifying the security association
      (SA) life span.

   The units on duration are specified by the deployment of the security
   association.  The operators managing these security association must
   have common units for Security Asssociation duration.

9.  Error handling

   The Error handling for SD-WAN VPN support has two components: error
   handling for Tunnel Encapsulation signaling (Encaps-EC and TEA) and
   the SD-WAN NLRI.  An SD-WAN NLRI, a Tunnel Encapsulation attribute
   MUST always accompany the SD-WAN NLRI.

9.1.  Error handling for the Tunnel Encapsulation Signaling

   The error handling for the tunnel encapsulation signaling (Encaps-EC
   and TEA) adheres to the error handling and validation specified by
   [RFC9012].

Dunbar, et al.            Expires 17 April 2025                [Page 29]
Internet-Draft            SDWAN Edge Discovery              October 2024

   The Tunnel encapsulation signaled with the client routes indicates
   the Egress endpoint via Next Hop in the Encaps-EC or the TEA SubTLV
   for Tunnel Egress Endpoints.  As indicated in sections 5.1 and 5.2,
   the SD-WAN Hybrid tunnel follows the validation section 6, 8, and 13
   from [RFC9012].

   The SD-WAN client routes associate the same NLRIs that [RFC9012]
   associates with the Encaps-EC and the TEA using the validation
   specified in [RFC9012] in sections 6, 8, and 13.  When the SD-WAN
   Hybrid Tunnel is associated with the SD-WAN NLRI, and all RFC9012
   validation rules in section 6, 8, and 13 are extended to apply to the
   SD-WAN NLRI.

   [RFC9012] contains the necessary detail to specify validation for the
   new SubTLVs present for the SD-WAN Tunnel type.  However, to aid
   users of this document the following recap of validation of [RFC9012]
   is provided below.  The validation from section 13 of [RFC9012]
   includes:

   *  Invalid tunnel type must be treat if the TLV was not present.

   *  A malformed subTLVs must be treated as an unrecognized subTLV
      except for Tunnel Egress Endpoint.  If Tunnel Egress Endpoint is
      malformed, the entire TLV must be ignored.

   *  Multiple incidents of Tunnel Egress Endpoint, Encapsulation, DS,
      UDP Destination Port, Embedded Label Handling, MPLS Label Stack,
      Prefixes-SID cause the first incident of these subTLVs to be
      utilized.  Subsequent TLVs after the first one per type are
      ignored (per RFC9012), but propogated.

   *  If a subTLV is meaningless for a tunnel type, the subTLV is
      ignored, but the subTLV is not considered malformed or removed
      from the Tunnel Attribute propagated with the NLRI.

   For SD-WAN client routes with a TEA with a SD-WAN Hybrid Tunnel type,
   the Extended Port subTLV and the IPSec SubTLVs (IPsec SA-ID, IPSec
   nonce, IPSec Public Key, IPsec Proposal, and Simplified IPsec SA) are
   meaningful, but may be rarely sent.

   For SD-WAN NLRI underlay routes, the the Extended Port subTLV and the
   IPSec SubTLVs (IPsec SA-ID, IPSec nonce, IPSec Public Key, IPsec
   Proposal, and Simplified IPsec SA) are valid and meaningful.
   Incorrect fields within any of these 5 TLVs or subSubTLVs within the
   TLVs should cause the subTLV to be treated as malformed SubTLV.  Per
   [RFC9012], a malformed subTLV is treated as an unrecognized subTLV.
   Multiple copies of each SubTLV may be included in a single TLV.

Dunbar, et al.            Expires 17 April 2025                [Page 30]
Internet-Draft            SDWAN Edge Discovery              October 2024

9.2.  Error Handling for NLRI

   The SD-WAN NLRI [AFI 1/SAFI = 74] utilizes a route type field to
   describe the format of the NLRI.  This specification only allows an
   NLRI with a type value of 1.  An NLRI with a type of field of another
   value is ignored and not processed.  The implementation MAY log an
   error upon a reception of an type value outside of Route Type 1.
   Error handling for the SD-WAN NLRI also adheres to the BGP Update
   error handling specified in [RFC7606].

   Section 6.1 specifies that Route Type 1 has a tuple of (Port-Local-
   ID, SD-WAN-Color, SD-WAN-Node-ID).  Port-Local-ID may be zero if the
   NLRI applies to multiple ports.  The BGP Peer receiving the NLRI must
   have pre-configured inbound filters to set the preference for the SD-
   WAN NLRI tuple.

   Since a Port-Local-ID value of zero indicates the NLRI applies to
   multiple ports, it is possible to have the following NLRI within a
   packet (or received in multiple packets):

      Port-Local-ID (0), SD-WAN-Color (10), SD-WAN-Node-ID (2.2.2.2),

      Port-Local-ID (0), SD-WAN-Color (20), SD-WAN-Node-ID (2.2.2.2),
      and

      Port-Local-ID (0), SD-WAN-Color (30), SD-WAN-Node-ID (2.2.2.2).

   These NLRI may simply indicate that there are three groups of tunnels
   for SD-WAN-Node-ID (2.2.2.2) assigned three colors.  For example,
   these tunnels could represent three types of gold, silver and bronze
   network service.

   The local policy configuration in the BGP peer receiving this NLRI
   must determine the validity of the route based on policy.  Local
   configuration and policy must be carefully constrain the SD-WAN-NLRI,
   tunnels, and IPsec security associations in to create a "walled
   garden".

   In the future, other proposals for a SD-WAN NLRI may specify a
   different route type.  Those proposals must specify the following:

      validation for new Route Type in the SD-WAN-NLRI, and

      how the new Route Type interacts with the Route Type 1.

Dunbar, et al.            Expires 17 April 2025                [Page 31]
Internet-Draft            SDWAN Edge Discovery              October 2024

9.3.  SDWAN NLRI and Tunnel Encapsulation Attribute

   The SD-WAN NLRI (AFI=1/SAFI=74) must be paired with Tunnel
   Encapsulation attribute with a tunnel TLV for tunnel type SD-WAN-
   Hybrid.  If the SD-WAN NLRI exist in an BGP UPDATE without a Tunnel
   Encapsulation Attribute (TEA) with a tunnel TLV for tunnel type SD-
   WAN-Hybrid, the NLRI is considered malformed and Treat-as-withdraw
   approach (per RFC7606).

   The SD-WAN NLRI should not be paired with Encapsulation Extended
   Community.  If an SD-WAN NLRI is paried Encapsulation Extended
   Community rather than a Tunnel Encapsulation Attribute, the SD-WAN
   NLRI is considered malformed and Treat-as-withdraw approach (per
   [RFC7606]) should be used.

10.  Manageability Considerations

   Unlike MPLS VPN whose PE nodes are all controlled by the network
   operators, SD-WAN edge nodes can be installed anywhere, in shopping
   malls, in 3rd party Cloud DCs, etc.

   It is very important to ensure that client routes advertisement from
   an SD-WAN edge node are legitimate.  The RR needs to ensure the SD-
   WAN Hybrid Tunnels and routes run over the appropriate Security
   associations.

10.1.  Detecting Misaligned Tunnels

   It is critical that the Hybrid SD-WAN Tunnel have correctly forward
   traffic based on the local policy on the client routes, the tunnel
   egress and tunnel ingress, and the security association.  The RR
   reflector and the BGP peer must check that the client routes, tunnel
   egress, tunnel ingress, and security associations align with expected
   values for a tunnel.

10.2.  IPsec Attributes Mismatch

   Each BGP peer (e.g. a C-PE) advertises a SD-WAN SAFI Underlay NLRI to
   the other BGP peers via a BGP Route Reflector to establish pairwise
   IPsec Security Associations (SA) between itself and other remote BGP
   Peers.  During the SD-WAN SAFI NLRI advertisement, the BGP Peer
   originating may pass information about security asssociation in one
   of three forms:

   *  an identifier for a pre-configured and established IPsec Security
      Association,

Dunbar, et al.            Expires 17 April 2025                [Page 32]
Internet-Draft            SDWAN Edge Discovery              October 2024

   *  a simplified set of security parameters for setting up a IPsec
      Security association (Transform, IPsec Mode, AH and ESP
      Algorithms, rekey counter, 2 public keys, nonce, and duration of
      security association), or

   *  a flexible set of security parameters where Nonce, Public Key, and
      SA Proposal are uniquely specified.

   For existing IPsec Security associations, the receiving BGP peer can
   simply utilize one of these existing security associations to pass
   data.  If multiple IPsec associations are pre-configured, the local
   policy on the SD-WAN Edge Node may may help select which security
   association is chosen for the SD-WAN Hybrid Tunnel.

   If the receiving and originating BGP peer engage in a set-up for the
   IPsec security associations for the link within the SD-WAN Hybrid
   tunnel, IPsec mechansims require that there are matching IPsec
   transforms.  Without common IPsec transforms, the IPsec set-up
   process cannot operate.

10.2.1.  SD-WAN Hybrid Tunnel Mechanisms for Passing IPSec Security
         Association Info

   The TEA passes in the Tunnel TLV for the SD-WAN Hybrid Tunnel these
   three sets of information in the following subTLVs:

   IPsec-SA-ID:  passes the previous configured (pre-configured or
      generated) IPsec SA identifiers.

   Simplified IPsec SA SubTLV:  specifies a simplified set of
      information upon which to set-up the IPsec security associations
      for the tunnel.

   A sequence of the following SubTLVs:  IPsec SA Rekey Counter SubTLV,
      IPsec Public Key SubTLV, and a IPSec Proposal SubTLV.
      Configuration on the local node uses this information and any
      information in the underlay to create security associations.

   The BGP Peer's need to send the IPSec SA attributes received on the
   SD-WAN NLRI in the TEA between the local and remote WAN ports.  If
   there is a match on the SA Attributes between the two ports, the
   IPSec Tunnel is established.  If there is a mismtach on the SA
   Attributes, no IPsec Tunnel is established.

Dunbar, et al.            Expires 17 April 2025                [Page 33]
Internet-Draft            SDWAN Edge Discovery              October 2024

   The C-PE devices do not try to negotiate the base IPSec-SA parameters
   between the local and the remote ports in the case of simple IPSec SA
   exchange or the Transform sets between local and remote ports.  If
   there is a mismatch in the IPsec SA, then no IPsec Tunnel is created.
   If there is a mismatch on the Transform sets in the case of full-set
   of IPSec SA Sub-TLVs, no tunnel is created.

10.2.2.  Example creation of IPsec Security Association over SD-WAN
         Hybrid tunnel

   This section provides one example of how IPsec Security associationes
   are created over the SD-WAN Hybrid tunnel.  Figure 1 in Section 3
   shows an establish an IPsec Tunnel being created between C-PE1 and
   C-PE2 WAN Ports A2 and B2 (A2: 192.10.0.10 - B2:192.0.0.1).

   To create this tunnel C-PE1 needs to advertise the following
   attributes for establishing the IPsec SA:

   *  NextHop: 192.10.0.10

   *  SD-WAN Node ID: 1.1.1.1

   *  SD-WAN-Site-ID: 15.0.0.2

   *  Tunnel Encap Attr (Type=SD-WAN) -

      -  Extended Port Attribute Sub-TLV containing

         o  Transport SubSubTLV - with information on ISP3.

      -  IPSec information for detailed information about the ISP3

      -  IPsec SA Rekey Counter Sub-TLV,

      -  IPsec SA Public Key Sub-TLV,

      -  Proposal Sub-TLV (type = ENCR, transform ID = 1)

         o  type: ENCR

         o  Transform ID: 1

         o  Tranform attributes = trans 1 [from RFC7296]

   C-PE2 needs to advertise the following attributes for establishing
   IPsec SA:

   Next Hop:  192.0.0.1

Dunbar, et al.            Expires 17 April 2025                [Page 34]
Internet-Draft            SDWAN Edge Discovery              October 2024

   SD-WAN Node ID:  2.2.2.2

   SD-WAN-Site-ID:  15.0.0.2

   Tunnel Encap Attr (Type=SD-WAN)
      *  Extended Port Attribute SubTLV

         -  Transport SubSubTLV - with information on ISP3.

      *  IPsec SA Rekey Counter Sub-TLV,

      *  IPsec SA Public Key Sub-TLV,

      *  IPSec Proposal Sub-TLV with

         -  transform type: ENCR

         -  Transform ID = 1

         -  Transform attributes = trans 2

   As there is no matching transform between the WAN ports A2 and B2 in
   C-PE1 and C-PE2, respectively, no IPsec Tunnel will be established.

11.  Security Considerations

   The document describes the encoding for SD-WAN edge nodes to
   advertise its properties to their peers to its RR, which propagates
   to the intended peers via untrusted networks.

   The secure propagation is achieved by secure channels, such as TLS,
   SSL, or IPsec, between the SD-WAN edge nodes and the local controller
   RR.

   SD-WAN edge nodes might not have secure channels with the RR.  In
   this case, BGP connection has be established over IPsec or TLS.

   This document describes the encoding for SD-WAN edge nodes to
   advertise their properties to their peers via their respective Route
   Reflector (RR), which then propagates the information to the intended
   peers.  SD-WAN edge nodes to advertise its properties to their peers
   via a secure connection (TLS, SSL, or IPsec) to the RR which
   propagates to the intended peers over a secure connection (TLS, SSL,
   or IPsec).

   In a walled garden SD-WAN deployment where all SD-WAN edges and the
   central controller are under one administrative control and the
   network operates within a closed environment, the threat model is

Dunbar, et al.            Expires 17 April 2025                [Page 35]
Internet-Draft            SDWAN Edge Discovery              October 2024

   primarily on internal threats, misconfigurations, and localized
   physical risks.  Unauthorized physical access to SD-WAN edge devices
   in remote locations is a concern.  Such access might allow attackers
   to compromise the edge devices and potentially manipulate the
   advertised Client prefixes with VPN IDs (or Route Targets) that do
   not belong to them.  This can lead to unauthorized data interception
   and traffic redirection.

   Therefore, it is necessary to ensure physical security controls are
   in place at remote locations, including locks, surveillance, and
   access controls.  Additionally, the RR needs to verify the BGP
   advertisements from each SD-WAN edge to ensure that their advertised
   VPN IDs (or Route Targets) are truly theirs.  This verification helps
   prevent unauthorized advertisement of prefixes and ensures the
   integrity of the routing information within the SD-WAN environment.
   Ensuring secure communication between SD-WAN edge nodes and the
   central controller within a walled garden deployment is crucial.  It
   is essential to utilize secure communication channels such as TLS or
   IPsec for all communications between edge nodes and the controller.

12.  IANA Considerations

12.1.  Hybrid (SD-WAN) Overlay SAFI

   IANA has assigned SAFI = 74 as the Hybrid (SD-WAN) SAFI.

12.2.  Tunnel Encapsulation Attribute Type

   IANA is requested to assign a type from the BGP Tunnel Encapsulation
   Attribute Tunnel Types as follows [RFC8126]:

    Value   Description    Reference
   -----   ------------   ---------
    25       SD-WAN-Hybrid   (this document)

12.3.  Tunnel Encapsulation Attribute Sub-TLV Types

   IANA is requested to assign the following sub-Types in the BGP Tunnel
   Encapsulation Attribute Sub-TLVs registry:

Dunbar, et al.            Expires 17 April 2025                [Page 36]
Internet-Draft            SDWAN Edge Discovery              October 2024

      Value   Type Description            Reference     Section
      -----   -----------------------     ------------- -------
       64  IPSEC-SA-ID Sub-TLV            This document  8.1
       65  Extended Port Property Sub-TLV This document  7.0
       66  Underlay Transport Sub-TLV     This document  7.1
       67  IPsec SA Rekey Counter Sub-TLV         This document  8.2
       68  IPsec Public Key Sub-TLV       This document  8.3
       69  IPsec SA Proposal Sub-TLV      This document  8.4
       70  Simplified IPsec SA sub-TLV    This document  8.5

13.  References

13.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC4271]  Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
              Border Gateway Protocol 4 (BGP-4)", RFC 4271,
              DOI 10.17487/RFC4271, January 2006,
              <https://www.rfc-editor.org/info/rfc4271>.

   [RFC4360]  Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended
              Communities Attribute", RFC 4360, DOI 10.17487/RFC4360,
              February 2006, <https://www.rfc-editor.org/info/rfc4360>.

   [RFC4364]  Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
              Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February
              2006, <https://www.rfc-editor.org/info/rfc4364>.

   [RFC4456]  Bates, T., Chen, E., and R. Chandra, "BGP Route
              Reflection: An Alternative to Full Mesh Internal BGP
              (IBGP)", RFC 4456, DOI 10.17487/RFC4456, April 2006,
              <https://www.rfc-editor.org/info/rfc4456>.

   [RFC4760]  Bates, T., Chandra, R., Katz, D., and Y. Rekhter,
              "Multiprotocol Extensions for BGP-4", RFC 4760,
              DOI 10.17487/RFC4760, January 2007,
              <https://www.rfc-editor.org/info/rfc4760>.

   [RFC7296]  Kaufman, C., Hoffman, P., Nir, Y., Eronen, P., and T.
              Kivinen, "Internet Key Exchange Protocol Version 2
              (IKEv2)", STD 79, RFC 7296, DOI 10.17487/RFC7296, October
              2014, <https://www.rfc-editor.org/info/rfc7296>.

Dunbar, et al.            Expires 17 April 2025                [Page 37]
Internet-Draft            SDWAN Edge Discovery              October 2024

   [RFC7606]  Chen, E., Ed., Scudder, J., Ed., Mohapatra, P., and K.
              Patel, "Revised Error Handling for BGP UPDATE Messages",
              RFC 7606, DOI 10.17487/RFC7606, August 2015,
              <https://www.rfc-editor.org/info/rfc7606>.

   [RFC8126]  Cotton, M., Leiba, B., and T. Narten, "Guidelines for
              Writing an IANA Considerations Section in RFCs", BCP 26,
              RFC 8126, DOI 10.17487/RFC8126, June 2017,
              <https://www.rfc-editor.org/info/rfc8126>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC8277]  Rosen, E., "Using BGP to Bind MPLS Labels to Address
              Prefixes", RFC 8277, DOI 10.17487/RFC8277, October 2017,
              <https://www.rfc-editor.org/info/rfc8277>.

   [RFC8489]  Petit-Huguenin, M., Salgueiro, G., Rosenberg, J., Wing,
              D., Mahy, R., and P. Matthews, "Session Traversal
              Utilities for NAT (STUN)", RFC 8489, DOI 10.17487/RFC8489,
              February 2020, <https://www.rfc-editor.org/info/rfc8489>.

   [RFC9012]  Patel, K., Van de Velde, G., Sangli, S., and J. Scudder,
              "The BGP Tunnel Encapsulation Attribute", RFC 9012,
              DOI 10.17487/RFC9012, April 2021,
              <https://www.rfc-editor.org/info/rfc9012>.

13.2.  Informative References

   [IANA-AH]  IANA, "IANA-AH", <https://www.iana.org/assignments/isakmp-
              registry/isakmp-registry.xhtml#isakmp-registry-9>.

   [IANA-ESP] IANA, "IANA-ESP", <https://www.iana.org/assignments/
              isakmp-registry/isakmp-registry.xhtml#isakmp-registry-9>.

   [Net2Cloud]
              L. Dunbar, A Malis, C. Jacquenet, M. Toy and K. Majumdar,
              "Dynamic Networks to Hybrid Cloud DCs: Problem Statement
              and Mitigation Practice", September 2023,
              <https://datatracker.ietf.org/doc/draft-ietf-rtgwg-
              net2cloud-problem-statement/>.

   [RFC5114]  Lepinski, M. and S. Kent, "Additional Diffie-Hellman
              Groups for Use with IETF Standards", RFC 5114,
              DOI 10.17487/RFC5114, January 2008,
              <https://www.rfc-editor.org/info/rfc5114>.

Dunbar, et al.            Expires 17 April 2025                [Page 38]
Internet-Draft            SDWAN Edge Discovery              October 2024

   [RFC5903]  Fu, D. and J. Solinas, "Elliptic Curve Groups modulo a
              Prime (ECP Groups) for IKE and IKEv2", RFC 5903,
              DOI 10.17487/RFC5903, June 2010,
              <https://www.rfc-editor.org/info/rfc5903>.

   [RFC9016]  Varga, B., Farkas, J., Cummings, R., Jiang, Y., and D.
              Fedyk, "Flow and Service Information Model for
              Deterministic Networking (DetNet)", RFC 9016,
              DOI 10.17487/RFC9016, March 2021,
              <https://www.rfc-editor.org/info/rfc9016>.

Appendix A.  Acknowledgments

   Acknowledgements to Wang Haibo, Shunwan Zhuang, Hao Weiguo, and
   ShengCheng for implementation contribution.  Many thanks to Yoav Nir,
   Graham Bartlett, Jim Guichard, John Scudder, and Donald Eastlake for
   their review and suggestions.

Contributors

   Below is a list of other contributing authors:

   *  Gyan Mishra,

   *  Shunwan Zhuang,

   *  Sheng Cheng, and

   *  Donald Eastlake.

Authors' Addresses

   Linda Dunbar
   Futurewei
   Dallas, TX,
   United States of America
   Email: ldunbar@futurewei.com

   Kausik Majumdar
   Microsoft Azure
   California,
   United States of America
   Email: kmajumdar@microsoft.com

Dunbar, et al.            Expires 17 April 2025                [Page 39]
Internet-Draft            SDWAN Edge Discovery              October 2024

   Susan Hares
   Hickory Hill Consulting
   United States of America
   Email: shares@ndzh.com

   Robert Raszuk
   Arrcus
   United States of America
   Email: robert@raszuk.net

   Venkit Kasiviswanathan
   Arista
   United States of America
   Email: venkit@arista.com

Dunbar, et al.            Expires 17 April 2025                [Page 40]