BGP Usage for SD-WAN Overlay Networks
draft-ietf-bess-bgp-sdwan-usage-20

Network Working Group                                        L. Dunbar
Internet Draft                                               Futurewei
Intended status: Informational                              A. Sajassi
Expires: March 7, 2024                                         Cisco
                                                             J. Drake
                                                           Independet
                                                             B. Najem
                                                          Bell Canada
                                                      February 7, 2024

                  BGP Usage for SD-WAN Overlay Networks
                   draft-ietf-bess-bgp-sdwan-usage-20

Abstract
   The document discusses the usage and applicability of BGP as the
   control plane for multiple scenarios of SD-WAN (Software Defined
   WAN) overlay networks. The document aims to demonstrate how the
   BGP-based control plane is used for large-scale SD-WAN overlay
   networks with little manual intervention.

   SD-WAN edge nodes are commonly interconnected by multiple types of
   underlay networks owned and managed by different network
   providers.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current
   Internet-Drafts is at
   https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time.  It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

xxx, et al.                                                   [Page 1]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with
   respect to this document. Code Components extracted from this
   document must include Simplified BSD License text as described in
   Section 4.e of the Trust Legal Provisions and are provided without
   warranty as described in the Simplified BSD License.

Table of Contents

   1. Introduction...................................................3
   2. Conventions used in this document..............................4
   3. Use Case Scenario Description and Requirements.................5
      3.1. Requirements..............................................5
         3.1.1. Supporting SD-WAN Segmentation.......................5
         3.1.2. Client Service Requirement...........................6
         3.1.3. SD-WAN Traffic Segmentation..........................6
         3.1.4. Zero Touch Provisioning..............................7
         3.1.5. Constrained Propagation of SD-WAN Edge Properties....8
      3.2. Scenario #1: Homogeneous Encrypted SD-WAN.................9
      3.3. Scenario #2: Differential Encrypted SD-WAN...............10
      3.4. Scenario #3: Private VPN PE based SD-WAN.................12
   4. Provisioning Model............................................13
      4.1. Client Service Provisioning Model........................13
      4.2. Policy Configuration.....................................14
      4.3. IPsec Related Parameters Provisioning....................14
   5. BGP Controlled SD-WAN.........................................14
      5.1. Why BGP as Control Plane for SD-WAN?.....................14
      5.2. BGP Walk Through for Homogeneous Encrypted SD-WAN........15
      5.3. BGP Walk Through for Differential Encrypted SD-WAN.......17
      5.4. BGP Walk Through for Application Flow-Based Segmentation.18
      5.5. Benefit of Using Recursive Next Hop Resolution...........20
   6. SD-WAN Forwarding Model.......................................20
      6.1. Forwarding Model for Homogeneous Encrypted SD-WAN........21
         6.1.1. Network and Service Startup Procedures..............21
         6.1.2. Packet Walk-Through.................................21
      6.2. Forwarding Model for Hybrid Underlay SD-WAN..............22
         6.2.1. Network and Service Startup Procedures..............22

Dunbar, et al.                                                [Page 2]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

         6.2.2. Packet Walk-Through.................................22
      6.3. Forwarding Model for PE based SD-WAN.....................24
         6.3.1. Network and Service Startup Procedures..............24
         6.3.2. Packet Walk-Through.................................24
   7. Manageability Considerations..................................25
   8. Security Considerations.......................................25
   9. IANA Considerations...........................................26
   10. References...................................................26
      10.1. Normative References....................................26
      10.2. Informative References..................................27
   11. Acknowledgments..............................................28

1. Introduction

   Software Defined Wide Area Network (SD-WAN), as described in
   [MEF70.1], provides overlay connectivity services that optimize
   the transport of IP packets over one or more underlay connectivity
   services by recognizing applications and determining forwarding
   behavior by applying policies to them. Here are some of the main
   characteristics of "SD-WAN" networks:

     - Transport Augmentation, referring to utilizing paths over
       different underlay networks. There are often multiple parallel
       overlay paths between any two SD-WAN edges; some are private
       networks over which traffic can traverse with or without
       encryption; others require encryption, e.g., over untrusted
       public networks.
     - Instead of all traffic hauled to corporate headquarters for
       centralized policy control, direct Internet breakout from
       remote branch offices is allowed.
     - Some traffic can be steered onto specific overlay paths based
       on the packets matching a predefined condition instead of
       destination IP addresses. The matching condition can be one or
       multiple fields of the IP header of the packets. More detailed
       attributes for steering traffic are described in the Table7
       and Table 8 of [MEF70.1]. Using IPv6 [RFC8200] packets as an
       example, the Flow Label, the source address, a specific
       extension header field, or a combination of multiple IP header
       fields can be used to steer traffic.
     - The traffic forwarding can also be based on specific
       performance criteria (e.g., packet delay, packet loss, jitter)
       to provide better performance by choosing the underlay that
       meets or exceeds the specified policies.

Dunbar, et al.                                                [Page 3]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   This document describes the SD-WAN Use Cases, Provisioning, and
   forwarding. It outlines the utilization of BGP as a control plane
   for SD-WAN overlay networks and services to address some of the
   problems described in [Net2Cloud-Problem]. It's important to
   distinguish the BGP instance as the control plane for SD-WAN
   overlay from the BGP instances governing the underlay networks.
   Additionally, it assumes the existence of a secure channel between
   the SD-WAN controller and the SD-WAN edges for BGP Control Plane
   communication.

2. Conventions used in this document

   Cloud DC:   Third party data centers that host applications and
               workloads owned by different organizations or tenants.

   Controller: Used interchangeably with SD-WAN controller to manage
               SD-WAN overlay networks in this document. In the
               specific context of BGP-controlled SD-WAN, the
               controller functions as an integral component of the
               BGP Route Reflector.

   CPE:        Customer Premise Equipment

   Homogeneous Encrypted SD-WAN: An SD-WAN network in which all
               traffic to/from the SD-WAN edges are carried by IPsec
               tunnels regardless of underlay networks. I.e., the
               client traffic is carried by IPsec tunnel even over
               MPLS private networks.

   NSP:        Network Service Provider.

   PE:         Provider Edge

   SD-WAN Edge Node: An edge node, which can be physical or virtual,
               maps the attached clients' traffic to the wide area
               network (WAN) overlay tunnels.

   SD-WAN:     An overlay connectivity service that optimizes the
               transport of IP packets over one or more Underlay
               connectivity services by recognizing applications and
               determining forwarding behavior by applying policies
               to them. [MEF-70.1].

Dunbar, et al.                                                [Page 4]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   SD-WAN IPsec SA: IPsec Security Association between two WAN ports
               of the SD-WAN nodes or between two SD-WAN nodes.

   SD-WAN over Hybrid Underlay Networks: SD-WAN over Hybrid Underlay
               Networks typically have edge nodes utilizing bandwidth
               resources from different types of underlay networks,
               some being private networks and others being public
               Internet.

   WAN Port:   A Port or Interface facing a Network Service Provider
               (NSP), with an address allocated by the NSP.

   C-PE:       SD-WAN Edge node, which can be Customer Premises
               Equipment (CPE) for customer-managed SD-WAN, or
               Provider Edge (PE) for provider-managed SD-WAN
               services.

   ZTP:        Zero Touch Provisioning

3. Use Case Scenario Description and Requirements

   This section describes some essential requirements for SD-WAN
   networks and several SD-WAN scenarios used by the subsequent
   sections to explain how the BGP control plane is applied.

3.1. Requirements

3.1.1. Supporting SD-WAN Segmentation

   "SD-WAN Segmentation" is a frequently used term in SD-WAN
   deployment, referring to policy-driven network partitioning. An
   SD-WAN segment is a virtual private network (SD-WAN VPN)
   consisting of a set of edge nodes interconnected by tunnels, such
   as IPsec tunnels and MPLS VPN tunnels.

   This document assumes that SD-WAN VPN configuration on PE devices
   will, as with MPLS VPN, make use of VRFs. Additionally, it assumes
   that one SD-WAN VPN can be mapped to one or multiple virtual
   topologies governed by the SD-WAN controller's policies.

   When using BGP for SD-WAN, the Client Route UPDATE is the same as
   MPLS VPN. Route Target in the BGP Extended Community can be used
   to differentiate the routes belonging to different SD-WAN VPNs.

Dunbar, et al.                                                [Page 5]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   As SD-WAN is an overlay network arching over multiple types of
   networks, MPLS L2VPN[RFC4761][RFC4762]/L3VPN[RFC4364][RFC4659] or
   pure L2 underlay can continue using the VPN ID (Virtual Private
   Network Identifier), VN-ID (Virtual Network Identifier), or VLAN
   (Virtual LAN) in the data plane to differentiate packets belonging
   to different SD-WAN VPNs. For packets carried by an IPsec tunnel,
   the IPsec tunnel's inner encapsulation header can have the SD-WAN
   VPN Identifier to distinguish the packets belonging to different
   SD-WAN VPNs.

3.1.2. Client Service Requirement

   The client service requirements describe the SD-WAN edge's ports,
   also known as SD-WAN client interface, which connect the client
   network to the SD-WAN service.

   The SD-WAN client interface should support IPv4 & IPv6 address
   prefixes and Ethernet (as described in [IEEE802.3] standard).

   It is worth noting that the "SD-WAN client interface" is called
   SD-WAN UNI (User Network Interface) in [MEF 70.1] with a set of
   attributes (described in Section 11 in MEF 70.1); these attributes
   (in MEF 70.1) describe the expected behavior and requirements to
   support the connectivity to the client network.

   The client service should support the SD-WAN UNI service
   attributes at the SD-WAN edge as described in MEF 70.1, Section
   11.

3.1.3. SD-WAN Traffic Segmentation

   SD-WAN Traffic Segmentation enables the separation of the traffic
   based on the business and the security needs of different user
   groups and/or application requirements. Each user group and/or
   application may need different isolated topologies and/or policies
   to fulfill the business requirements.

   For example, a retail business requires the point-of-sales (PoS)
   application to be on a different topology from other applications.
   The PoS application is routed only to the payment processing
   entity at a hub site; other applications can be routed to all
   other sites.

   The traffic from the PoS application follows a tree topology
   (denoted as "----" in the figure below), whereas other traffic can
   follow a multipoint-to-multipoint topology (denoted as "===").

Dunbar, et al.                                                [Page 6]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

                              +--------+
              Payment traffic |Payment |
                +------+----+-+gateway +------+----+-----+
               /      /     | +----+---+      |     \     \
              /      /      |      |          |      \     \
           +-+--+  +-+--+  +-+--+  |   +-+--+  +-+--+  +-+--+
           |Site|  |Site|  |Site|  |   |Site|  |Site|  |Site|
           | 1  |  |  2 |  | 3  |  |   |4   |  |  5 |  | 6  |
           +--+-+  +--+-+  +--|-+  |   +--|-+  +--|-+  +--|-+
              |       |       |    |      |       |       |
            ==+=======+=======+====+======+=======+=======+===
            Multi-point connection for non-payment traffic

   Another example is an enterprise that wants to isolate the traffic
   from different departments, with each department having its unique
   topology and policy. The HR department may need to access specific
   applications that are not accessible by the engineering
   department. Also, contractors may have limited access to the
   enterprise resources.

3.1.4. Zero Touch Provisioning

   SD-WAN Zero-Touch Provisioning (ZTP) allows devices to be
   configured and provisioned centrally without the need to dispatch
   a network engineer to the field to configure the remote devices.
   When an SD-WAN edge is installed at a remote location, ZTP
   automates follow-up steps, including updates to the OS, software
   version, and configuration, before client traffic is forwarded.
   The ZTP can bootstrap a remote SD-WAN edge and establish a secure
   connection to the local SD-WAN Controller, making it convenient to
   add or delete an SD-WAN edge node (virtual or physical). ZTP for a
   remote SD-WAN edge usually includes the following steps:

     - The SD-WAN edge's customer information and its device
     identifier, such as the device serial number, are added to the
     SD-WAN Central Controller.

     - Upon power-up, the SD-WAN edge can establish the transport
     layer secure connection [BCP195] to its controller, whose URL
     (or IP address) and credential for connection request can be
     preconfigured on the edge device by the manufacture, external
     USB or secure Email given to the installer.

     - The SD-WAN Controller authenticates the ZTP request from the
     remote SD-WAN edge with its configurations. Once the

Dunbar, et al.                                                [Page 7]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

     authentication is successful, it can designate a local network
     controller near the SD-WAN edge to pass down the initial
     configurations via the secure channel. The local network
     controller manages and monitors the communication policies for
     traffic to/from the edge node.

3.1.5. Constrained Propagation of SD-WAN Edge Properties

   For an SD-WAN Edge to establish an IPsec tunnel to another one and
   announce the attached client routes to each other, both edges need
   to know each other's network properties, such as the IP addresses
   of the WAN ports, the edges' loopback addresses, the attached
   client routes, the supported encryption methods, etc.

   One SD-WAN edge node may only be authorized to communicate with a
   small number of other SD-WAN edge nodes. In this circumstance, the
   property of the SD-WAN edge node should not be propagated to other
   nodes that are not authorized to communicate. But a remote SD-WAN
   edge node, upon powering up, may not have the right policies to
   know which peers are authorized to communicate. Therefore, SD-WAN
   deployment needs to have a central point to distribute the
   properties of an SD-WAN edge node to its authorized peers.

   BGP is well suited for this purpose. RFC4684 has specified the
   procedure to constrain the distribution of BGP UPDATE to only a
   subset of nodes. Each edge node informs the Route-Reflector (RR)
   [RFC4456] on its interested SD-WAN VPNs. The RR only propagates
   the BGP UPDATE from an edge to others within the same SD-WAN VPN.

   As the connection between an SD-WAN edge and its RR can be over an
   insecured network, an SD-WAN edge must establish a secure
   connection to its designated RR upon power-up. The BGP UPDATE
   messages must be sent over the secure channel to the RR.

Dunbar, et al.                                                [Page 8]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

                              +---+
          Authorized Peers G1 |RR |   Authorized Peer G2
                +======+====+=+   +======+====+=====+
               /      /     | +---+      |     \     \
              /      /      |            |      \     \
           +-+--+  +-+--+  +-+--+      +-+--+  +-+--+  +-+--+
           |C-PE|  |C-PE|  |C-PE|      |C-PE|  |C-PE|  |C-PE|
           | 1  |  |  2 |  | 3  |      |4   |  |  5 |  | 6  |
           +----+  +----+  +----+      +----+  +----+  +----+
                Tenant 1                   Tenant 2
          Figure 1: Authorized Peer Groups managed by RR

   Tenant separation is achieved by the SD-WAN VPN identifiers
   represented in the control plane and data plane, respectively.

3.2. Scenario #1: Homogeneous Encrypted SD-WAN

   Homogeneous Encrypted SD-WAN refers to an SD-WAN network with edge
   nodes encrypting all traffic over the WAN underlay to other edge
   nodes, regardless of whether the underlay is private or public, as
   shown in Figure 2 below. For lack of better terminology, we call
   this Homogeneous Encrypted SD-WAN throughout this document.

   Here are some typical scenarios for using Homogeneous Encryption:

   -  A small branch office connecting to its HQ offices via the
   Internet. All traffic to/from this small branch office must be
   encrypted, usually achieved by IPsec Tunnels [RFC6071].

   -  A store in a shopping mall may need to securely connect to its
   applications in one or more Cloud DCs via the Internet. A common
   way of achieving this is to establish IPsec SAs to the Cloud DC
   gateway to carry the sensitive data to/from the store.

   As described in [SECURE-EVPN], the granularity of the IPsec SAs
   for Homogeneous Encryption can be per site, per subnet, per
   tenant, or per address. Once the IPsec SA is established for a
   specific subnet/tenant/site, all traffic to/from the
   subnet/tenant/site is encrypted.

Dunbar, et al.                                                [Page 9]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

                                     +---+
                      +--------------|RR |------------+
                     /  Untrusted    +-+-+             \
                    /                                   \
                   /                                     \
     +----+  +---------+                             +------+  +----+
     | CN3|--|         P1-----+ -------------+------ P1     |--| CN3|
     +----+  | C-PE1   P2-----+              |       | C-PE2|  +----+
     +----+  |         P3-----+     Wide     +------ P2     |  +----+
     | CN2|--|         |      |     area     +------ P3     |--| CN1|
      +-+--+  +---------+      |   network    |       +------+  +-+--+
        \                     |              |                  /
         \   +---------+      | all packets  |       +------+  /
          +--|         P1-----+ encrypted    +------ P1     |-+
              | C-PE3   P2-----+     by       |       | C-PE4|
     +----+  |         P3-----+ IPsec SAs    +------ P2     |  +----+
      | CN1|--|         P4-----+--------------+       |      |--| CN2|
      +----+  +---------+                             +------+  +----+

   CN: Client Networks, which is same as Tenant Networks used by NVo3

                Figure 2: Homogeneous Encrypted SD-WAN

   One of the properties of Homogeneous Encryption is that the SD-WAN
   Local Network Controller, e.g., RR in BGP-controlled SD-WAN, might
   be connected to C-PEs via an untrusted public network, therefore,
   requiring a secure connection between RR and C-PEs.

   A Homogeneous Encrypted SD-WAN shares certain characteristics with
   the widely deployed IPsec VPN. However, while IPsec VPNs typically
   operate in a point-to-point manner among a limited number of
   nodes, SD-WAN networks can comprise a large number of edge nodes
   and a centralized controller responsible for managing
   configurations across these nodes.

   Existing private VPNs (e.g., MPLS based) can use Homogeneous
   Encrypted SD-WAN to extend over the public network to remote sites
   to which the VPN operator does not own or lease infrastructural
   connectivity, as described in [SECURE-EVPN].

3.3. Scenario #2: Differential Encrypted SD-WAN

   The Differential Encrypted SD-WAN refers to an SD-WAN network in
   which traffic over the existing VPN is forwarded natively without
   encryption, and the traffic over the Public Internet is encrypted.
   Differential Encrypted SD-WAN is over hybrid private VPN and
   public Internet underlays. Since IPsec requires additional
   processing power and the encrypted traffic over the Internet does

Dunbar, et al.                                               [Page 10]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   not have the premium SLA (Service Lever Agreement) commonly
   offered by Private VPNs, especially over a long distance, it is
   more desirable for traffic over a private VPN to be forwarded
   without encryption.

   One C-PE might have the Internet-facing WAN ports managed by
   different NSPs with the WAN ports' addresses assigned by the
   corresponding NSPs. Clients might have policies to specify:

   1) Some flows can only be forwarded over private VPNs.
   2) Some flows can be forwarded over either private VPNs or the
     public Internet. The packets over the public Internet are
     encrypted.
   3) Some flows, especially Internet-bound browsing ones, can be
     handed off to the Internet without any encryption.

   Suppose a flow traversing multiple segments, such as A<->B, B<->C,
   C<->D, has Policy 2) above. The flow can cross different underlays
   in different segments, such as over Private underlay between A<->B
   without encryption or over the public Internet between B<->C
   protected by an IPsec SA.

   As shown in the figure below, C-PE-1 has two different types of
   interfaces (A1 to Internet and A2 & A3 to VPN). The C-PE's
   loopback address and the attached client addresses may or may not
   be visible to the NSPs. The WAN ports' addresses can be allocated
   by the service providers or dynamically assigned (e.g., by DHCP).

Dunbar, et al.                                               [Page 11]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

                                       +---+
                        +--------------|RR |----------+
                       /  Untrusted    +-+-+           \
                      /                                 \
                     /                                   \
     +----+  +---------+  packets encrypted over     +------+  +----+
     | CN3|--|         A1-----+ Untrusted    +------ B1     |--| CN1|
     +----+  | C-PE1   A2-\                          | C-PE2|  +----+
     +----+  |         A3--+--+              +---+---B2     |  +----+
     | CN2|--|         |   |PE+--------------+PE |---B3     |--| CN3|
     +----+  +---------+   +--+   trusted    +---+   +------+  +----+
                              |      WAN     |
     +----+  +---------+   +--+   packets    +---+   +------+  +----+
     | CN1|--|         C1--|PE| go natively  |PE |-- D1     |--| CN1|
     +----+  | C-PE3   C2--+--+ without encry+---+   | C-PE4|  +----+
             |         |      +--------------+       |      |
             |         |                             |      |
     +----+  |         |      without encrypt over   |      |  +----+
     | CN2|--|         C3--+---- Untrusted  --+------D2     |--| CN2|
     +----+  +---------+                             +------+  +----+

     CN: Client Network
                Figure 3: SD-WAN with Hybrid Underlays

     Also, the connection between C-PEs and their Controller (RR)
     might be via the untrusted public network. It is necessary to
     have secure channel for communication between RR and C-PEs.

     There could be multiple SD-WAN edges (C-PEs) sharing common
     property, such as a geographic location. Some applications over
     SD-WAN may need to traverse specific geographic areas for
     various reasons, such as to comply with regulatory rules, to
     utilize specific value-added services, or others.

     Services may not be congruent, i.e., the packets from A-> B may
     traverse one underlay network, and the packets from B -> A may
     go over a different underlay.

3.4. Scenario #3: Private VPN PE based SD-WAN

   This scenario refers to an existing VPN (e.g., EVPN[RFC7432] or
   IPVPN) being expanded by adding extra ports facing the public
   Internet to accommodate any additional bandwidth requirement
   between two PEs.

Dunbar, et al.                                               [Page 12]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   Here are some differences from the Hybrid Underlay scenario
   (Section 3.3):

       - For MPLS-based VPN, PEs would have MPLS as payload
          encapsulated within the IPsec tunnel egressing the Internet
          WAN ports, MPLS-in-IP/GRE-in-IPsec.
       - The BGP RR is connected to PEs in the same way as the VPN,
          i.e., via a trusted network.

   PE-based SD-WAN can be used by VPN service providers to
   temporarily increase bandwidth between sites when not sure if the
   demand will sustain for an extended period or as a temporary
   solution prior to building or leasing a permanent infrastructure.

                           +======>|PE2|
                         //        +---+
                        //          ^
                       //           || VPN
                      //     VPN    v
                      |PE1| <====> |RR| <=>   |PE3|
                      +-+-+        +--+       +-+-+
                        |                       |
                        +--- Public Internet -- +
                                 Offload

          Figure 4: Additional Internet paths added to the VPN

     For Ethernet-based client traffic, Private VPN PE based SD-WAN
     should support VLAN-based service interfaces (EVPN Instances),
     VLAN bundle service interfaces, or VLAN-Aware bundling service
     interfaces. EVPN service requirement as described in Section 3.1
     of [RFC8388] are applicable to the SD-WAN Ethernet-based Client
     services. For IP-based client interfaces, L3VPN service
     requirements are applicable.

4. Provisioning Model

4.1. Client Service Provisioning Model

   Client service provisioning can follow the same approach as MPLS
   VRFs (Virtual Routing and Forwarding) [RFC4364][RFC4659]. A client
   VPN can establish the communication policy by specifying the BGP

Dunbar, et al.                                               [Page 13]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   Route Targets to be imported and exported. Alternatively,
   conventional Match and Action ACLs (Access Control List) can
   identify the specific routes allowed or denied to or from the
   client VPN.

   When an SD-WAN edge node is dedicated to one client with a single
   virtual network, all prefixes attached to the client port(s) on
   the edge node can be considered to be inside a single VRF, and the
   RR can manage the policies for import/export policies for that
   VRF.

4.2. Policy Configuration

   One of the characteristics of an SD-WAN service is that packets
   can be forwarded over multiple types of underlays. Policies are
   needed to govern which underlay paths can carry a flow, as
   described by Section 8 of [MEF70.1]. A flow is a collection of
   packets between the same source and destination pair that are
   subject to the same forwarding and policy decisions at the ingress
   SD-WAN edge node, and are identified by the settings of one or
   more fields in the packet headers. For example, client-prefix-x
   can only be mapped to a MPLS topology.

4.3. IPsec Related Parameters Provisioning

   Edge nodes must negotiate diverse cryptographic parameters to
   establish IPsec tunnels between them. Alternatively, the edges can
   retrieve the attribute values from their controller, thereby
   streamlining the configuration process. In the context of a BGP-
   controlled SD-WAN, BGP UPDATE messages can disseminate IPsec-
   related attribute values for each node, facilitating peer
   selection of mutually supported values-instead of the process
   facilitated by IPsec IKEv2 [RFC7296].

5. BGP Controlled SD-WAN

5.1. Why BGP as Control Plane for SD-WAN?

   In the case of a modest-sized SD-WAN network with a limited number
   of nodes, the hub-and-spoke model, employing Next Hop Resolution
   Protocol (NHRP) or a centralized hub managing edge nodes,
   including the mapping of local and public addresses along with
   tunnel identifiers, has proven effective. However, when dealing
   with a larger SD-WAN network, exceeding 100 nodes and encompassing

Dunbar, et al.                                               [Page 14]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   diverse underlays, the conventional approach becomes notably
   intricate, convoluted, and susceptible to errors.

   Here are some of the compelling reasons for using BGP:

   -  Simplified peer authentication process:

     With a secure management channel established between an edge
     node and an RR, the RR can perform peer authentication on behalf
     of the edge node. The RR has policies on peer communication and
     the built-in capability to constrain the propagation of the
     UPDATE messages to the authorized edge nodes [RFC4684].

   - Scalable IPsec tunnel management

     When multiple IPsec tunnels are established between two edge
     nodes, BGP Tunnel Attribute Update can associate multiple IPsec
     tunnels with the client routes. In an IPsec VPN, separate
     routing instances need to run in parallel in each IPsec Tunnel
     if the client routes need be load shared among the IPsec
     tunnels.

   - Simplified IPsec tunnel traffic selection configurations

     The IPsec tunnel's traffic selector or admission control can be
     inherently realized by specifying importing/exporting the Route
     Targets representing the SD-WAN VPNs.

5.2. BGP Walk Through for Homogeneous Encrypted SD-WAN

   For the BGP-controlled Homogeneous Encrypted SD-WAN, a C-PE can
   advertise its attached client routes and the properties of the
   IPsec SA in one BGP UPDATE message.

   In the figure below, the BGP UPDATE message from C-PE2 to RR can
   have the client routes encoded in the MP-NLRI Path Attribute and
   the IPsec Tunnel associated information encoded in the Tunnel-
   Encapsulation [RFC9012] Attributes. More examples are described in
   the [SECURE-EVPN].

Dunbar, et al.                                               [Page 15]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

                        +---------|RR |----------+
                       / Untrusted+---+           \
                      /                            \
                     /                              \
             +---------+                       +---------+
           --+         |-----------------------|         |-192.0.2.4/30
             |         |                       | C-PE2   |- VLAN = 15
             | C-PE1   |                     +-|192.0.2.2|
           --|192.0.2.1|                     | |         |-192.0.2.8/30
             +---------+                     | +---------+
                                             |
                                             |
                                             |
             +---------+                     |
           --|         |---------------------+
             |         |
             | C-PE3   |
           --|192.0.2.3|
             +---------+
                Figure 5: Homogeneous Encrypted SD-WAN

   Alternatively, the C-PE2 can use two separate BGP UPDATE messages
   to reduce the size of the BGP UPDATE messages, especially for
   IPsec tunnels terminated at edge nodes or WAN ports, as IPsec SA
   tunnels have many attributes which can change at different
   frequencies than clients' routes update, such as IPsec SA keys
   periodic changes.

   When using two separate BGP UPDATE messages, which is described by
   Section 4 and 8 of [RFC9012], UPDATE U1 has its Nexthop to the
   node loopback address and is recursively resolved to the IPsec SA
   tunnel detailed attributes advertised by the UPDATE U2 for the
   Node Loopback address.

   Here are the details of the UPDATE messages:

     - Suppose that a given packet "C" destined towards the client
       addresses attached to C-PE2 (e.g., prefix 192.0.2.4/30) can be
       carried by any IPsec tunnels terminated at C-PE2.
     - The path along which "C" is to be forwarded is determined by
       BGP UPDATE U1.
     - UPDATE U1 does not have a Tunnel Encapsulation attribute.
     - UPDATE U1 can include the Encapsulation Extended Community
       with the option to have the Color Extended Community.
     - The address of the next-hop of UPDATE U1 is router C-PE2.

Dunbar, et al.                                               [Page 16]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

     - UPDATE U2 has a Tunnel Encapsulation attribute to describe the
       IPsec SA detailed attributes.

   UPDATE U1:

     - MP-NLRI Path Attribute:
         192.0.2.4/30
         192.0.2.8/30
     - Nexthop: 192.0.2.2 (C-PE2)
     - Encapsulation Extended Community: TYPE = IPsec

   UPDATE U2:
     - MP-NLRI Path Attribute:
         192.0.2.2 (C-PE2)
     - Tunnel Encapsulation Path Attributes (as described in [SECURE-
     EVPN]) for IPsec SA detailed attributes, including the WAN
     address to be used as the IP address of the IPsec encrypted
     packets.

   If different client routes attached to C-PE2 need to be reached by
   separate IPsec tunnels, the Color Extended Community [RFC9012] is
   used to associate routes with the tunnels. See Section 8 of
   [RFC9012].

   Suppose C-PE2 does not have a policy on the authorized peers for
   the specific client routes. Then, the RR then needs to check the
   client routes' policies to constrain the BGP UPDATE message
   propagation only to the remote authorized edge nodes.

5.3. BGP Walk Through for Differential Encrypted SD-WAN

   In this scenario, some client routes can be forwarded over any one
   of the tunnels terminating at the edge node. Some client routes
   can only be forwarded over specific tunnels (such as only MPLS
   VPN).

   An edge node can use the Color Extended Community (Section 4 & 8
   of [RFC9012]) in its BGP UPDATE message to associate the client
   routes with the specific tunnels.

Dunbar, et al.                                               [Page 17]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   For example, in Figure 5 above, suppose that Route 192.0.2.4/30
   can be carried by either MPLS or IPsec and Route 192.0.2.8/30 can
   only be carried by MPLS; C-PE2 can use the following UPDATE
   messages:

   UPDATE #1a for Route 192.0.2.4/30:

     - MP-NLRI Path Attribute:
         192.0.2.4/30
         Nexthop: 192.0.2.2 (C-PE2)
     - Encapsulation Extended Community: TYPE = SD-WAN-Hybrid
     - Color Extended Community: RED

   UPDATE #1b for Route 192.0.2.8/30:
     - MP-NLRI Path Attribute:
         192.0.2.8/30
         Nexthop: 192.0.2.2 (C-PE2)
     - Encapsulation Extended Community: TYPE = MPLS-in-GRE
     - Color Extended Community: YELLOW

   UPDATE #2a: for IPsec tunnels terminated at the node:
     - MP-NLRI Path Attribute:
         192.0.2.2 (C-PE2)
     - Tunnel Encapsulation Path Attributes: TYPE=SD-WAN-Hybrid
      Including the information about the WAN ports for receiving
      IPsec encrypted packets, the IPsec properties, etc.
     - Color Extended Community: RED

   UPDATE #2b: for MPLS-in-GRE terminated at the node:
     - MP-NLRI Path Attribute:
         192.0.2.2 (C-PE2)
     - Tunnel Encapsulation Path Attributes: TYPE=SD-WAN-Hybrid
     - Color Extended Community: YELLOW

   SD-WAN-Hybrid Tunnel Type is specified by [SD-WAN-EDGE-Discovery].

5.4. BGP Walk Through for Application Flow-Based Segmentation

   Suppose an application flow is identified by source or destination
   IP addresses. Application Flow-based Segmentation described in
   3.1.2 can be realized by constraining the BGP UPDATE messages,

Dunbar, et al.                                               [Page 18]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   such that only the nodes that meet the criteria of the application
   flow receive these updates. The following BGP Update messages can
   be advertised to ensure that the Payment Application only
   communicates with the Payment Gateway shown in Figure 6:

   BGP UPDATE #1a from C-PE2 to RR for the P2P topology that is only
   propagated to Payment GW node:

      - MP-NLRI Path Attribute:
            - 192.0.2.9/30
            - Nexthop: 192.0.2.2
      - Encapsulation extended community: TYPE = IPsec
      - Color Extended Community: BLUE

   BGP UPDATE #1b from C-PE2 to RR is propagated to C-PE1 & C-PE3 for
   the routes to be reached by C-PE1 and C-PE3:

      - MP-NLRI Path Attribute:
            - 192.0.2.4/30
            - 192.0.2.8/30
            - Nexthop:192.0.2.2
       - Encapsulation extended community: TYPE =IPsec
       - Color Extended Community: RED

   BGP UPDATE #2a for the detailed IPsec attributes for IPsec tunnels
   terminated at C-PE2 192.0.2.2 is propagated to C-PE1 & C-PE3.

     - MP-NLRI Path Attribute:
         192.0.2.2 (C-PE2)
     - Tunnel Encapsulation Path Attributes: TYPE=IPsec (for all
     IPsec SAs)
     - Color Extended Community: RED

   UPDATE #2b for the IPsec SA to the Payment GW is only propagated
   to Payment GW:
     - MP-NLRI Path Attribute:
         192.0.2.2 (C-PE2)
     - Tunnel Encapsulation Path Attributes: TYPE=IPsec (for the
     IPsec SA to Payment GW).
     - Color Extended Community: Blue

Dunbar, et al.                                               [Page 19]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

                                |Payment|
                       +------->|  GW   |<----+
                      /         +-------+      \
                     /        Blue Tunnel       \
                    /for Payment App:192.0.2.9/30\
                   /                              \
           +------/--+                        +----\----+
         --|-----+   |                        |     +---| 192.0.2.9/30
           |         |     Red Tunnels        |         |
         --| C-PE1   |------------------------|         |-192.0.2.4/30
           |192.0.2.1|                        |  C-PE2  |
         --|         |------------------------|192.0.2.2|-192.0.2.8/30
           |   ------+                       +|         |- VLAN=25;
                                            / |         |192.0.2.10/30
           +---------+                     /  +---------+
         --|         |--------------------+
           | C-PE3   |                   /
           |192.0.2.3|                  /
         --|         |-----------------+
           +---------+
         Figure 6: Application Based SD-WAN Segmentation

5.5. Benefit of Using Recursive Next Hop Resolution

   Using the Recursive Next Hop Resolution described in Section 8 of
   [RFC9012], the clients' route UPDATE messages become very compact,
   and any attribute changes of the underlay network tunnels, such as
   IPsec key refreshing, can be advertised independently of the
   client route update. This method is handy when the underlay
   tunnels are IPsec based, which requires periodic message exchange
   for the pairwise re-keying process.

6. SD-WAN Forwarding Model

   This section describes how client traffic is forwarded in BGP
   Controlled SD-WAN for the use cases described in Section 3.

   The procedures described in Section 6 of RFC8388 are applicable
   for the SD-WAN client traffic. Like the BGP-based VPN/EVPN client
   routes UPDATE message, Route Target can distinguish routes from
   different clients.

Dunbar, et al.                                               [Page 20]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

6.1. Forwarding Model for Homogeneous Encrypted SD-WAN

6.1.1. Network and Service Startup Procedures

   A single IPsec security association (SA) protects data in one
   direction. In the Homogeneous Encrypted SD-WAN Scenario, two SAs
   must be present to secure traffic in both directions between two
   C-PE nodes, two client ports, or two prefixes. Using Figure 2 of
   Section 3.2 as an example, for client CN2 attached to C-PE1, C-
   PE3, and C-PE4 to have a full-mesh connection, six one-directional
   IPsec SAs must be established: C-PE1 <-> C-PE3; C-PE1 <-> C-PE4;
   C-PE3 <-> C-PE4.

   SD-WAN services to clients can be IP-based or Ethernet-based. An
   SD-WAN edge can learn client routes from the client-facing ports
   via OSPF, RIP, BGP, or static configuration for its IP-based
   services. For Layer-2 SD-WAN services, the relevant EVPN
   parameters, such as the ESI (Ethernet Segment Identifier), EVI,
   and CE-VID (Customer Edge Virtual Instance Identifier) to EVI
   mapping, can be configured similarly to EVPN described in RFC8388.

   Instead of running IGP within each IPsec tunnel done by the IPsec
   VPN, BGP RR can propagate UPDATE messages of the client routes
   attached to an SD-WAN edge node to its authorized peers.

   In addition, the BGP-RR (SD-WAN Controller) facilitates the IPsec
   SA establishment and rekey management as described in [SECURE-
   EVPN]. The controller manages how clients' routes are associated
   with individual IPsec SA. Therefore, it is no longer necessary to
   manually configure the IPsec tunnel's endpoint addresses on each
   SD-WAN edge node and set up policies for the allowed client
   prefixes.

6.1.2. Packet Walk-Through

   For unicast packets forwarding:

     An IPsec SA terminated at a C-PE node can have multiple client
     routes multiplexed in the IPsec SA (or tunnel). Packets to/from
     the client prefixes are encapsulated in an inner tunnel, such as
     GRE or VxLAN. Different client traffic can be differentiated by
     a unique value in the inner encapsulation key or ID field. The
     GRE or VxLAN tunnel is encapsulated by an outer IP header whose
     destination & source addresses are the C-PE nodes loopback
     addresses and most likely has the Protocol-code = ESP (50).

     C-PE Node-based IPsec tunnel is inherently protected when the C-
     PE has multiple WAN ports to different underlay paths. As shown

Dunbar, et al.                                               [Page 21]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

     in Figure 2, when one of the underlay paths fails, the IPsec
     traffic can be forwarded to or received from a different
     physical port.

     When a C-PE receives an IPsec encrypted packet from its WAN
     ports, it decrypts the packet and forwards the inner packet to
     the client port based on the inner packet's destination address.

   For multicast packets forwarding:

     IPsec was created to be a security protocol between two and only
     two devices, so multicast service using IPsec is problematic. An
     IPsec peer encrypts a packet so that only one other IPsec peer
     can successfully perform the de-encryption. A straightforward
     way to forward a multicast packet for the Homogeneous Encrypted
     SD-WAN is to encapsulate the multicast packet in separate
     unicast IPsec SA tunnels. More optimized forwarding multicast
     packets for the Homogeneous Encrypted SD-WAN is out of the scope
     of this document.

6.2. Forwarding Model for Hybrid Underlay SD-WAN

   In this scenario, as shown in Figure 3 of Section 3.3, traffic
   forwarded over the trusted VPN paths can be native (i.e.,
   unencrypted). The traffic forwarded over untrusted networks need
   to be protected by IPsec SA.

6.2.1. Network and Service Startup Procedures

   Infrastructure setup: The proper MPLS infrastructure must be
   configured among the edge nodes, i.e., the C-PE1/C-PE2/C-PE3/C-PE4
   of Figure 3. The IPsec SA between wAN ports or nodes must be set
   up as well. IPsec SA related attributes on edge nodes can be
   distributed by BGP UPDATE messages as described in Section 5.

   There could be policies governing how flows can be forwarded, as
   specified by [MEF70.1].  For example, "Private-only" indicates
   that the flows can only traverse the MPLS VPN underlay paths.

6.2.2. Packet Walk-Through

   For unicast packets forwarding:

     When the C-PE-a in Figure 7 receives a packet from a client
     port, if the packet belongs to a flow that can only be forwarded
     over the MPLS VPN, the forwarding processing is the same as the

Dunbar, et al.                                               [Page 22]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

     MPLS VPN. Otherwise, the C-PE node can make the local decision
     in choosing the least cost path, including the previously
     established MPLS paths and IPsec Tunnels, to forward the packet.
     Packets forwarded over the trusted MPLS VPN can be native
     without additional encryption, while the packets sent over the
     untrusted networks must be encrypted by IPsec SA.

     For a c-PE with multiple WAN ports provided by different NSPs,
     separate IPsec SAs can be established for the different WAN
     ports. In this case, the C-PE have multiple IPsec tunnels in
     addition to the MPLS path to choose from to forward the packets
     from the client ports.

     If the IPsec SA is chosen, the packet is encapsulated by the
     IPsec header and encrypted by the IPsec SA before forwarding it
     to the WAN.

     For packets received from an MPLS path, processing is the same
     as MPLS VPN.

     For IPsec SA encrypted packets received from the WAN ports, the
     packets are decrypted, and the inner payload is decapsulated and
     forwarded per the forwarding table of the C-PE. For all packets
     from the Internet-facing WAN ports, the additional anti-DDoS
     mechanism has to be enabled to prevent potential attacks from
     the Internet-facing ports. Control Plane should not learn routes
     from the Internet-facing WAN ports.

                        +--------------|RR |----------+
                       /               +-+-+           \
                      /                                 \
                     /                                   \
     +----+  +---------+  packets encrypted over    +---------+  +----+
     | CN3|--|         A1-----+ Untrusted    +----- B1        |--| CN1|
     +----+  | C-PE-a  A2-----+              +------B2 C-PE-b |  +----+
             |192.0.2.1|                            |192.0.2.2|
     +----+  |         A3  |PE+--------------+PE |--B3        |--| CN3|
     +----+  +---------+   +--+   trusted    +---+  +---------+  +----+
                              |     VPN      |
                          +-----------+
                      Figure 7: Over hybrid SD-WAN

   For multicast packets forwarding:

     For multicast traffic, MPLS multicast [RFC6513, RFC6514, or
     RFC7988] can be used to forward multicast traffic.

Dunbar, et al.                                               [Page 23]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

     If IPsec tunnels are chosen for a multicast packet, the packet
     is encapsulated and encrypted by multiple separate IPsec tunnels
     to the desired destinations.

6.3. Forwarding Model for PE based SD-WAN

6.3.1. Network and Service Startup Procedures

   In this scenario, all PEs have secure interfaces facing the
   clients and facing the MPLS backbone, with some PEs having extra
   ports to the untrusted public Internet. The public Internet paths
   are for offloading low priority traffic when the MPLS paths get
   congested. The PEs are already connected to their RRs, and the
   configurations for the clients and policies are already
   established.

6.3.2. Packet Walk-Through

   For PEs to offload some MPLS packets to the Internet path, each
   MPLS packet is wrapped by an outer IP header as MPLS-in-IP or
   MPLS-in-GRE [RFC4023]. The outer IP address can be an interface
   address or the PE's loopback address.

   When IPsec Tunnel mode is used to protect an MPLS-in-IP packet,
   the entire MPLS-in-IP packet is placed after the IPsec tunnel
   header.

   When IPsec transport mode is used to protect the MPLS packet, the
   MPLS-in-IP packet's IP header becomes the outer IP header of the
   IPsec packet, followed by an IPsec header, and then followed by
   the MPLS label stack. The IPsec header must set the payload type
   to MPLS by using the IP protocol number specified in section 3 of
   RFC4023.

   If IPsec transport mode is applied to an MPLS-in-GRE packet, the
   GRE header follows the IPsec header.

   The IPsec SA's endpoints should not be the client-facing interface
   addresses unless the traffic to/from those clients always goes
   through the IPsec SA even when the MPLS backbone has enough
   capacity to transport the traffic.

   When the PEs' Internet-facing ports are behind the NAT [RFC3715],
   an outer UDP field can be added outside the encrypted payload
   [RFC3948]. Three UDP ports must be open on the PEs: UDP port 4500
   (used for NAT traversal), UDP port 500 (used for IKE), and IP

Dunbar, et al.                                               [Page 24]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   protocol 50 (ESP). IPsec IKE (Internet Key Exchange) between the
   two PEs would be over the NAT [RFC3947] as well.

   Upon receiving a packet from a client port, the forwarding
   processing is the same as the MPLS VPN. If the MPLS backbone path
   to the destination is deemed congested, the IPsec tunnel towards
   the target Pes is used to encrypt the MPLS-in-IP packet.

   Upon receiving a packet from the Internet-facing WAN port, the
   packet is decrypted, and the inner MPLS payload is extracted to be
   sent to the MPLS VPN engine.

   Same as Scenario #2, the additional anti-DDoS mechanism must be
   enabled to prevent potential attacks from the Internet-facing
   port. Control Plane should not learn routes from the Internet-
   facing WAN ports.

7. Manageability Considerations

     BGP-controlled SD-WAN utilizes the BGP RR to facilitate the
     routes and underlay properties distribution among the authorized
     edge nodes. With RR having the preconfigured policies about the
     authorized peers, the peer-wise authentications of the IPsec IKE
     (Internet Key Exchange) are significantly simplified.

8. Security Considerations

   Adding an Internet-facing WAN port to a C-PE can introduce the
   following security risks:

   1) Potential DDoS attacks from the Internet-facing ports.

   2) Potential risk of malicious traffic being injected via the
   Internet-facing WAN ports.

   3) For the Differential Encrypted SD-WAN deployment model, there
   is a risk of unauthorized traffic injected into the Internet-
   facing WAN ports being leaked to the L2/L3 VPN networks.

   Therefore, the additional anti-DDoS mechanism must be enabled on
   all Internet-facing ports to prevent potential attacks from those
   ports. Control Plane should not learn any routes from the
   Internet-facing WAN ports.

Dunbar, et al.                                               [Page 25]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   In SD-WAN deployments where no secure management channel exists
   between the SD-WAN controller and the SD-WAN edges, TLS or IPsec
   can be established to bridge the gap. While beyond the scope of
   this document, conducting a comprehensive analysis is imperative
   to ensure the security of BGP over TLS [BGP-OVER-TLS].

9. IANA Considerations

       No Action is needed.

10. References

10.1. Normative References

   [BCP195]  Consists of RFC8996 and RFC9325.

   [RFC3715] B. Aboba, W. Dixon, "IPsec-Network Address Translation
             (NAT) Compatibility Requirements", March 2004.

   [RFC3947] T. Kivinen, et al, "Negotiation of NAT Traversal in the
             IKE", Jan. 2005.

   [RFC3948] A. Huttunen, et al, "UDP Encapsulation of IPsec ESP
             Packets", Jan 2005.

   [RFC4023] T. Worster, Y. Rekhter, E. Rosen, "Encapsulating MPLS in
             IP or Generic Routing Encapsulation (GRE)", March 2005.

   [RFC4364] E. Rosen, Y. Rekhter, "BGP/MPLS IP Virtual Private
             networks (VPNs)", Feb 2006.

   [RFC4456] T. Bates, E. Chen, R. Chandra, "BGP Route Reflection: An
             Alternative to Full Mesh Internal BGP (IBGP)", April
             2006.

   [RFC4659] J. De clercq, et al, "BGP-MPLS IP Virtual Private
             Network (VPN) Extension for IPv6 VPN", RFC4659, Sept
             2006.

Dunbar, et al.                                               [Page 26]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   [RFC4761] K. Kompella and Y. Rekhter, "Virtual Private LAN Service
             (VPLS) Using BGP for Auto-Discovery and Signaling",
             RFC4761, Jan. 2007.

   [RFC4762] M. Lasserre and V. Kompella, "Virtual Private LAN
             Service (VPLS) Using Label Distribution Protocol (LDP)
             Signaling", RFC4762, Jan. 2007.

   [RFC4684] P. Marques, et al, "Constrained Route Distribution for
             BGP/MPLS IP VPNs", November 2006.

   [RFC6071] S. Frankel, S. Krishan, "IP Security (IPsec) and
             Internet Key Exchange (IKE) Document Roadmap", Feb 2011.

   [RFC7296] C. Kaufman, et al, "Internet Key Exchange Protocol
             Version 2 (IKEv2)", Oct 2014.

   [RFC7432] A. Sajassi, et al, "BGP MPLS-Based Ethernet VPN",
             RFC7432, Feb 2015.

   [RFC8200] S. Deering and R. Hinden, "Internet Protocol, Version 6
             (IPv6) Specification". July 2017.

   [RFC8388] J. Rabadan, et al, "Usage and Applicability of BGP MPLS-
             Based Ethernet VPN", RFC8388, May 2018.

   [RFC9012] K.Patel, et al "The BGP Tunnel Encapsulation Attribute",
             RFC9012, April 2021.

10.2. Informative References

   [BGP-OVER-TLS] T. Wirtgen, "BGP over TLS/TCP", draft-wirtgen-bgp-
             tls-00, Oct. 2023.

   [SD-WAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K.
             Majumdar, "BGP UPDATE for SD-WAN Edge Discovery", draft-
             ietf-idr-sdwan-edge-discovery-12, Oct 2023.

   [SECURE-EVPN] A. Sajassi, et al, "Secure EVPN", draft-ietf-bess-
             secure-evpn-00, Work-in-progress, June 2023.

Dunbar, et al.                                               [Page 27]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

   [Net2Cloud-Problem] L. Dunbar and A. Malis, "Dynamic Networks to
             Hybrid Cloud DCs: Problems and Mitigation Practices",
             draft-ietf-rtgwg-net2cloud-problem-statement-34, Jan
             2024.

   [IEEE802.3] "IEEE Standard for Ethernet" by The Institute of
             Electrical and Electronics Engineers (IEEE) 802.3.

   [MEF70.1] SD-WAN Service Attributes and Service Framework,
             https://www.mef.net/resources/mef-70-1-sd-wan-service-
             attributes-and-service-framework/. Nov 2021.

11. Acknowledgments

   Acknowledgements to Andrew Alston, Adrian Farrel, Jim Guichard,
   Joel Halpern, John Scudder, Darren Dukes, Andy Malis, Donald
   Eastlake, Stephen Farrell, and Victo Sheng for their review and
   contributions.

   This document was prepared using 2-Word-v2.0.template.dot.

Dunbar, et al.                                               [Page 28]
Internet-Draft           BGP Usage for SD-WAN        February 7, 2024

Authors' Addresses

   Linda Dunbar
   Futurewei
   Email: ldunbar@futurewei.com

   Ali Sajassi
   Cisco
   Email: sajassi@cisco.com

   John Drake
   Independent
   Email: je_drake@yahoo.com

   Basil Najem
   Bell Canada
   Email: basil.najem@bell.ca

Contributor's Addresses

   David Carrel
   Graphiant
   Email: carrel@graphiant.com

   Ayan Banerjee
   Cisco
   Email: ayabaner@cisco.com

Dunbar, et al.                                               [Page 29]