Skip to main content

BGP Usage for SD-WAN Overlay Networks



(Andrew Alston)

No Objection

No Record

Deb Cooley
Francesca Palombini
Gunter Van de Velde
Mahesh Jethanandani
Orie Steele
Warren Kumari
Éric Vyncke

Summary: Needs a YES. Has 2 DISCUSSes.

John Scudder
Discuss (2024-02-27 for -20) Sent

I have several concerns with this document, and I can't support it being published in its current form. Details are below, listed roughly in order of severity.

### What's it for?

My most fundamental concern is that I can't understand what this document is for. It doesn't specify a procedure. It doesn't provide enough detail to allow me to understand how to use the underlying technologies to build an SD-WAN service. (Also some of the details that are provided are wrong, see subsequent item.) It's not an architecture or framework document in the usual sense. It does motivate at a high level, the understanding that one *can* use BGP as the control plane, and IPSec as the data plane [*], to build an SD-WAN service, if one knows how... but do we need an RFC for that?

The closest I could find to a thesis statement was the abstract's "The document aims to demonstrate how the BGP-based control plane is used for large-scale SD-WAN overlay networks with little manual intervention." I guess if we allow the word "demonstrate" to mean "motivate at a high level" rather than "specify", then we can say it's not too far off. I don't see why we need an RFC for that, though.

[*] Or the concatenation of IPSec and "traditional" BESS VPN technologies, in the "differential" case.

### Is it in charter?

Looking at the BESS charter, I don't see how this document fits. If it weren't for the preceding question, I probably wouldn't have thought to look. But as it stands, I have to ask: why does the WG think this is a chartered work item?

### Error in how RFC 9012 is used

In Section 5.2, the Encapsulation Extended Community is misused. See for related explanation of the error. The error recurs in Sections 5.3 and 5.4.

Also, there is no such thing as “TYPE = IPsec” (referenced in Sections 5.2 and 5.4), see

This one should be straightforward to fix. It does lead me to be worried about the level of review the document has received, though.

### Section 3.1.5, security model

   BGP is well suited for this purpose. RFC4684 has specified the
   procedure to constrain the distribution of BGP UPDATE to only a
   subset of nodes. Each edge node informs the Route-Reflector (RR)
   [RFC4456] on its interested SD-WAN VPNs. The RR only propagates
   the BGP UPDATE from an edge to others within the same SD-WAN VPN.
RFC 4684 is driven by demand -- a PE will advertise RT membership NLRI to "request" that it receive routes with the given RT. So, this means the security model is that the client router is implicitly trusted to declare its VPN membership(s) truthfully and accurately. I don't see this addressed in the Security Considerations.
Comment (2024-02-27 for -20) Sent

### What’s a “client route”?

The document talks about "client routes", "client route updates", etc. It never defines the concept. I assume that these are analogous to a CE route in an RFC 4364 VPN, but the casual usage and lack of definition are unhelpful.


There’s no such thing as "MP–NLRI". I guess you mean MP_REACH_NLRI and MP_UNREACH_NLRI. If you want to collectively refer to these as "MP-NLRI" then I think you need to add a definition.

### Anti-DDoS

Section 6.2.2 has

                                                     For all packets
     from the Internet-facing WAN ports, the additional anti-DDoS
     mechanism has to be enabled to prevent potential attacks from
     the Internet-facing ports.
What anti-DDoS mechanism is that? None is specified here, nor referenced.

### Figure 7

Figure 7 is mangled to the point of being unreadable.
Roman Danyliw
Discuss (2024-02-27 for -20) Sent
(revised position)

** Section 8.
   In SD-WAN deployments where no secure management channel exists
   between the SD-WAN controller and the SD-WAN edges, TLS or IPsec
   can be established to bridge the gap. While beyond the scope of
   this document, conducting a comprehensive analysis is imperative
   to ensure the security of BGP over TLS [BGP-OVER-TLS].

This text would benefit from further specificity.  My feedback below is very similar to my original DISCUSS position on -15 of this document.

-- Is the “secure management channel” described here the protocol that carries the BGP?  

-- Assuming it is, what is a “secure management channel” that would allow BGP to travel over the Internet that isn’t protected by IPsec (i.e., why can’t this text roughly read, “BGP over IPsec MUST be used”).  When and how should the BGP communication be secured?  Section 3.1.5 does mentioned that the BGP connection must be secured (i.e., “As the connection between an SD-WAN edge and its RR can be over an insecured network, an SD-WAN edge must establish a secure connection to its designated RR upon power-up. The BGP UPDATE messages must be sent over the secure channel to the RR.”)

-- The imperative, comprehensive analysis of [BGP-OVER-TLS] is not sufficiently motivated and the reference to an unadopted individual document is confusing.  I agree with the recommendation of the SECDIR reviewer to drop [BGP-OVER-TLS].  Can the basis for including it be explained?

** (Missed this on my original ballot on -15) The need for “additional anti-DDoS mechanism” is mentioned in the text in Sections 6.2.2, 6.3.2, 8.  Can the expected mechanisms please be clarified.
Comment (2024-02-27 for -20) Sent
Thank you to Stephen Farrell for the SECDIR review.

I support the DISCUSS position of John Scudder.

** Section 2.  Editorial. If Section 3.2. defines “Homogeneous Encrypted SD-WAN” and the term isn’t used prior to that. Consider if this needs the definition.

** Section 2.
   SD-WAN IPsec SA: IPsec Security Association between two WAN ports
               of the SD-WAN nodes or between two SD-WAN nodes.

Is it “two SD-WAN nodes” or “two SD-WAN edge nodes”?  If the former, what is an “SD-WAN” node that isn’t an edge node?

** Section 3.1.2.  This sections seems to suggest that various MEF 70.1 behavior needs to be supported.  This suggests that [MEF70.1] needs to be a normative reference.

** Section 3.1.3.  Editorial.
   SD-WAN Traffic Segmentation enables the separation of the traffic
   based on the business and the security needs of different user
   groups and/or application requirements. Each user group and/or
   application may need different isolated topologies and/or policies
   to fulfill the business requirements.

Isn’t a security need also a need of the business?  Otherwise, the second sentence should read “… to fulfill the business and security requirements”.

** Section 3.1.3.  Editorial.
   The traffic from the PoS application follows a tree topology
   (denoted as "----" in the figure below), whereas other traffic can
   follow a multipoint-to-multipoint topology (denoted as "===").

In the figure, where is the PoS application, in one of the “Site #”?  Is there an assumption that traffic “flows up” from the sites to the payment gateway?  Where is the a “---” link from the “====”-connected “multi-point connection for non-payment traffic”?

** Section 3.1.4
     - Upon power-up, the SD-WAN edge can establish the transport
     layer secure connection [BCP195] to its controller, whose URL
     (or IP address) and credential for connection request can be
     preconfigured on the edge device by the manufacture, external
     USB or secure Email given to the installer.

Same feedback as provided on the ballot of the -15 version of this document:
Can the last two configuration options be clarified.

-- per “external USB”. Do you mean a USB _drive_ of some kind that is plugged in and the edge device knows to read what ever is on the drive to configure itself?  Does this mean anyone with physical access to the USB plug can power cycle the machine and can reconfigure it?

--  per “secure email”, does this mean that the installer configures the device based on something typed in?  What’s “secure” about the email?

Neither of these mechanisms seem like "Zero Touch".  They seem like the definition of human-in-the-loop.

** Section 3.1.4.  My understanding of Section 3.1.* was that it was series of requirements to be fulfilled by using BGP with SDWAN.  Per Zero Touch Provisioning, in what way is BGP involved in this provisioning? I’m not sure how this section fits into the scope of the document.

** Section 3.2
   -  A small branch office connecting to its HQ offices via the
   Internet. All traffic to/from this small branch office must be
   encrypted, usually achieved by IPsec Tunnels [RFC6071].

   -  A store in a shopping mall may need to securely connect to its
   applications in one or more Cloud DCs via the Internet. A common
   way of achieving this is to establish IPsec SAs to the Cloud DC
   gateway to carry the sensitive data to/from the store.

What is the technical difference between an “IPsec Tunnel” per the branch office use case and a “IPsec SA” in the store in the mall use case?

** Section 4.3
   In the context of a BGP-
   controlled SD-WAN, BGP UPDATE messages can disseminate IPsec-
   related attribute values for each node

Per the ballot on -15 of this draft, I asked “Please provide a reference to the specification which provides guidance on the BGP TLVs to provision IPsec.  Is that draft-ietf-idr-sdwan-edge-discovery?  This should be a normative reference.”  I still believe additional clarity is required.

** Section 5.2.
   In the figure below, the BGP UPDATE message from C-PE2 to RR can
   have the client routes encoded in the MP-NLRI Path Attribute and
   the IPsec Tunnel associated information encoded in the Tunnel-
   Encapsulation [RFC9012] Attributes.

Per, is that value=25

** [SECURE-EVPN] is referenced a few times in the course of the draft.  Is the WG confident that this expired draft is appropriate to use?

** Section 8.
   Adding an Internet-facing WAN port to a C-PE can introduce the
   following security risks:

   1) Potential DDoS attacks from the Internet-facing ports.

   2) Potential risk of malicious traffic being injected via the
   Internet-facing WAN ports.

   3) For the Differential Encrypted SD-WAN deployment model, there
   is a risk of unauthorized traffic injected into the Internet-
   facing WAN ports being leaked to the L2/L3 VPN networks.

   Therefore, the additional anti-DDoS mechanism must be enabled on
   all Internet-facing ports to prevent potential attacks from those

I’m having trouble understanding what is considered in-scope for these Security Considerations.  I was under the impression that the focus was the use of BGP as the control plane for SD-WAN communication.  Does it also include the “tunnels”/links configured by the SD-WAN via BGP.

-- If the security properties of tunnels/links configured by the SD-WAN BGP are in scope, then a reminder that links transiting non-Internet links are subject to security properties negotiated in the SLAs of that provided.

Specifically, on -15 of this document, I previously balloted “** Section 8.  The different deployment models also seem to place different levels of trust in the service providers.  Consider mentioning these differences.”

-- Likely because I don’t understand the definition of “Internet-facing ports” in the this context, I don’t see a difference between risk (2) and (3).  (2) seems like a more general version of (3).

-- What is the relationship between “anti-DDoS mechanism” and risk (2) of malicious traffic being injected?  Malicious traffic injection isn’t necessarily a DDoS.

-- I observe that there are no inherited Security Considerations from IPsec or BGP mentioned here.  

Per the ballot on -15 of the document, “This document seems to build on a number of other technologies.  Do their security considerations not apply (e.g., BGP, whatever ZTP technologies is chosen)?”, this feedback still applies.

** Section 10.1  Please provide the appropriate citation for [BCP195]
Erik Kline
No Objection
Comment (2024-02-25 for -20) Sent
# Internet AD comments for draft-ietf-bess-bgp-sdwan-usage-20
CC @ekline

* comment syntax:

* "Handling Ballot Positions":

## Comments

### S1

* Not entirely sure, but consider whether MEF 70.2 is better (or up-to-date)
  reference. (If you do change this reference, the Table 7 & 8 mentions need
  to be revised to Table 8 & 9, if I'm not mistaken).

* I cannot seem to find mention of the IPv6 Flow Label in MEF 70.1, nor in
  MEF 70.2.  I agree that it seems like it should be supported, but these
  tables seem to list requirements for support, and flow label seems to be

### S6.3.2

* Is it necessary for IKE to be open if the requisite IPsec tunnel parameters
  can be shared via a BGP UPDATE message instead (vis. S4.3)?

## Nits

### S3.3

* The C3<->D2 connector reads "without encrypt over Untrusted".  Should that
  be "without encryption"?

### S6.1.1

* "six one-directional" ->
  "six unidirectional"?

### S6.2.1

* "wAN ports" -> "WAN ports"

### S6.3.2

* "target Pes" -> "target PEs"?
Jim Guichard
No Objection
Comment (2024-02-28 for -20) Sent
- Abstract: First paragraph use of 'The document' instead of 'This document' seems awkward, I suggest using the latter. Further the second paragraph seems completely out of place, and I would suggest removing it as it does not appear to provide any value. 

- Section 3.1.1 (1st paragraph) - Add references for both IPsec and MPLS VPN on first usage. Same comment for VRFs.

- Section 3.1.1 (2nd paragraph) - Please expand on what the text "Additionally, it assumes that one SD-WAN VPN can be mapped to one or multiple virtual topologies governed by the SD-WAN controller's policies" means. From the written text I am unable to understand it.

- Section 3.1.1 (3rd paragraph) - please explain what a 'Client Route' is. I assume that you mean a route generated by an attached SD-WAN site, but the text does not say that. In addition, please correct the text 'Route Target in the BGP Extended Community' - Route Target Community is defined in RFC4360 so please add with reference.

- Section 3.1.1 (4th paragraph) - "For packets carried by an IPsec tunnel, the IPsec tunnel's inner encapsulation header can have the SD-WAN VPN Identifier to distinguish the packets belonging to different SD-WAN VPNs". Can they? is there an RFC or draft defining that? 

- Section 3.4 - add references for 'MPLS-in-IP/GRE-in-IPsec'.

- Section 4.3 - "In the context of a BGP-controlled SD-WAN, BGP UPDATE messages can disseminate IPsec-related attribute values for each node..." - do you mean using RFC5566 here? if so, please add a reference - if not then please add a reference on how BGP should disseminate the IPsec-related attribute values.

- Section 5.1 - add reference for NHRP (RFC2332)
Murray Kucherawy
No Objection
Comment (2024-02-29 for -20) Not sent
I'm jumping on the "I support John's DISCUSS" pile.
Paul Wouters
(was Discuss) No Objection
Comment (2024-03-04 for -21) Sent for earlier
I support John's and Roman's DISCUSSes.

[updated to remove a minor point I had as part of my DISCUSS outside of John's and Roman's positions, so modified my DISCUSS to a comment supporting John/Roman]
Zaheduzzaman Sarker
No Objection
Comment (2024-02-28 for -20) Not sent
No objection from transport layer protocol point of view. However, I would like know answer to the charter scope question, hence, supporting John's discussion on the charter scope point.
Deb Cooley
No Record
Francesca Palombini
No Record
Gunter Van de Velde
No Record
Mahesh Jethanandani
No Record
Orie Steele
No Record
Warren Kumari
No Record
Éric Vyncke
No Record
Andrew Alston Former IESG member
Yes (for -20) Not sent