Skip to main content

BGP Usage for SD-WAN Overlay Networks



(Andrew Alston)

No Objection

Note: This ballot was opened for revision 15 and is now closed.

Roman Danyliw
Discuss (2023-10-02 for -15) Sent
** Section 3.1.5.  
   The BGP UPDATE messages must be sent over the secure channel
   (TLS, DTLS, etc.) to the RR.

Can this text say a bit more on the expectations of secure transport?  My understanding was the IPsec was the common practice if confidentiality was desired.  Are there pointers for BGP over DTLS? Over TLS?  The closest I was able to find was BGP over QUIC per

** Section 8.  This document seems to build on a number of other technologies.  Do their security considerations not apply (e.g., BGP, whatever ZTP technologies is chosen)?
Comment (2023-10-02 for -15) Sent
Thank you to Stephen Farrell for the SECDIR review.

** Section 1.  Editorial. s/Corporate HQ/corporate headquarters/.  Expand acronym on first use.

** Section 1.
     - Some traffic can be forwarded by edge nodes, based on their
       application identifiers

What are “application identifiers”?  Is an application in this context "HTTP" or "a particular video streaming service provider"?

** Section 2.
               NSP usually provides more
               advanced network services, such as MPLS VPN, private
               leased lines, or managed Secure WAN connections, often
               within a private, trusted domain. In contrast, an ISP
               usually provides plain Internet services over public
               untrusted domains.

There is a distinction between made between NSP vs. ISP.  I understand the idea of an NSP providing value added services.  What I could use help with is understanding what “plain Internet service over public untrusted domains” means.  How does an ISP provide services in a domain other than its own?
As far as I can tell, there is no subsequent text which relies on this nuanced definition of NSP vs. ISP.

** Section 3.1.4.

... such as the device series number

Should this read as “serial number”?

** Section 3.1.4
   - Upon power-up, the SD-WAN edge can establish the transport layer
     secure connection (such as TLS, DTLS) [BCP195] to its controller,
     whose URL (or IP address) can be preconfigured on the edge device
     by manufacture default, external USB or secure Email given to the

Can the last two configuration options be clarified.

-- per “external USB”. Do you mean a USB _drive_ of some kind that is plugged in and the edge device knows to read what ever is on the drive to configure itself?  Does this mean anyone with physical access to the USB plug can power cycle the machine and can reconfigure it?

--  per “secure email”, does this mean that the installer configures the device based on something typed in?  What’s “secure” about the email?

Neither of these mechanisms seem like "Zero Touch".  They seems like the definition of human-in-the-loop.

** Section 3.1.4
     - The SD-WAN Controller authenticates the ZTP request from the
     remote SD-WAN edge with its configurations.

If only a URL is passed per the previous step, how is authentication of the device realized?

** Section 3.1.5
   In this circumstance, the
   property of the SD-WAN edge node cannot be propagated to other nodes
   that are not authorized to communicate.
   Therefore, SD-WAN
   deployment needs to have a central point to distribute the
   properties of an SD-WAN edge node to its authorized peers.

-- I got confused here.  What is a property?  How do properties propagate (or not)?

-- What does the SD-WAN distribute?  Is it a configuration or some kind of credential?

** Section 3.2
   Homogeneous Encrypted SD-WAN has some properties similar to the
   commonly deployed IPsec VPN, albeit the IPsec VPN is usually point-
   to-point among a small number of nodes and with heavy manual
   configuration for IPsec between nodes. In contrast, an SD-WAN
   network can have many edge nodes and a central controller to manage
   the configurations on the edge nodes.

There is a contrast being made that I don’t understand.  If the SD-WAN can manage the configuration of many edge notes, why can’t it manage IPSec VPNs on many edge nodes.  Doesn’t Section 4.3 describe IPsec provisioning?  Aren’t IPsec VPNs part of what can secure some of the links over commodity internet links?

** Section 3.3
     Also, the connection between C-PEs and their Controller (RR) might
     be via the untrusted public network. It is necessary to encrypt
     the communication between RR and C-PEs, by TLS, DTLS, etc.

-- Is the use of TLS and DTLS are hard requirement?  

-- Why do other sections say “secure transport (e.g., TLS, DTLS ...)”?

** Section 3.4.  Editorial.
   Throughout this document, this scenario is also called Internet
   Offload for Private VPN, or PE-based SD-WAN.

My crude search suggest that these scenario names are not used in the rest of the document.

** Section 4.2.  Editorial.
   Policies are needed
   to govern which underlay paths can carry an application flow, as
   described by Section 8 of MEF70.1

Something got mangled in the document rendering.  What is “MEF70.1”?

** Section 4.3
   SD-WAN edge nodes must negotiate the supported IPsec encryption
   algorithms (AES256, AES192, AES128, etc.), the hash algorithm (SHA2
   512, SHA2 384, SHA2256, etc.), and the DH groups to establish IPsec
   tunnels between them.

Recommend removing the different algorithm names.  At the level of abstraction of this text, it isn't clear that it is necessary to even call which IPsec parameters need to get negotiated. 

SD-WAN edge nodes must negotiate various cryptographic parameters to establish IPsec tunnels between them.

** Section 4.3.
   For a BGP-controlled SD-WAN, BGP UPDATE
   messages can propagate each node's IPsec-related attribute values
   for peers to choose the common values supported, traditionally done
   by IPsec IKEv2 [RFC7296].

Please provide a reference to the specification which provides guidance on the BGP TLVs to provision IPsec.  Is that draft-ietf-idr-sdwan-edge-discovery?  This should be a normative reference.

** Section 5.1

   For an SD-WAN network with a small number of nodes, the traditional
   hub and spoke model utilizing Next Hop Resolution Protocol (NHRP) or
   Dynamic Smart VPN (DSVPN)/Dynamic Multipoint VPN (DMVPN) protocol
   has been found to work reasonably well.

-- Please provide a citation to DSVPN and DMVPN?

-- What does “work[s] reasonably well” intended to convey?  Does it not “work”?

** Section 5.2
   In the figure below, the BGP UPDATE message from C-PE2 to RR can
   have the client routes encoded in the MP-NLRI Path Attribute and the
   IPsec Tunnel associated information encoded in the Tunnel-Encap
   [RFC9012] Path Attributes as described in the [SECURE-EVPN].

Where in [SECURE-EVPN] is this relevant guidance?  Please provide the relevant section in the text.

** Section 8.
   2) Potential risk of illegal traffic being injected via the
   Internet-facing WAN ports.

What is “illegal” traffic?  What are the issues of legality?  Is this perhaps “malicious traffic” or “traffic prohibited by policy”?

** Section 8.  The different deployment models also seem to place different levels of trust in the service providers.  Consider mentioning these differences.
Jim Guichard
No Objection
Comment (2023-09-25 for -15) Sent
Several nits & comments for the authors to consider:

## Section 1:
     - Some traffic can be forwarded by edge nodes, based on their
       application identifiers instead of destination IP addresses, by
       placing the traffic onto specific overlay paths based on the
       application-specific policies.

* Do the authors have a reference for this - does this imply DPI requirements or some other method to determine the application carried within the packets? What specifically is an 'application identifier'?

## Section 3.1.1

* Please expand on MPLS-VPN on the first usage with reference - RFC8388 does not document MPLS-VPN as a well-known term.

## Section 3.1.1:
   For IP-based client interfaces, L3VPN service requirements are

* What are these 'L3VPN service requirements'? is there a reference so that the reader can understand what these requirements are?

## Section 3.1.3

* The figure has no reference - it should be 'Figure 1' - Please correct this.

## Section 3.2

* Expand 'SA' to 'Security Association (SA)' on first usage.

## Section 3.4

* Please provide references for 'MPLS-in-IP/GRE-in-IPsec'.

## Section 5.1
   Dynamic Smart VPN (DSVPN)/Dynamic Multipoint VPN (DMVPN) protocol
   has been found to work reasonably well.

* Neither DSVPN nor DMVPN are protocols - suggest removing 'protocol' from the above text. Also, you should provide a reference for both of these for readers who are not familiar with these solutions.

## Section 5.2
   When using two separate BGP UPDATE messages, which is described by
   Section 4 and 8 of [RFC9012], UPDATE U1 has its Nexthop to the node
   loopback address and is recursively resolved to the IPsec SA tunnel
   detailed attributes advertised by the UPDATE U2 for the Node
   Loopback address.

* Section 4 of RFC9012 simply describes the Encapsulation Extended Community so I am not sure how that is relevant to the above paragraph. Section 8 describes recursive next-hop resolution which seems to be what you are describing here so is probably the only reference that you need.
Warren Kumari
No Objection
Comment (2023-10-02 for -15) Sent
I am balloting NoObjection, but am quite uncomfortable with it -- I will steal some of Eric V's ballot text - "... I find it ***weak*** on several points (see below) and more accurate language will be welcome rather than hand waving. I strongly suggest that the authors/WG have a second pass on the document."

The document is very hand-wavey, and, based on the number of open comments and nits, does not appear to have had significant detailed review; I almost balloted DISCUSS because of this.

This document would benefit from copy-editing.

1: The document opens with: "Here are some of the main characteristics of "SD-WAN" networks: [...]"
 -- I don't really think that most of these are "the main characteristics of "SD-WAN" networks" - e.g: "Instead of all traffic hauled to Corporate HQ for centralized policy control, direct Internet breakout from remote branch offices is allowed." - this is a characteristic of many many different technologies and VPN solutions. The implication of this sentence so early in the document is that a: these are differentiating characteristics, and b: that they are the **main** characteristics.  

2: Editorial: "Instead of all traffic hauled to Corporate HQ for centralized policy control, direct Internet breakout from remote branch offices is allowed." - "Instead of hauling all traffic to ..." or "Instead of having all traffic hauled to..." or...

3: "Some traffic can be forwarded by edge nodes, based on their application identifiers instead of destination IP addresses, by placing the traffic onto specific overlay paths based on the application-specific policies." - the subject of this sentence is very unclear. What is this "some traffic" (why not all? what does "some" do in this sentence?). Same for "their" in "their application identifiers". 

4: "CPE-Based VPN: Virtual Private Secure network formed among CPEs. This differentiates from more commonly used PE-based VPNs". How does "VPN" expand to "Virtual Private Secure network"? **How** this this "differentiates" from more commonly used PE-based VPNs? ("This differs from more commonly used PE-based VPNs as it is formed between CPE" would sort of work, but it is still very unclear *why* this is being noted.

5: "SD-WAN:     Software Defined Wide Area Network is an overlay network ..." - this is a much better definition than how the document was opened. I suggest that it be used if you want to introduce what SD-WAN **is**.

6: "Conventions used in this document [...] SD-WAN over Hybrid Networks: SD-WAN over Hybrid Networks typically have edge nodes...
You never actually use that term - you *do* use Hybrid Underlay (e.g "SD-WAN with Hybrid Underlays") a bunch of times though.

7: "For multicast packets forwarding" -> "For multicast packet forwarding"
Éric Vyncke
No Objection
Comment (2023-10-02 for -15) Sent for earlier
# Éric Vyncke, INT AD, comments for draft-ietf-bess-bgp-sdwan-usage-15

Thank you for the work put into this document. It could be useful but, honestly, I find it ***weak*** on several points (see below) and more accurate language will be welcome rather than hand waving. I strongly suggest that the authors/WG have a second pass on the document.

Please find below some non-blocking COMMENT points (but replies would be appreciated even if only for my own education), and some nits.

Special thanks to Matthew Bocci for the shepherd's detailed write-up including the WG consensus ***and*** the justification of the intended status. 

Please note that Juan-Carlos Zúñiga is the Internet directorate reviewer (at my request) and you may want to consider this int-dir review as well when it will be available (no need to wait for it though):

I hope that this review helps to improve the document,




## Mixing IPsec SA and IPsec-protected tunnels

I am afraid that the text often mixes "IPsec SA" and "IPsec-protected tunnels" (the latter could be VXLAN/GRE/IPinIP tunnels).

## Absence of QUIC

Suggest to also add QUIC as a secure channel in addition to TLS.

## Abstract

Unsure about the usefulness of the last paragraph.

Isn't "SW-WAN overlay" an pleonasm ? I.e., are there SD-WAN without an overlay ?

## Section 1

`based on their application identifiers instead of destination IP addresses` what are those "application identifiers" ?

## Section 2

Should "PE-based VPN" be defined on its own ?

`plain Internet services` what are those services ? I guess mainly "IP packet forwarding", but this should be qualified.

Please explain what "ZTP" is.

## Section 3.1.1

Which part of the inner header is relevant to `the IPsec tunnel's inner encapsulation header can have the SD-WAN VPN Identifier to distinguish the packets belonging to different SD-WAN VPNs.` ?

## Section 3.1.2

What are `L3VPN service requirements` ?

Please expand "HR" as it is not part of

## Section 3.1.5

Even of a SD-WAN edge does not receive a BGP UPDATE for a specific prefix, a manual static route could still forward traffic to this prefix. Should this security issue be mentioned ?

## Section 3.2

Please add a reference to "NVo3".

"Homogeneous" also seems to indicate that all IPsec SAs use the same security parameters (crypto alg, key length, ...). Is it mandatory for this document ? If not, then suggest to use "ubiquitous encrypted SD-WAN".

## Section 3.3

Unsure whether I agree with `it is more desirable for traffic over a private VPN to be forwarded without encryption.`.

## Section 5.1

The first paragraph mentions DMVPN/DSVPN that appears to be marketing terms from some vendors. Are those terms required ?

## Sections 5.2, 5.3, ...

Rather than using legacy IPv4 addresses, IPv6 examples would be more suitable in 2023.

## Section 5.4

If is assumed to be a prefix, then it is *NOT* as it is part of

## Section 6 does it belong to a BGP document ?

I fail to see how the whole section 6 is part of a BGP document. It is nice to read but what is the link with BGP ?

## Section 6.1.1

`six one-directional IPsec SAs must be established:` but then only 3 are show, suggest to either do not list the bidirectional arrows or use unidirectional arrows.

## Section 6.1.2

What is "inner encapsulation key" in `Different client traffic can be differentiated by a unique value in the inner encapsulation key or ID field` ?

Why are loopback addresses used here ? I would have assumed that for a public edge CPE, it would be the address of the public interface.

Neither IPv6 nor IPv4 headers have a header field "Protocol-code". Please use the right term.

The use of 'protected' for IPsec tunnel is ambiguous (at first read, I read it as *very secure/confidential*) in `C-PE Node-based IPsec tunnel is inherently protected`. E.g., in section 6.2 'protected' is used with a different semantic.

Is it really Figure 2 from pages before ?

I would expect that VXLAN/GRE encapsulation would cope better with multicast traffic. Would RFC 3740 and 6407 be useful for mcast traffic ?

## Section 6.2 (and other places)

The usual term is 'clear text' rather than 'native'.

## Section 6.2.2

Please describe or add a reference for `additional anti-DDoS mechanism`.

Figure 8 probably misses a link between A3 and PE.

## Section 6.3

Unsure whether the NAT part is useful, again what has this to do with BGP ? And/or how can a BGP session traverse a NAT ?

What is `Scenario #2` ?

## Section 8

I would have expected some security consideration about the centralization of security in the RR.

## Section 11

It is inconsistent with the authors' list in the header.


## Section 1

s/IP Packets/IP packets/

A couple of places where "Tunnel" should rather be "tunnel".
Andrew Alston Former IESG member
Yes (for -15) Unknown

Lars Eggert Former IESG member
(was Discuss) No Objection
No Objection (2024-02-08 for -20) Sent for earlier
# GEN AD review of draft-ietf-bess-bgp-sdwan-usage-15

CC @larseggert

Thanks to Dan Romascanu for the General Area Review Team (Gen-ART) review

## Comments

### Section 10.1, paragraph 1
     [BCP195]  RFC8996, RFC9325.
Not really acceptable as a normative reference.

### Missing references

No reference entries found for these items, which were mentioned in the text:
`[RFC8388]`, `[RFC6514]`, `[RFC7988]`, and `[RFC6513]`.

### Inclusive language

Found terminology that should be reviewed for inclusivity; see for background and more

 * Terms `traditionally` and `traditional`; alternatives might be `classic`,
   `classical`, `common`, `conventional`, `customary`, `fixed`, `habitual`,
   `historic`, `long-established`, `popular`, `prescribed`, `regular`,
   `rooted`, `time-honored`, `universal`, `widely used`, `widespread`
 * Terms `natively` and `native`; alternatives might be `built-in`,
   `fundamental`, `ingrained`, `intrinsic`, `original`

## Nits

All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via, so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

### Boilerplate

Document still refers to the "Simplified BSD License", which was corrected in
the TLP on September 21, 2021. It should instead refer to the "Revised BSD

There is other outdated content in the boilerplate.

### Uncited references

Uncited references: `[MEF70.1]`.

### Outdated references

Document references `draft-ietf-rtgwg-net2cloud-problem-statement-26`, but
`-30` is the latest available revision.

### Grammar/style

#### Section 3.1.3, paragraph 5
uthorized to communicate with a small number of other SD-WAN edge nodes. In t
Specify a number, remove phrase, use "a few", or use "some". (Also elsewhere.)

#### Section 6.2.2, paragraph 6
 March 2004. [RFC3947] T. Kivinen, et al, "Negotiation of NAT Traversal in t
A period is misplaced or missing. (Also elsewhere.)

#### Section 6.2.2, paragraph 9
ov 2021. 11. Acknowledgments Acknowledgements to Andrew Alston, Adrian Farre
Do not mix variants of the same word ("acknowledgement" and "acknowledgment")
within a single text.

## Notes

This review is in the ["IETF Comments" Markdown format][ICMF], You can use the
[`ietf-comments` tool][ICT] to automatically convert this review into
individual GitHub issues. Review generated by the [`ietf-reviewtool`][IRT].