Network Working Group Eric C. Rosen
Internet Draft Cisco Systems, Inc.
Expiration Date: January 2004
July 2003
Applicability Statement for BGP/MPLS IP VPNs
draft-ietf-l3vpn-as2547-01.txt
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Abstract
This document provides an Applicability Statement for the VPN
solution described in [2547bis] and other documents listed in the
References section.
Rosen [Page 1]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
Table of Contents
1 Introduction ......................................... 2
2 SP Provisioning Model ................................ 4
3 Supported Topologies and Traffic Types ............... 6
4 Isolated Exchange of Data and Routing Information .... 7
5 Access Control and Authentication .................... 9
6 Security ............................................. 9
6.1 Protection of User Data .............................. 9
6.2 SP Security Measures ................................. 10
7 Addressing ........................................... 12
8 Interoperability and Interworking .................... 13
9 Network Access ....................................... 13
9.1 Physical/Link Layer Topology ......................... 13
9.2 Temporary Access ..................................... 13
9.3 Access Connectivity .................................. 14
10 Service Access ....................................... 14
10.1 Internet Access ...................................... 14
10.2 Hosting, ASP, Other Services ......................... 15
11 SP Routing ........................................... 15
12 Migration Impact ..................................... 16
13 Scalability .......................................... 16
14 QoS, SLA ............................................. 19
15 Management ........................................... 21
15.1 Management by the Provider ........................... 21
15.2 Management by the Customer ........................... 21
16 Acknowledgments ...................................... 22
17 References ........................................... 22
18 Author's Address ..................................... 24
1. Introduction
This document provides an Applicability Statement for the VPN
solution described in [2547bis] and other documents listed in the
References section. We refer to these as "BGP/MPLS IP VPNs". It
does so by comparing the characteristics of BGP/MPLS IP VPNs against
the requirements specified in [PPVPN-REQs].
A VPN service is provided by a "Service Provider" (SP) to a
"Customer" (sometimes referred to as an "Enterprise") . BGP/MPLS IP
VPNs are intended for the situation in which:
Rosen [Page 2]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
- The Customer:
* uses the VPN only for carrying IP packets.
* does not want to manage a routed backbone; the customer may
be using routing within his sites, but wishes to outsource
the inter-site routing to the SP.
* wants the SP to make the backbone and its routing completely
transparent to the customer's own routing.
If the customer has a routed infrastructure at his sites, he
does not want his site routing algorithms to need to be aware
of any part of the SP backbone network, other than the PE
routers to which the sites are attached. In particular, the
customer does not want his routers to need to be aware of
either the native structure of the SP backbone, or of an
overlay topology of tunnels through the SP backbone.
- The Service Provider:
* has an IP backbone, with MPLS-enabled edge routers, and
possibly (though not necessarily) with MPLS-enabled core
routers.
* wants to provide a service which meets the customer
requirements above.
* does not want to maintain a distinct overlay topology of
tunnels for each customer.
The basic principle is to model each VPN as a self-contained
"internet", where each site makes one or more access connections to a
SP, sends the SP its routing information, and then relies on the SP
to distribute routing information to and from the other sites in that
same VPN. The service differs from Internet service, however, in
that the SP strictly controls the distribution of this routing
information so that routes from within a VPN are not sent outside the
VPN, unless that is explicitly authorized by the customer. In fact,
even within the VPN, the distribution of routes may be controlled by
the SP so as to meet some policy of the customer.
The routers at a given customer site need not be routing peers of the
routers at other customer sites, and indeed need not know anything
about the internal structure of other customer sites. In fact,
different routing protocols may run at the different sites, with each
site using whatever protocol is most appropriate for that particular
site.
Rosen [Page 3]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
If EBGP is used on the PE/CE access links, the SP and the customer do
not peer in any IGP.
BGP/MPLS IP VPNs are optimized for the situation in which a customer
(an enterprise) expects a service provider to operate and maintain
the customer's "backbone" (i.e., the customer's inter-site routing).
As such, the service provider becomes a "business partner" of the
enterprise. The technical mechanisms accommodate the case in which a
number of closely cooperating SPs can jointly offer the VPN service
to a customer, in that the BGP-based route distribution mechanisms
can operate between different SPs. If a set of SPs have sufficient
agreements with respect to QoS, SLA, etc., then the customer's VPN
could have sites attached to different SPs from that set.
BGP/MPLS IP VPNs are NOT an appropriate solution for the customer who
wants to purchase local connectivity from whatever ISP happens to
offer the best price at a given site, nor for the customer who wants
his VPN backbone to consist of tunnels running over a public Internet
service.
[2547bis] specifies the inter-AS mechanisms that allow a single VPN
to have sites attached to different SPs. However, the design center
is not an environment where a given VPN is spread among a very large
number (e.g., hundreds) of SPs.
In cases where remote offices, individual telecommuters, etc., must
use the public Internet to access the VPN, it is possible to "tunnel"
the remote traffic to a PE router, and the PE router will treat the
traffic as if it had arrived over an interface connected to the PE.
Remote PPP connections can be tunneled via L2TP to a PE router; IPsec
tunnels can also be used to tunnel traffic to a PE router across the
public Internet. Of course when the public Internet is used, issues
such as QoS and SLAs must be carefully considered.
2. SP Provisioning Model
If a particular VPN attaches to a particular PE router, the SP must
configure that PE router with a VRF ("Virtual Routing and Forwarding
table"), a routing table that is specific to the specified VPN.
(This is known as a VFI, "VPN Forwarding Instance", in the language
of [PPVPN-REQ] and [PPVPN-FRMWRK].) Each interface or sub-interface
at that PE which attaches to a site in the specified VPN (i.e., each
local access link of that VPN) must be configured so as to be
associated with that VRF. Each such interface may be unnumbered, or
may be assigned an address which is unique within the VPN's address
space. In general, a routing algorithm needs to be run on each of
these links (though static routing can be used instead). The routing
Rosen [Page 4]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
algorithm can be EBGP, or an IGP such as RIP or OSPF. (If OSPF is
used, the procedures of [2547-OSPF] must be implemented.) In an IGP
is run on the access links, the IGP MUST be a separate IGP instance,
different than the IGP instance running among the backbone routers,
and different than the IGP instance running on the access links of
any other VPN. Static routing is also allowed.
The VRF is populated automatically with routes distributed from
locally attached CE routers via whatever routing algorithm is run on
the PE/CE links. It is also populated automatically with routes
distributed from other VRFs via BGP. Standard routing decision
processes are used to automatically select the proper routes. Static
configuration of routes in the VRF is optional.
Each PE router must run BGP, and must be pre-configured with the
identities of a small set of BGP Route Reflectors, with which it is
to peer via IBGP.
In lieu of using Route Reflectors, one could configure each PE with
the identities of all the other PEs, and set up a full mesh of IBGP
connections. While this might be adequate for small networks, it
would not scale well to large networks; the use of Route Reflectors
is necessary to achieve scalability. See section 4.3.3 of [2547bis]
for a more complete discussion of the use of Route Reflectors, and
related scalability mechanisms such as Outbound Route Filtering.
Each VRF must be configured with three parameters:
- A Route Distinguisher. This is a globally unique 8-byte value.
Each VRF may have a unique RD, or there may be a single unique RD
for an entire VPN. When BGP is used to distribute VPN routing
information across the SP backbone, this value is prepended to
the VPN's IPv4 address prefixes, creating a new address family,
the VPN-IPv4 address family. Thus even when two VPNs have
overlapping IPv4 address spaces, they have unique VPN-IPv4
address spaces.
- One or more Export Route Targets. An RT is a globally unique 8-
byte value which BGP carries, as the Extended Communities Route
Target attribute, along with routes that are exported form the
VRF.
- One or more Import Route Targets. This RT is used to select
routes to be imported from other VRFs into this VRF.
In the simplest cases and most common cases, the Export RT, Import
RT, and RD can be identical, and all VRFs in the same VPN will
distribute routes to each other (a typical intranet). In more
Rosen [Page 5]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
complex cases, they can be set differently, allowing a very fine
degree of control over the distribution of routes among VRFs. This
can be used to create extranets, or to enforce various customer
policies. In complicated cases, particular Export RTs can be
assigned to particular routes using router management mechanisms.
One advantage to not requiring the RD to be the same as any RT is
that this may allow an RD value to be automatically determined for
each VRF; RT values on the other hand must always be configured.
Adding a new site to a VPN is a matter of attaching the site's CE
router to a PE router, configuring the interface, and, if a VRF for
that VPN already exists in the PE router, associating that interface
with the VRF. If a VRF for that VPN does not already exist in the
PE, then one must be configured as specified above. Changes to the
configuration of a PE are automatically reflected via BGP to the
other PEs.
The RTs and RDs are made unique by being structured as an SP
identifier followed by a number which is assigned by the identified
SP. SPs may be identified by their AS numbers, or by a registered IP
address owned by that SP.
Although RTs are encoded as BGP Extended Communities, the encoding
itself distinguishes them from any other kind of BGP Extended
Community.
3. Supported Topologies and Traffic Types
The scheme is optimized for full inter-site connectivity, in the
sense that this is what the simplest configurations provide.
However, the SP has full control, through the mechanism of Route
Targets, of the distribution of routing information among the set of
VRFs. This enables the SP to provide hub-and-spoke or partial mesh
connectivity as well as full mesh connectivity.
Note that, strictly speaking, the scheme does not create a topology,
as it does not create layer 2 connections among the sites. It does
however allow for control over the IP connectivity among the sites.
It is also possible to constrain the distribution of routing in
arbitrary ways, e.g., so that data from site A to site B must travel
through a third site C. (In fact, if it is desired to do so, this
level of control can be specified at the granularity of a single
route.)
It is possible for some of the routes from a particular customer site
A to be distributed to one set of remote sites, while other routes
Rosen [Page 6]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
from site A are distributed to a different set of remote sites. This
is done with the Route Target mechanism previously described.
Unicast IP traffic is fully supported. Customer IP packets are
passed transparently.
Multicast IP traffic is optionally supported, if the SP provides the
optional mechanisms of [2547-MCAST]. There are however scaling
implications to the use of these mechanisms. Discussion of these
implications is deferred.
Non-IP traffic is not supported. If support for non-IP traffic is
necessary, either the SP must additionally provide a layer 2
tunneling service, or the customer must use IP tunneling.
In general, customer routers at different sites do not become routing
peers. However, a customer may, if he so desires, allow routers at
different sites to be routing peers over a link which is NOT part of
the VPN service. Such peering relationships are known as "IGP
backdoors". To ensure the proper operation of routing when IGP
backdoors are present, each VPN route that is distributed by the SP
is distributed along with a corresponding routing metric. This
enables the customer's IGP to compare the "backdoor routes" properly
with the routes that use the SP backbone. In the particular case
wherea customer running OSPF within his sites wishes to have IGP
backdoors, he should run OSPF on the PE-CE link, and the PEs should
run the procedures of [2547-OSPF]. (The CEs do NOT require any
special OSPF procedures.)
4. Isolated Exchange of Data and Routing Information
The Route Target mechanism is used to control the distribution of
routing information, so that routes from one VPN do not get sent to
another. VPN routes are treated by BGP as a different address family
than general Internet routes. Routes from a VRF do not get leaked to
the Internet unless the VRF has been explicitly configured to allow
it (and this is NOT the default).
The way in which a particular VPN is divided into sites, or the
topology of any particular VPN site, are hidden from the Internet and
from other VPNs. (Of course, if a particular site can receive
Internet traffic, and if it responds to traceroute probes from the
Internet, then any user of the Internet can learn something about the
site topology. The fact that the site is in a VPN does not make this
any easier or any harder.)
Similarly, Internet routes do not get leaked into the VPN, unless a
Rosen [Page 7]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
VRF of that VPN is explicitly configured to import the Internet
routes.
Proper configuration is essential to maintaining the isolation. In
particular, each access link must be associated with the proper VRF
for that access link, and each VRF must be configured with the proper
set of RTs.
A number of means for exchanging reachability information between the
PE and CE devices are supported: static routing, EBGP, and RIP are
supported by the procedures of [2547bis]. If the procedures of
[2547-OSPF] are implemented, OSPF may be used; if OSPF is used, and
any VPN access link is an area 0 link, the procedures of [2547-OSPF-
Area0] must be used as well. If OSPF is used between two VPN sites
which are in the same OSPF area, and if it is desired for routes over
the VPN backbone to be preferred to the OSPF intra-site routes, then
the "sham link" procedures of [2547-OSPF] must be used.
The routing protocols used among the customer routers are not in any
way restricted by the VPN scheme, as whatever IGP is used within the
VPN, the PE/CE access links may run EBGP, or may otherwise be in a
different routing domain than the site's internal links.
BGP is used for passing routing information among SPs. BGP may be
authenticated by use of the TCP MD5 option, or by operating through
an IPsec tunnel.
Data traveling between two customer sites is encapsulated while in
transit through the backbone. The encapsulation contains sufficient
information to ensure that the packet is sent to the proper PE
router, and then, in conjunction with the VRF and related information
at that PE, to the proper CE routers.
If two VPNs attach to the same PE, there is strict separation of
forwarding at that PE, as well as strict separation of the routing
information.
Isolation of traffic is similar to that provided by classical L2 VPNs
(FR-based and ATM-based). As in classical L2 VPNs, the customer must
rely on the SP to properly configure the backbone network to ensure
proper isolation, and to maintain the security of his communications
gear.
Rosen [Page 8]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
5. Access Control and Authentication
No particular means of PE/CE authentication is specified for BGP/MPLS
IP VPNs. PE/CE mutual authentication may be done via any mechanism
supported by the routing protocol in which the CE and PE are peers
(e.g., use of the TCP MD5 authentication when the CE/PE protocol is
BGP), or by any other mechanism that may be desired. With such
mechanisms in place, a CE may not join a VPN until the CE
authenticates itself to the Service Provider.
There is however no standardized method which requires a CE to
authenticate itself to the customer network (rather than to the SP)
before the CE is allowed to join the VPN. This is for further study.
No particular means is specified for controlling which user data
packets can be forwarded by BGP/MPLS IP VPNs. BGP/MPLS IP VPNs are
compatible with ACLs and any other filtering features that are
supported on the PE routers. Routing can be set up so that extranet
traffic is directly through a firewall, if that is desired.
It is possible for various sorts of "tunnel interfaces" to be
associated with a VRF. In this case, whatever authentication is
natively used in the establishment of the tunnel interface may be
used. For example, an IPsec tunnel can be used as an "access link"
to attach a remote user or site to a VRF. The authentication
procedure in this case is part of IPsec, not part of the VPN scheme.
Where L2TP is used, each PPP session carried in an L2TP tunnel can be
associated with a VRF. The SP's AAA server can be used to determine
the VPN to which the PPP session belongs, and then the customer's
AAA server can be given the opportunity to authenticate that session
as well.
6. Security
6.1. Protection of User Data
No particular means of ensuring user data security is specified for
BGP/MPLS IP VPNs.
The optional procedures of [2547-IPsec] may be used to provide
authentication and/or encryption of user data as it travels from the
ingress PE to the egress PE. However, the data is exposed at those
two PEs, as well as on the PE/CE access links.
The customer may provide his own user data security by using IPsec
tunnels that terminate within the customer sites. Such tunnels are
Rosen [Page 9]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
transparent to the VPN scheme. Schemes that discover the remote
tunnel endpoints automatically and then set up the tunnels
automatically as needed are the best fit with this VPN technology.
Note that there is no requirement in general that IPsec tunnels
between customer sites terminate at CE routers.
The use of end-to-end transport mode IPsec by the customer is also
transparent to the VPN scheme. In fact, the VPN scheme is compatible
with any use of security by the customer, as long as a cleartext IP
header is passed from CE to PE.
When data must cross the Internet to reach the ingress PE router,
IPsec tunnels between the enduser and the PE router can be used; the
PE router must then associate each IPsec tunnel with the proper VRF.
This association would have to be based on user-specific information
provided by the IKE exchange, such as a VPN-id.
If data is going from one SP network to another, and must cross the
public Internet to get between those two networks, IPsec tunnels can
be used to secure the data. This would require bilateral agreement
between the two SPs. BGP connections can also be passed through an
IPsec tunnel if this is deemed necessary, in order to protect user
data, by a pair of SPs. QoS/SLA factors would have to be carefully
considered in this case.
6.2. SP Security Measures
The SP is responsible for preventing illegitimate traffic from
entering a VPN. VPN traffic is always encapsulated while traveling
on the backbone, so preventing illegitimate traffic is a matter of
ensuring that the PE routers to the encapsulation/decapsulation
correctly, and that encapsulations have not been "spoofed", i.e.,
that the encapsulated packets were actually encapsulated by PE
routers.
This requires the SP to take various security measures. The PE and P
routers must themselves be secure against breakins (either from
someone physically present or from the Internet), and neither P nor
PE routers should form routing adjacencies to other P or PE routers
without benefit of some kind of security. This may be authentication
in the IGP, or physical security.
The CE/PE access link should be secured in some manner, though the
provider may make it the responsibility of the customer to ensure
that the CE is secure from compromise. If the CE/PE access link is a
tunnel over the Internet, then of course some sort of authentication
protocol should always be used.
Rosen [Page 10]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
LDP sessions and BGP sessions between PE and/or P routers should be
authenticated. This can be done via the TCP MD5 option, or by use of
IPsec.
If the SP is providing the VPN service over an MPLS backbone, it
should not accept MPLS packets from its external interfaces (i.e.
interfaces to CE devices or to other providers' networks) unless the
top label of the packet was legitimately distributed to the system
from which the packet is being received. If the packet's incoming
interface leads to a different SP (rather than to a customer), an
appropriate trust relationship must also be present, including the
trust that the other SP also provides appropriate security measures.
If the SP is providing the VPN service by using an IP (rather than an
MPLS) encapsulation, or if it accepts IP-encapsulated VPN packets
from other SPs, it should apply filtering at its borders so that it
does not accept from other SPs or from customers any IP packets which
are addressed to the PE routers, unless appropriate trust
relationships are in place.
Cryptographic authentication of the encapsulated data packets is
certainly advantageous when there are multiple SPs providing a single
VPN.
When a dynamic routing protocol is run on the link between a CE
router and a PE router, routing instability in the private network
may have an effect on the PE router. For example, an unusually large
number of routing updates could be sent from the CE router to the PE
router, placing an unusually large processing load on the PE router.
This can potentially be used as a Denial of Service attack on the PE
router.
This issue can be mitigated via resource partitioning in the PE, in
order to limit the amount of resources (e.g., CPU and memory) which
any one VPN is permitted to use in PE routers. Also, rate limits may
be applied to the routing traffic sent from the CE to the PE.
Alternately, when this problem is detected, the CE to PE interface
may be shut down.
Network management traffic from the CE to the PE may be rate limited
(for example, to prevent network management traffic from CE to PE to
be used in a DOS attack).
Rosen [Page 11]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
7. Addressing
Overlapping customer addresses are supported. There is no
requirement that such addresses be in conformance with RFC 1918.
There is no requirement that customer VPN addresses be distinct from
addresses in the SP network.
Any set of addresses used in the VPN can be supported, irrespective
of how they are assigned, how well they aggregate, whether they are
public or private. However, the set of addresses which are reachable
from a given site must be unique.
Network address translation for packets leaving/entering a VPN is
possible, and is transparent to the VPN scheme.
There is nothing in the architecture to preclude the mechanisms from
being extended to support IPv6, provided that the appropriate IPv6-
capable routing algorithms are in place. I.e., PE/CE routing must
support IPv6, and the PE-PE BGP must support the labeled IPv6 address
family. The latter has not been specified, but its specification is
obvious from the specification of the labeled IPv4 address family.
The IGP used in the SP backbone need not be IPv6 capable in order to
support customer IPv6 networks.
In theory, the same could be said of other network layers, but in
practice a customer who has non-IP traffic to carry must expect to
carry it either in site-to-site IP tunnels, or using some additional
service (such as a layer 2 service) from the SP.
Layer 2 addresses and identifiers are never carried across the SP
backbone.
No restrictions are placed on the customer's addressing schemes or
policies. Note though that the SP may place restrictions on the
number of routes from a given customer site, or may charge
differentially depending on the number of such routes, and such
restrictions may have implications for the customer's addressing
scheme. In particular, addressing schemes which facilitate route
aggregation on a per-site basis will result in the most efficient use
of the SP's resources, and this may be reflected in SP charging
policies.
Rosen [Page 12]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
8. Interoperability and Interworking
Interoperability should be ensured by proper implementation of the
published standards.
Direct PE-PE interworking over the SP backbone with other VPN
solutions is not supported.
As all the different types of L3VPNs are IP networks, they can of
course interwork in the same way that any two IP networks can
interwork. For example, a single site can contain a CE router of one
VPN scheme and a CE router of another VPN scheme, and these CE
routers could be IGP peers, or they might even be the same CE router.
This would result in the redistribution of routes from one type of
VPN to the other, providing the necessary interworking.
9. Network Access
9.1. Physical/Link Layer Topology
The architecture and protocols do not restrict the link layer or the
physical layer in any manner.
9.2. Temporary Access
Temporary access via PPP is possible, using industry standard PPP-
based authentication mechanisms. For example:
- A dial-up user (or other PPP user) is authenticated by the PE,
using the SP's AAA server, based on a login string or on the
number dialed.
- The SP's AAA server returns a VPN-id to PE.
- The PE assigns the user to a VRF, based on that VPN-id.
- The user is then authenticated by a AAA server within the VPN
(i.e., managed by the customer rather than by the SP). This AAA
server would typically be addressed through the VRF (i.e., may be
in VPN's private address space).
- The user gets disconnected if either authentication step is
unsuccessful.
IPsec access to a VRF is also possible. In this case, the security
association is between the enduser and the SP.
Rosen [Page 13]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
In these ways, a user can access a BGP/MPLS IP VPN via the public
Internet.
There is no explicit support for mobility, other than what is stated
above.
9.3. Access Connectivity
Homing of a CE to two or more PEs is fully supported, whether the PEs
are on the same SP network or not.
If a CE is connected to two or more PEs, all its CE-PE links can be
used to carry traffic in both directions. In particular, traffic
from different ingress PEs to a particular CE may arrive at that CE
over different PE-CE links. This depends on the backbone network
routing between the CE and the various ingress PEs.
If a VRF on a particular ingress PE contains several routes to a
particular destination, then traffic from that ingress PE can be
split among these routes. If these routes end with different PE-CE
links, then traffic from that ingress PE will be split among those
links.
BGP contains a multitude of knobs which allow a SP to control the
traffic sent on one PE-CE link as opposed to the other. One can also
make use of the Link Bandwidth extended community [BGP-EXT-COMM] to
control how traffic is distributed among multiple egress PE-CE links.
The VPN scheme is of course compatible with the use of traffic
engineering techniques (RSVP-TE based or otherwise) in the backbone
network.
10. Service Access
10.1. Internet Access
Internet access and VPN access are possible from the same site. This
is even possible over the same interface, as long as the VPN's
internal addresses are distinct from the addresses of the systems
which must be reached via the Internet. This requires only that
Internet routes as well as VPN routes be imported into the VRF
associated with that interface. This may be as simple as putting a
default route to the Internet into that VRF.
The "route to the Internet" which is in a particular VRF need not
lead directly to the Internet, it may lead to a firewall or other
Rosen [Page 14]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
security device at another site of the VPN. The VPN customer can
cause this to happen simply by exporting a default route from the
site with the firewall. Generally, a site with a firewall will use a
different virtual interface for Internet access than for VPN access,
since the firewall needs to distinguish the "clean interface" from
the "dirty interface".
In such a configuration, the customer would export his routes to the
Internet via the firewall's dirty interface, but would export the
same routes to the VPN via the clean interface. Thus all traffic
from the Internet would come through the dirty interface, then though
the firewall, and possibly go to another VPN site though the clean
interface. This also allows any necessary NAT functionality to be
done in the firewall.
10.2. Hosting, ASP, Other Services
Any externally provided service can be accessed from the VPN,
provided that it can be addressed with an address that is not
otherwise in use within the VPN. Access can be firewalled or non-
firewalled. If the client accessing the service does not have a
globally unique IP address, and a single server provides a service to
multiple VPNs, NAT will have to be applied to the client's packets
before they reach the server. This can be done at a customer site,
or by a VRF-specific NAT function in a PE router.
11. SP Routing
Routing through the backbone is independent of the VPN scheme, and is
unaffected by the presence or absence of VPNs. The only impact is
that the backbone routing must carry routes to the PE routers.
The VPN routes themselves are carried in BGP as a distinct address
family, different than the address family that is used to carry
"ordinary" IP routes. These routes are passed from PE router to Route
Reflector to PE router, and are never seen by the P routers. The
Route Reflectors that carry the VPN routes can be entirely separate
from the Route Reflectors that carry the "ordinary" IP routes.
The fact that two PE routers support a common VPN does not require
those PE routers to form an IGP routing adjacency between themselves.
The number of adjacencies in the backbone IGP is independent of and
unrelated to the number of VPNs supported by any set of PE routers.
No VPN-specific protection and restoration mechanisms are needed;
these are general routing considerations, and the VPN scheme is
Rosen [Page 15]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
compatible with any protection and restoration mechanisms that may be
available.
The SP does not manage the customer's IGP in any way, and routes are
never leaked between the SP's IGP and any customer's IGP.
If the PE/CE protocol is EBGP, the SP and the customer do not ever
participate in a common IGP.
12. Migration Impact
Generally, this means replacement of an existing legacy backbone with
VPN backbone. The general migration mechanism would be to hook up
the sites one at a time to the VPN backbone, and to start giving the
routes via the VPN backbone preference to routes via the legacy
backbone. Details depend on the legacy backbone's IGP. In general,
one would have to manipulate the IGP metrics to provide the proper
route preference.
If the legacy backbone routing protocol is OSPF, then migration is
best done with OSPF as the PE/CE protocol and the PE supporting the
[2547-OSPF] procedures, OR with BGP as the PE/CE protocol, and the CE
supporting the BGP/OSPF interaction specified in [2547-OSPF].
With other legacy backbone routing protocols, the proper metrics must
be set at the point (PE or CE) where the BGP routes from the SP
network are being redistributed into the legacy IGP.
13. Scalability
There is no upper limit on the number of VPNs per SP network, as
there is no one box in the SP network which needs to know of all
VPNs. Knowledge of a particular VPN is confined to the PE routers
that attach to sites in that VPN, and to the BGP Route Reflectors
which receive routing data from those PEs; other systems maintain no
state at all for the VPN. Note though that there is no need for any
one Route Reflector to know of all VPNs.
If the SP is providing the VPN service over an MPLS backbone, then
the backbone IGP must carry a host route for every LSP egress node
within the routing domain. Every PE router in the routing domain is
an LSP egress node. If there are VPNs attached to PE routers that
are within the routing domain, as well as PE routers which are in
some second routing domain, then the border routers leading towards
the second routing domain will also be LSP egress nodes. Thus the
sum of the number of PE routers plus number of border routers within
Rosen [Page 16]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
a routing domain is limited by the number of routes that can be
carried within the domain's IGP. This does not seem to create any
practical scalability issue.
There is no upper limit on the number of site interfaces per VPN, as
state for a particular interface is maintained only at the PE router
to which that interface attaches. The number of site interfaces per
VPN at a given PE router is limited only by the number of interfaces
which that PE router can support.
The number of routes per VPN is constrained only by the number of
routes that can be supported in BGP, the number of routes which can
be maintained in the PEs that attach to that VPN, and the number of
routes which can be maintained in the BGP Route Reflectors which hold
the routes of that VPN.
The major constraint in considering scalability is the number of
routes which a given PE can support. In general, a given PE can
support as many VPNs as it has interfaces (including virtual
interfaces or "sub-interfaces", not just physical interfaces), but it
is constrained in the total number of routes it can handle. The
number of routes a given PE must handle depends on the particular set
of VPNs it attaches to, and the number of routes in each such VPN,
and the number of "non-VPN" Internet routes (if any) that it must
also handle.
The SP may need to engage in significant planning to ensure that
these limits are not often reached. If these limits are reached, it
may be necessary either to replace the PE with one of larger
capacity, or to reorganize the way in which access links lead from
CEs to PEs, in order to better concentrate the set of access links
from sites that are in the same VPN. Rehoming a site to a different
PE may not involve actual rewiring; if the access technology is
switched, this is a matter of provisioning, but may still be a
significant undertaking. If it is necessary to have down time while
performing the rehoming, the customer is impacted as well. Rehoming
can also be done "virtually", by creating a layer 2 tunnel from a
CE's "old" PE to its "new" PE.
An important consideration to remember is that one may have any
number of INDEPENDENT BGP systems carrying VPN routes. This is
unlike the case of the Internet, where the Internet BGP system must
carry all the Internet routes. The difference stems from the fact
that all Internet addresses much be reachable from each other, but a
given VPN address is only supposed to be reachable from other
addresses in the same VPN.
Scalability is also affected by the rate of changes in the
Rosen [Page 17]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
reachability advertisements from CE to PE, as changes reported by a
CE to its attached PE may be propagated to the other PEs. BGP
mechanisms to control the rate of reported changes should be used by
the SP.
Another constraint on the number of VPNs which can be supported by a
particular PE router is based on the number of routing instances
which the PE router can support. If the PE/CE routing is static, or
is done by BGP, the number of routing protocol instances in a PE
device does not depend on the number of CEs supported by the PE
device. In the case of BGP, a single BGP protocol instance can
support all CEs that exchange routing information using BGP. If the
CE/PE router is done via RIP or OSPF, then the PE must maintain one
RIP or OSPF instance per VRF. Note that the number of routing
instances which can be supported may be different for different
routing protocols.
Inter-AS scenarios constructed according to option (b) of section 10
of [2547bis] require BGP "border routers" to hold the routes for a
set of VPNs. If two SPs share in a small number of VPNs, a single
border router between them provide adequate capacity. As the number
of shared VPNs increases, additional border routers may be needed to
handle the increased number of routes. Again, no single border
router would handle all the routes from all the VPNs, so an increase
in the number of VPNs can always be supported by adding more border
routers.
Inter-AS scenarios constructed according to option (c) of section 10
of [2547bis] eliminates the need for border routers to contain VPN
routes (thus improving scalability in that dimension), but at the
cost of requiring that each AS have a route to the PEs in the others.
(Inter-AS scenarios constructed according to option (a) of section 10
of [2547bis] do not scale well.)
The solution of [2547bis] is intended to simplify CE and site
operations, by hiding the structure of the rest of the VPN from a
site, and by hiding the structure of the backbone. Thus CEs need
have only a single sub-interface to the backbone, CEs at one site
need not even be aware of the existence of CEs at another, and CEs at
one site need not be routing peers of CEs at another. CEs are never
routing peers of P routers. These factors help to scale the
customer's network, but limiting the number of adjacencies each CE
must see, and by limiting the total number of links which the
customer's IGP must handle.
The solution of [2547bis] is also intended to simplify the SP's VPN
provisioning, so that potentially the SP will have to do little more
Rosen [Page 18]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
than say which sites belong to which VPNs. However, as the system
scales up, planning is needed to determine which PEs should home
which VPNs, and which BGP RRs should take which VPNs' routing
information.
P routers maintain NO per-VPN state at all; the only requirement on
them is to maintain routes to the PE routers. When MPLS is used, a P
router must also maintain one multipoint-to-point LSP for each such
route.
However, certain VPN multicast schemes require per-multicast-group
state in the P routers, summed over all VPNs. Others require only no
state in the P routers at all, but will result in sending more
unnecessary traffic. The complete set of tradeoffs for multicast is
not that well understood yet.
Note that as the scaling of a particular PE is primarily a matter of
the total number of routes which it must maintain, scalability is
facilitated if the addresses are assigned in a way that permits them
to be aggregated. (I.e., if the customers have a sensible addressing
plan.)
When a dynamic routing protocol is run on the link between a CE
router and a PE router, routing instability in the private network
may have an effect on the PE router. For example, an unusually large
number of routing updates could be sent from the CE router to the PE
router, placing an unusually large processing load on the PE router.
This issue can be mitigated via resource partitioning in the PE, in
order to limit the amount of resources (e.g., CPU and memory) which
any one VPN is permitted to use in PE routers. Also, rate limits may
be applied to the routing traffic sent from the CE to the PE.
Alternately, when this problem is detected, the CE to PE interface
may be shut down.
14. QoS, SLA
The provision of appropriate QoS capabilities may require any
combination of the following:
- QoS in the access network.
- Admission control (policing) by the PE router on the ingress
access links.
Rosen [Page 19]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
- Traffic conditioning (shaping) by the PE router on the ingress
access links.
- Traffic engineering in the backbone.
- Intserv/diffserv classification by the PE, for traffic arriving
from the CE. Once the PE classifies the user packets, this
classification needs to be preserved in the encapsulation (MPLS
or IP) used to send the packet across the backbone.
- DSCP mapping.
- DSCP transparency.
- Random Early Discard in the backbone.
None of these features are VPN-specific. The ability to support them
depends on whether the features are available on the edge and core
platforms, rather than on any particular VPN scheme.
MPLS support for differentiated services is detailed in RFC3270
[MPLS-DIFFSERV]. DSCP mapping and transparency is covered in section
2.6 of that document. BGP/MPLS IP VPNs can support either the "hose
model" or the "pipe model", though the hose model is a more natural
fit and is what would tend to be used by default; the addition of
pipes would tend to be done on a more limited basis.
It is possible to use traffic engineering to provide, e.g.,
guaranteed bandwidth between two PEs for the traffic of a given VPN.
The VRF entries for that VPN in each PE need to be modified so that
the traffic to the other PE is directed onto the traffic engineered
path. How this is done is a local matter.
Many of the requirements specified in [PPVPN-REQS] stipulate that the
NMS should support SLAs monitoring and verification between the SP
and the various customers by measurement of the indicators defined
within the context of the SLA. The measurement of these indicators
(i.e.: counters) can be achieved when 2547-like VPNs are used by
employing a combination of the PPVPN-MPLS-VPN-MIB [PPVPN-MIB] as well
as other standard MIBs such as the IF-MIB [IF-MIB]. Devices
supporting these MIBs can calculate SLAs based on real-time
performance measurements using indicators and threshold crossing
alerts. Devices can make these thresholds configurable either via a
management interface such as SNMP.
Rosen [Page 20]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
15. Management
The PPVPN Requirements document [PPVPN-REQS] stipulates that The term
"Provider Provisioned VPN" refers to VPNs for which the service
provider participates in management and provisioning of the VPN. RFC
BGP/MPLS IP VPNs can be provisioned and managed to meet these
requirements. The following subsections will outline how devices
supporting RFC 2547-like VPNs can satisfy these requirements.
15.1. Management by the Provider
The SP manages all the VPN-specific information in the PE device.
This can be done using the PPVPN-MPLS-VPN-MIB [PPVPN-MIB], in
combination with other standard MIBs such as IF-MIB [IF-MIB], and
other MPLS MIBs [LSRMIB], [LDPMIB], [TEMIB], [FTNMIB].
Devices supporting RFC 2547-like VPNs that employ the management
interface characteristics described above will also support the ITU
TMN 'FCAPS' functionalities as required in the the PPVPN Requirements
document. These include: Fault, Configuration, Accounting,
Provisioning, and Security.
In BGP/MPLS IP VPNs, the SP is not required to manage the CE devices.
However, if it is desired for the SP to do so, the SP may manage CE
devices from a central site, provided that a route to the central
site is exported into the CE's VPN, and the central site is in a VPN
into which the routes to the managed CE devices have been imported.
This is a form of extranet.
If the central site is managing CE devices from several VPNs, those
CE devices must have mutually unique addresses. Note that this does
not enable the CE devices from different VPNs to reach each other.
The CE devices have no VPN-specific information in them. Hence the
fact that they are connected together into a VPN does not require
them to have any VPN-specific management MIBs or capabilities.
15.2. Management by the Customer
CE devices may be managed from within the VPN, transparently to the
SP. The CE devices have no VPN-specific information in them, and
the fact that they are tied together into a VPN does not impact the
customer's management of them.
Customer access to a PE device is totally at the discretion of the
SP, but is not required by the solution. The PE device is a routing
Rosen [Page 21]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
peer of a CE device, and can be pinged, etc.
If a customer is permitted to access the PE router for management
purposes, the functions available to any particular customer need to
be strictly controlled, and the use of resource partitioning may be
appropriate.
Network management traffic from the CE to the PE may be rate limited
(for example, to prevent network management traffic from CE to PE to
be used in a DOS attack).
16. Acknowledgments
Many thanks to Jeremy De Clercq, Luyuan Fang, Dave McDysan, Ananth
Nagarajan, Yakov Rekhter, and Muneyoshi Suzuki, for their comments,
criticisms, and help in preparing this document. Thanks also to
Thomas Nadeau for his help with the section on management, to
Francois LeFaucheur for his help with the section on QoS, and to Ross
Callon for his review of the document.
17. References
[2547bis] "BGP/MPLS VPNs", Rosen, Rekhter, et. al., draft-ietf-
l3vpn-rfc2547bis-00.txt, May 2003
[2547-GRE-IP] "Use of PE-PE GRE or IP in RFC2547 VPNs", Rekhter,
Rosen, draft-ietf-l3vpn-gre-ip-2547-00.txt, April 2003
[2547-OSPF] "OSPF as the PE/CE Protocol in BGP/MPLS IP VPNs", draft-
ietf-l3vpn-ospf-2547-00.txt, Rosen, Psenak, Pillay-Esnault, June 2003
[2547-OSPF-DNbit] "Using an LSA Options Bit to Prevent Looping in
BGP/MPLS IP VPNs", Rosen, Psenak, Pillay-Esnault, draft-ietf-ospf-
2547-dnbit-00.txt, June 2003
[2547-IPsec] "Use of PE-PE IPsec in RFC2547 VPNs", Rosen, De Clercq,
Paridaens, T'Joens, Sargor, draft-ietf-l3vpn-ipsec-2547-00.txt,
February 2003
[2547-MCAST] "Multicast in MPLS/BGP VPNs", Rosen, Cai, Tappan,
Wijsnands, Rekhter, Farinacci, draft-rosen-vpn-mcast-05.txt, April
2003
[BGP-EXT-COMM] "BGP Extended Communities Attribute", Rekhter, Tappan,
Sangli, draft-ietf-idr-bgp-ext-communities-05.txt, May 2002
Rosen [Page 22]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
[FTNMIB] "Multiprotocol Label Switching (MPLS) FEC-To-NHLFE (FTN)
Management Information Base", Nadeau, Srinivasan, Viswanathan,
draft-ietf-mpls-ftn-mib-07.txt, June 2003
[LDPMIB] "Definitions of Managed Objects for the Multiprotocol Label
Switching, Label Distribution Protocol (LDP)", Cucchiara, et al.,
draft-ietf-mpls-ldp-mib-12.txt, July 2003
[LSRMIB] "MPLS Label Switch Router Management Information Base",
Srinivasan, Viswanathan, Nadeau, draft-ietf-mpls-lsr-mib-11.txt>,
July 2003
[MPLS-DIFFSERV] RFC 3270, "Multi-Protocol Label Switching (MPLS)
Support of Differentiated Services", Le Faucheur, Wu, Davie, Davari,
Vaananen, Krishnan, Cheval, Heinanen, May 2002
[PPVPN-FRMWRK] "A Framework for Provider Provisioned Virtual Private
Networks", Callon, Suzuki, De Clercq, Gleeson, Malis, Muthukrishnan,
Rosen, Sargor, Yu, draft-ietf-l3vpn-framework-00.txt, March 2003
[PPVPN-REQS] "Service requirements for Provider Provisioned Virtual
Private Networks", Carugi, McDysan, Fang, Johansson, Nagarajan,
Sumimoto, Wilder, draft-ietf-l3vpn-requirements-00.txt, April 2003
[PPVPN-MIB] "MPLS/BGP Virtual Private Network Management Information
Base Using SMIv2", Nadeau, et al., draft-ietf-l3vpn-mpls-vpn-mib-
00.txt, November 2002
[IF-MIB] "The Interfaces Group MIB using SMIv2", McCloghrie,
Kastenholtz, RFC 2233, November 1997
[RFC2570] "Introduction to Version 3 of the Internet-standard Network
Management Framework", Case, Mundy, Partain, Stewart, RFC 2570, April
1999
[RFC2571] "An Architecture for Describing SNMP Management
Frameworks", Harrington, Presuhn, Wijnen, RFC 2571, April 1999
[RFC2572] "Message Processing and Dispatching for the Simple Network
Management Protocol (SNMP)" Case, Harrington, Presuhn, Wijnen, RFC
2572, April 1999
[TEMIB] "MPLS Traffic Engineering Management Information Base Using
SMIv2", Srinivasan, Viswanathan,Nadeau, draft-ietf-mpls-te-mib-
09.txt, November 2002
Rosen [Page 23]
Internet Draft draft-ietf-l3vpn-as2547-01.txt July 2003
18. Author's Address
Eric C. Rosen
Cisco Systems, Inc.
1414 Massachusetts Avenue
Boxborough, MA 01719
E-mail: erosen@cisco.com
Rosen [Page 24]
