Network Working Group Rahul Aggarwal (Juniper Networks)
Internet Draft Yakov Rekhter (Juniper Networks)
Expiration Date: December 2008
Intended Status: Informational
PIM/GRE Based MVPN Deployment Experience and Recommendations
draft-rekhter-mboned-mvpn-deploy-00.txt
Status of this Memo
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
IPR Disclosure Acknowledgement
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Abstract
Multicast VPN, based on the Virtual Router model using PIM with GRE
tunnels, has been in operation in production networks for several
years. This document describes some of the experiences gained from
implementation and deployment of such Multicast VPNs. Further it
describes the practices used by such deployments based on the
experience.
Aggarwal, Rekhter [Page 1]
Internet Draft draft-rekhter-mboned-mvpn-deploy-00.txt June 2008
Specification of Requirements
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
1. Introduction
The first proposal for multicast support for BGP/MPLS IP VPNs
[RFC4364], informally known as "draft-rosen", was presented in San
Diego IETF, 2000. At that time multicast support was not in the L3VPN
WG charter. Since 2000 draft-rosen went through several revisions,
with the last revision informally known as draft-rosen-08.
At San Diego IETF in 2004 multicast support has been added to the
L3VPN WG charter. At the present moment L3VPN WG has several working
group documents that specify mechanisms for multicast support in
BGP/MPLS IP VPNs ([MVPN-ARCH], [BGP-MPLS-MVPN]).
At the present moment multi-vendor interoperable implementations are
known to exist only based on a particular version of draft-rosen,
informally known as draft-rosen-06, but not based on the later
version of draft-rosen - draft-rosen-08.
While the mechanisms described in draft-rosen-06 form a subset of the
mechanisms described in [MVPN-ARCH], the additional mechanisms
introduced in draft-rosen-08 are not part of the mechanisms described
in [MVPN-ARCH]. Specifically the mechanisms to support PIM-SSM for
Inclusive P-Tunnels and inter-AS operations option (B) specified in
draft-rosen-08 are not part of the mechanisms described in [MVPN-
ARCH]. The mechanisms to support PIM-SSM for Inclusive P-Tunnels and
inter-AS operations option (B) specified in [MVPN-ARCH] are not
interoperable with the corresponding mechanisms in draft-rosen-08.
While unicast BGP/MPLS IP VPNs solution is based on the aggregated
routing architecture [RFC4110], the approach taken by draft-rosen is
based on a completely different architecture, namely Virtual Routers
[RFC4110].
The differences between unicast BGP/MPLS IP VPNs and draft-rosen are
not only in the architecture, but also in the mechanisms:
+ While unicast BGP/MPLS IP VPNs use BGP to exchange (unicast)
VPN's routing information among PEs, draft-rosen uses PIM to
exchange (multicast) VPN's routing information among PEs.
Aggarwal, Rekhter [Page 2]
Internet Draft draft-rekhter-mboned-mvpn-deploy-00.txt June 2008
+ While unicast BGP/MPLS IP VPNs use MPLS in the data plane (with
GRE as an option for inter-PE tunnels), draft-rosen restricts the
data plane to just GRE or IP-in-IP tunnels. (While the draft
mentions MPLS as a possible encapsulation, the draft specifies no
details on how to support it). Moreover, in terms of
implementations multi-vendor implementations exist only for GRE,
but not for IP-in-IP.
+ While unicast BGP/MPLS IP VPNs use LDP or RSVP-TE to set up
inter-PE MPLS tunnels, draft-rosen uses PIM for setting up (GRE-
based) inter-PE tunnels.
+ While the control plane used by unicast BGP/MPLS IP VPNs is
decoupled from its data plane, the control plane and the data
plane of draft-rosen are coupled in a sense that exactly the same
inter-PE tunnels are used to exchange both control and data.
As a result, deploying draft-rosen requires a set of control plane
and data plane mechanisms among the PEs that are completely different
from what is required by (unicast) BGP/MPLS IP VPNs.
2. Control Plane Scalability Considerations
Use of the Virtual Router architecture by draft-rosen means that a
given VRF on a given PE has to maintain PIM adjacencies with every
other VRF belonging to that MVPN on every other PE. Moreover, these
adjacencies have to be maintained not on a per PE, but on a per MVPN,
per PE granularity. A PE also has to maintain PIM adjacencies with
all the locally connected CEs (and these adjacencies are not due to
the use of the Virtual Router architecture). However, as long as the
average number of a given MVPN sites connected to a given PE is less
than the average number of PEs that have sites of that MVPN, then on
a given PE the overhead of maintaining PIM adjacencies with other PEs
will dominate the overhead of PIM adjacencies between that PE and its
locally connected CEs. Note that this overhead grows with the number
of PEs in an MVPN, as well as with the number of MVPNs.
To illustrate the overhead consider an example where a PE has 1,000
VRFs, and each of these VRFs corresponds to an MVPN that on average
is present on 100 PEs. The average number of PIM adjacencies that the
PE would need to maintain with other PEs is 100,000. The average rate
of PIM Hellos that the PE would need to process is 3,300 per second.
Aggarwal, Rekhter [Page 3]
Internet Draft draft-rekhter-mboned-mvpn-deploy-00.txt June 2008
3. Join and Prune Latency
One consequence of using unreliable transport for PE-PE MVPN
multicast routing infromation exchange with PIM is that if the first
PIM Join sent by a PE to another PE is lost, then this results in
Join latency of at least upto the duration of the PIM Join
retransmission. This duration is usually at least 30 seconds. Losing
subsequent PIM Joins may further increase this Join latency.
Similarly if the last PIM Prune sent by a PE to another PE is lost,
then this results in the receiver PEs receiving unwanted data until
the corresponding multicast state on the sender/upstream PE times
out, which, as estimated in [PORT], is roughly 3 minutes. That has
negative implications on the efficient use of bandwidth within the
Service Provider(s).
Moreover, if the last PIM Prune sent by a PE to another (upstream) PE
is lost, then this results in the upstream PE maintaining the
corresponding multicast state until that state times out, which as
estimated in [PORT] is roughly 3 minutes.
Additional issue of increased Join latency when switching to S-PMSIs
is described in the next section.
4. Possible packet losses when switching to S-PMSI
Signaling switching to S-PMSI is done by periodically (every 60 secs)
sending a UDP message that contains the identity of the provider
multicast tree used by an S-PMSI as well as the customer multicast
stream bound to that S-PMSI.
One consequence of using unreliable transport (UDP) for signaling
switching to S-PMSI is that if the first S-PMSI advertisement is
lost, then this results in losses of multicast data for up to 57 secs
(MDT-INTERVAL - MDT-DATA-DELAY).
Moreover, if all the PEs in a given MVPN do not always store all the
S-PMSI advertisements for that MVPN, irrespective of whether these
PEs have downstream receivers for the multicast traffic carried by
these S-PMSIs, then it may result in multicast data losses (Join
latency) on average of 30 secs and up to 60 secs (MDT-INTERVAL timer)
on the PEs. This is in the absence of any S-PMSI advertisement
losses. Losing an S-PMSI advertisement may further increase the
duration of the losses (increases this Join latency).
Aggarwal, Rekhter [Page 4]
Internet Draft draft-rekhter-mboned-mvpn-deploy-00.txt June 2008
5. Limitations when Handling Anycast Customer RP (C-RP)
Large enterprises may have multiple data centers where Anycast RP's
and sources of multicast traffic are located. Such MVPN customers
might require multicast applications to be simultaneously sourced
from each data center and delivered via corresponding Anycast RP's to
different sets of branch locations. Each data center and each branch
location may be in its own MVPN site.
When an MVPN uses anycast RP, where several customer's RPs (C-RPs)
share the same anycast address and are reachable via more than one
PE, then with the mechanisms provided by draft-rosen for a given
customer's multicast group (C-G) at any given point in time only one
of these C-RPs can be used to deliver (multicast) traffic to
receivers in other sites of that MVPN. That eliminates one of the
main benefits of anycast RP - the ability to share load among
multiple RPs.
6. Limitations when Handling Multi-homed Multicast Sources
When an MVPN site contains a given multicast source, and the site is
connected to two or more PEs, then at any given point in time only
one of these PEs could be used to forward multicast traffic from the
source to the receivers in other sites of that MVPN. This is in
contrast to unicast, where unicast traffic could be forwarded to the
source via all of these PEs.
7. Mandatory I-PMSI
Mechanisms used by draft-rosen mandate that each MVPN must have its
own I-PMSI. This is even if most/all of the multicast data is sent
using S-PMSIs. The overhead of maintaining I-PMSI, both in the
control and in the data plane, is especially significant in the
inter-AS scenario, as in this scenario the number of point-to-
multipoint tunnels required for a single I-PMSI is equal to the
number of PEs that span that I-PMSI.
Aggarwal, Rekhter [Page 5]
Internet Draft draft-rekhter-mboned-mvpn-deploy-00.txt June 2008
8. QoS
If a service provider uses DiffServ based QoS, the service provider
needs two different mechanisms for providing such QoS for its VPN
customers - one for unicast BGP/MPLS VPNs using MPLS and another for
multicast VPNs using draft-rosen. This is because the former uses
MPLS based DiffServ mechanisms such as mapping the MPLS EXP bits to
forwarding classes while the latter uses IP based DiffServ mechanisms
such as mapping the DSCP bits to forwarding classes.
Moreover, certain QoS mechanisms that are available for (unicast)
BGP/MPLS IP VPNs are not available for draft-rosen. Specifically,
MPLS Traffic Engineering and MPLS DiffServ Traffic Engineering that
are available for unicast VPN traffic are not available for draft-
rosen.
9. Protection/Restoration
Certain protection/restoration mechanisms that are available for
(unicast) BGP/MPLS IP VPNs are not available for draft-rosen.
Specifically, none of the MPLS Fast Re-route mechanisms that could be
used in conjunction with unicast VPN traffic are available for draft-
rosen.
10. Security Considerations
The Security Considerations section of [RFC4797] discusses security
issues in the context of unicast BGP/MPLS IP VPNs when PE-PE tunnels
are realized via GRE. These considerations are applicable in the
context of [draft-rosen] as well. However, draft-rosen presents
additional security considerations, above and beyond of what is
covered in [RFC4797]. The following describes some of them.
Inability to restrict joining an I-PMSI (Default MDT) of a given MVPN
to only the PEs that have VRFs of that MVPN may result in (a) leaking
multicast traffic originated within that MVPN to the receivers that
are not suppose to be part of that MVPN, and (b) various forms of
packet spoofing, as described below. Once an attacker joins an I-PMSI
of a given MVPN, the attacker, in addition to the ability to receive
multicast traffic originated within that MVPN, can also create
attacks based on packet spoofing.
The implications of packet spoofing in the context of draft-rosen are
more significant than in the context of [RFC4797], as in the context
of [RFC4797] spoofing can impact only the data traffic, while in the
context of draft-rosen spoofing can impact not only the data, but
Aggarwal, Rekhter [Page 6]
Internet Draft draft-rekhter-mboned-mvpn-deploy-00.txt June 2008
also the control traffic associated with the exchange of MVPN routing
information among PEs. This is because in the context of draft-rosen
the same GRE tunnels that are used to exchange MVPN data traffic
among PEs are also used to exchange MVPN multicast routing
information among PEs.
Securing control traffic associated with the exchange of MVPN routing
information among PEs by applying security mechanisms specified in
[RFC4601] to PIM that is used to exchange MVPN routing information
among PEs is problematic for the following reasons.
One option specified in [RFC4601] states that "A PIM router SHOULD
provide an option to limit the set of neighbors from which it will
accept Join/Prune, Assert, and Hello messages. Either static
configuration of IP addresses or an IPsec security association may be
used.". Use of the first option, static configuration, in the context
of draft-rosen to authenticate PIM messages used to exchange MVPN
routing information among PEs may be problematic, as it would require
a given PE for each MVPN present on the PE to have an apriori
knowledge of all the other PEs with whom the PE has at least one MVPN
in common. That gets especially problematic in the inter-provider
scenario where PEs in different providers may need to become PIM
neighbors, as it would require a provider to share this information
with other providers.
Another option specified in [RFC4601] is to use IPsec. As stated in
[RFC4601] "To use IPsec, the administrator of a PIM network
configures each PIM router with one or more security associations
(SAs) and associated Security Parameter Indexes (SPIs) that are used
by senders to authenticate PIM protocol messages and are used by
receivers to authenticate received PIM protocol messages." However,
neither [RFC4601], nor draft-rosen provide an automated mechanism for
establishing such security associations. In fact [RFC4601] "assumes
that manual configuration of SAs is performed". Use of this option
in the context of draft-rosen to authenticate PIM messages used to
exchange MVPN routing information among PEs is problematic, as it
involves manual configuration of SAs on PEs that have at least one
MVPN in common. It is especially problematic in the inter-provider
scenario where it may involve PEs in different providers.
At the CE-PE interfaces each PE must install packet filters to filter
out any UDP traffic on port 3232 that the PE may receive from any of
its attached CEs, as UDP port 3232 is used for the S-PMSI messages
exchanged among PEs. Failure to filter out such traffic can cause the
creation of extra multicast states on the Service Providers routers.
Additional security considerations that are applicable in the context
Aggarwal, Rekhter [Page 7]
Internet Draft draft-rekhter-mboned-mvpn-deploy-00.txt June 2008
of draft-rosen are described in [RFC4023].
11. IANA Considerations
This document does not require any action on the part of IANA.
12. Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at ietf-
ipr@ietf.org.
13. Copyright Notice
Copyright (C) The IETF Trust (2008).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Aggarwal, Rekhter [Page 8]
Internet Draft draft-rekhter-mboned-mvpn-deploy-00.txt June 2008
14. Disclaimer of validity:
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79. Copies of IPR disclosures made to the
IETF Secretariat and any assurances of licenses to be made available,
or the result of an attempt made to obtain a general license or
permission for the use of such proprietary rights by implementers or
users of this specification can be obtained from the IETF on-line IPR
repository at http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at ietf-
ipr@ietf.org.
15. Acknowledgements
We would like to thank Maria Napierala for pointing to the problems
with draft-rosen of supporting MVPN customers who use Anycast RP, and
MVPN customers who have (multicast) sources in multihomed sites.
Many thanks to Leonard Giuliano for his review and comments.
16. Normative References
[RFC2119] "Key words for use in RFCs to Indicate Requirement
Levels.", Bradner, RFC2119, March 1997.
Aggarwal, Rekhter [Page 9]
Internet Draft draft-rekhter-mboned-mvpn-deploy-00.txt June 2008
17. Non-normative References
[BGP-MPLS-MVPN] Aggarwal, R., Rosen, E., Morin, T., Rekhter, Y., "BGP
Encodings and Procedures for Multicast in MPLS/BGP IP VPNs", draft-
ietf-l3vpn-2547bis-mcast-bgp-04.txt
[MVPN-ARCH] Rosen, E., Aggarwal, R., "Multicast in MPLS/BGP IP VPNs"
draft-ietf-l3vpn-2547bis-mcast-06.txt
[PORT] Farinacci, D., et al., "A Reliable Transport Mechanism for
PIM", draft-farinacci-pim-port-01.txt
[RFC4023] Worster, T., Rekhter, Y., and E. Rosen, "Encapsulating MPLS
in IP or Generic Routing Encapsulation (GRE)", RFC 4023, March 2005.
[RFC4110] Callon, R., Suzuki, M., "A Framework for Layer 3 Provider-
Provisioned Virtual Private Networks (PPVPNs)", RFC4110, July 2005
[RFC4364] Rosen, E., Rekhter, Y., "BGP/MPLS IP Virtual Private
Networks (VPNs)", RFC4364, February 2006
[RFC4601], Fenner. B., et. al, "Protocol Independent Multicast -
Sparse Mode (PIM-SM): Protocol Specification (Revised)", RFC4601,
August 2006
[RFC4797] Rekhter, Y., Bonica, R., Rosen, E., "Protocol Independent
Multicast - Sparse Mode (PIM-SM): Protocol Specification (Revised)",
RFC4797, January 2007
18. Author Information
Rahul Aggarwal
Juniper Networks, Inc.
e-mail: rahul@juniper.net
Yakov Rekhter
Juniper Networks, Inc.
e-mail: yakov@juniper.net
Aggarwal, Rekhter [Page 10]