MPLS                                                  C. Villamizar, Ed.
Internet-Draft                                    Outer Cape Cod Network
Intended status: Informational                                Consulting
Expires: May 16, 2013                                  November 12, 2012

                 Use of Multipath with MPLS-TP and MPLS


   Many MPLS implementations have supported multipath techniques and
   many MPLS deployments have used multipath techniques, particularly in
   very high bandwidth applications, such as provider IP/MPLS core
   networks.  MPLS-TP has strongly discouraged the use of multipath
   techniques.  Some degradation of MPLS-TP OAM performance cannot be
   avoided when operating over many types of multipath implementations.

   Using MPLS Entropy label, MPLS can LSP can be carried over multipath
   links while also providing a fully MPLS-TP compliant server layer for
   MPLS-TP LSP.  This document describes the means of supporting MPLS as
   a server layer for MPLS-TP.  The use of MPLS-TP LSP as a server layer
   for MPLS LSP is also discussed.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on May 16, 2013.

Copyright Notice

   Copyright (c) 2012 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents

Villamizar                Expires May 16, 2013                  [Page 1]

Internet-Draft         MPLS-TP and MPLS Multipath          November 2012

   ( in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . . 3
   2.  Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 3
   3.  MPLS as a Server Layer for MPLS-TP  . . . . . . . . . . . . . . 5
   4.  MPLS-TP as a Server Layer for MPLS  . . . . . . . . . . . . . . 7
   5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 8
   6.  Security Considerations . . . . . . . . . . . . . . . . . . . . 8
   7.  References  . . . . . . . . . . . . . . . . . . . . . . . . . . 8
     7.1.  Normative References  . . . . . . . . . . . . . . . . . . . 8
     7.2.  Informative References  . . . . . . . . . . . . . . . . . . 8
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . . . 9

Villamizar                Expires May 16, 2013                  [Page 2]

Internet-Draft         MPLS-TP and MPLS Multipath          November 2012

1.  Introduction

   Today the requirement to handle large aggregations of traffic, can be
   handled by a number of techniques which we will collectively call
   multipath.  Multipath applied to parallel links between the same set
   of nodes includes Ethernet Link Aggregation [IEEE-802.1AX], link
   bundling [RFC4201], or other aggregation techniques some of which may
   be vendor specific.  Multipath applied to diverse paths rather than
   parallel links includes Equal Cost MultiPath (ECMP) as applied to
   OSPF, ISIS, or BGP, and equal cost LSP.  Some vendors support load
   split across equal cost MPLS LSP where the load is split
   proportionally to the reserved bandwidth of the set of LSP.

   RFC 5654 requirement 33 requires the capability to carry a client
   MPLS-TP or MPLS layer over a server MPLS-TP or MPLS layer [RFC5654].
   This is possible in all cases with one exception.  When an MPLS LSP
   exceeds the capacity of any single component link it may be carried
   by a network using multipath techniques, but may not be carried by an
   MPLS-TP LSP due to the inherent MPLS-TP capacity limitation imposed
   by MPLS-TP OAM packet ordering constraints.

   The term composite link is more general than terms such as link
   aggregation (which is specific to Ethernet) or ECMP (which implies
   equal cost paths within a routing protocol).  The use of the term
   composite link here is consistent with the broad definition in
   [ITU-T.G.800].  Multipath is very similar to composite link as
   defined by ITU, but specifically excludes inverse multiplexing.

2.  Definitions

       The term multipath includes all techniques in which

       1.  Traffic can take more than one path from one node to a

       2.  Individual packets take one path only.  Packets are not
           subdivided and reassembled at the receiving end.

       3.  Packets are not resequenced at the receiving end.

       4.  The paths may be:

           a.  parallel links between two nodes, or

           b.  may be specific paths across a network to a destination
               node, or

Villamizar                Expires May 16, 2013                  [Page 3]

Internet-Draft         MPLS-TP and MPLS Multipath          November 2012

           c.  may be links or paths to an intermediate node used to
               reach a common destination.

   Link Bundle
       Link bundling is a multipath technique specific to MPLS
       [RFC4201].  Link bundling supports two modes of operations.
       Either an LSP can be placed on one component link of a link
       bundle, or an LSP can be load split across all members of the
       bundle.  There is no signaling defined which allows a per LSP
       preference regarding load split, therefore whether to load split
       is generally configured per bundle and applied to all LSP across
       the bundle.

   Link Aggregation
       The term "link aggregation" generally refers to Ethernet Link
       Aggregation [IEEE-802.1AX] as defined by the IEEE.  Ethernet Link
       Aggregation defines a Link Aggregation Control Protocol (LACP)
       which coordinates inclusion of LAG members in the LAG.

   Link Aggregation Group (LAG)
       A group of physical Ethernet interfaces that are treated as a
       logical link when using Ethernet Link Aggregation is referred to
       as a Link Aggregation Group (LAG).

   Equal Cost Multipath (ECMP)
       Equal Cost Multipath (ECMP) is a specific form of multipath in
       which the costs of the links or paths must be equal in a given
       routing protocol.  The load may be split equally across all
       available links (or available paths), or the load may be split
       proportionally to the capacity of each link (or path).

   Loop Free Alternate Paths
       "Loop-free alternate paths" (LFA) are defined in RFC 5714,
       Section 5.2 [RFC5714] as follows.  "Such a path exists when a
       direct neighbor of the router adjacent to the failure has a path
       to the destination that can be guaranteed not to traverse the
       failure."  Further detail can be found in [RFC5286].  LFA as
       defined for IPFRR can be used to load balance by relaxing the
       equal cost criteria of ECMP, though IPFRR defined LFA for use in
       selecting protection paths.  When used with IP, proportional
       split is generally not used.  LFA use in load balancing is
       implemented by some vendors though it may be rare or non-existent
       in deployments.

   Composite Link
       The term Composite Link had been a registered trademark of Avici
       Systems, but was abandoned in 2007.  The term composite link is
       now defined by the ITU in [ITU-T.G.800].  The ITU definition

Villamizar                Expires May 16, 2013                  [Page 4]

Internet-Draft         MPLS-TP and MPLS Multipath          November 2012

       includes multipath as defined here, plus inverse multiplexing
       which is explicitly excluded from the definition of multipath.

   Inverse Multiplexing
       Inverse multiplexing either transmits whole packets and
       resequences the packets at the receiving end or subdivides
       packets and reassembles the packets at the receiving end.
       Inverse multiplexing requires that all packets be handled by a
       common egress packet processing element and is therefore not
       useful for very high bandwidth applications.

   Component Link
       The ITU definition of composite link in [ITU-T.G.800] and the
       IETF definition of link bundling in [RFC4201] both refer to an
       individual link in the composite link or link bundle as a
       component link.  The term component link is applicable to all

   LAG Member
       Ethernet Link Aggregation as defined in [IEEE-802.1AX] refers to
       an individual link in a LAG as a LAG member.  A LAG member is a
       component link.  An Ethernet LAG is a composite link.  IEEE does
       not use the terms composite link or component link.

   load split
       Load split, load balance, or load distribution refers to
       subdividing traffic over a set of component links such that load
       is fairly evenly distributed over the set of component links and
       certain packet ordering requirements are met.  Some existing
       techniques better acheive these objectives than others.

   A small set of requirements are discussed.  These requirements make
   use of keywords such as MUST and SHOULD as described in [RFC2119].

3.  MPLS as a Server Layer for MPLS-TP

   MPLS LSP may be used as a server layer for MPLS-TP LSP as long as all
   MPLS-TP requirements are met, including the requirement that packets
   within an MPLS-TP LSP are not reordered, including both payload and
   OAM packets.

   Supporting MPLS-TP LSP overa fully MPLS-TP conformant MPLS LSP server
   layer where the MPLS LSP are making use of multipath, requires
   special treatment of the MPLS-TP LSP such that those LSP only are not
   subject to the multipath load slitting.  This implies the following
   brief set of requirements.

Villamizar                Expires May 16, 2013                  [Page 5]

Internet-Draft         MPLS-TP and MPLS Multipath          November 2012

   MP#1  It MUST be possible to identify MPLS-TP LSP.

   MP#2  It SHOULD be possible to completely exclude MPLS-TP LSP from
         the multipath hash and load split.

   MP#3  It SHOULD be possible to insure that an MPLS-TP LSP will not be
         moved to another component link as a result of a composite link
         load rebalancing operation.

   MP#4  Where an RSVP-TE control plane is used, it MUST be possible for
         an ingress LSR which is setting up an MPLS-TP or MPLS LSP to
         determine at CSPF time whether a link or MPLS PSC LSP within
         the topology can support the MPLS-TP requirements of the LSP.

   There is currently no signaling mechanism defined to support
   requirement MP#1.  In the absense of a signaling extension, MPLS-TP
   can be identified through some form of configuration, such as
   configuration which provides an MPLS-TP compatible server layer to
   all LSP arriving on a specific interface or originating from a
   specific set of ingress LSR.  Alternately an MPLS-TP LSP can be
   created with and Entropy Label Indicator (ELI) and entropy label (EL)
   below the MPLS-TP label [I-D.ietf-mpls-entropy-label].

   Some hardware which exists today can support requirement MP#2.
   Signaling in the absense of MPLS Entropy Label can make use of link
   bundling with a specific component for MPLS-TP LSP and link bundling
   with the all-zeros component for MPLS LSP.  This prevents MPLS-TP LSP
   from being carried within MPLS LSP but does allow the co-existance of
   MPLS-TP and very large MPLS LSP.

   MPLS-TP LSP can be carried as client LSP within an MPLS server LSP if
   an Entropy Label Indicator (ELI) and entropy label (EL) is added
   after the server layer LSP label(s) in the label stack, just above
   the MPLS-TP LSP label entry [I-D.ietf-mpls-entropy-label].  This
   allows MPLS-TP LSP to be carried as client LSP within MPLS LSP and
   satisfies requirement MP#2 but requires that MPLS LSR be able to
   identify MPLS-TP LSP (requirement MP#1).

   MPLS-TP traffic can be protected from an degraded performance due to
   an imperfect load split if the MPLS-TP traffic is given queuing
   priority (using strict priority and policing or shaping at ingress or
   locally or weighted queuing locally).  This can be accomplished using
   the Traffic Class field and Diffserv treatment of traffic
   [RFC5462][RFC2475].  In the event of congestion due to load
   imbalance, other traffic will suffer as long as there is a minority
   of MPLS-TP traffic.

   If MPLS-TP LSP are carried within MPLS LSP and ELI and EL are used,

Villamizar                Expires May 16, 2013                  [Page 6]

Internet-Draft         MPLS-TP and MPLS Multipath          November 2012

   requirement MP#2 is satisfied, but without a signaling extension,
   requirement MP#3 is not satisfied if there is a need to rebalance the
   load on any composite link carrying the MPLS server LSP.  Load
   rebalance is generally needed only when congestion occurs, therefore
   restricting MPLS-TP to be carried only over MPLS LSP that are known
   to traverse only links which are expected to be uncongested can
   satisfy requirement MP#3.

   Requirement MP#4 can be supported using administrative attributes.
   Administrative attributes are defined in [RFC3209].  Some
   configuration is required to support this.

4.  MPLS-TP as a Server Layer for MPLS

   Carrying MPLS LSP which are larger than a component link over a
   MPLS-TP server layer requires that the large MPLS client layer LSP be
   accommodated by multiple MPLS-TP server layer LSPs.  MPLS multipath
   can be used in the client layer MPLS.

   Creating multiple MPLS-TP server layer LSP places a greater ILM
   scaling burden on the LSR.  High bandwidth MPLS cores with a smaller
   amount of nodes have the greatest tendency to require LSP in excess
   of component links, therefore the reduction in number of nodes
   offsets the impact of increasing the number of server layer LSP in
   parallel.  Today, only in cases where deployed LSR ILM are small
   would this be an issue.

   The most significant disadvantage of MPLS-TP as a Server Layer for
   MPLS is that the use MPLS-TP server layer LSP reduces the efficiency
   of carrying the MPLS client layer.  The service which provides by far
   the largest offered load in provider networks is Internet, for which
   the LSP capacity reservations are predictions of expected load.  Many
   of these MPLS LSP may be smaller than component link capacity.  Using
   MPLS-TP as a server layer results in bin packing problems for these
   smaller LSP.  For those LSP that are larger than component link
   capacity, their capacity are not increments of convenient capacity
   increments such as 10Gb/s.  Using MPLS-TP as an underlying server
   layer greatly reduces the ability of the client layer MPLS LSP to
   share capacity.  For example, when one MPLS LSP is underutilizing its
   predicted capacity, the fixed allocation of MPLS-TP to component
   links may not allow another LSP to exceed its predicted capacity.
   Using MPLS-TP as a server layer may result in less efficient use of
   resources may result in a less cost effective network.

   No additional requirements beyond MPLS-TP as it is now currently
   defined are required to support MPLS-TP as a Server Layer for MPLS.
   It is therefore viable but has some undesirable characteristics

Villamizar                Expires May 16, 2013                  [Page 7]

Internet-Draft         MPLS-TP and MPLS Multipath          November 2012

   discussed above.

5.  IANA Considerations

   This memo includes no request to IANA.

6.  Security Considerations

   This document specifies requirements with discussion of framework for
   solutions using existing MPLS and MPLS-TP mechanisms.  The
   requirements and framework are related to the coexistence of MPLS/
   GMPLS (without MPLS-TP) when used over a packet network, MPLS-TP, and
   multipath.  The combination of MPLS, MPLS-TP, and multipath does not
   introduce any new security threats.  The security considerations for
   MPLS/GMPLS and for MPLS-TP are documented in [RFC5920] and

7.  References

7.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

7.2.  Informative References

              Kompella, K., Drake, J., Amante, S., Henderickx, W., and
              L. Yong, "The Use of Entropy Labels in MPLS Forwarding",
              draft-ietf-mpls-entropy-label-06 (work in progress),
              September 2012.

              Fang, L., Niven-Jenkins, B., Mansfield, S., and R.
              Graveman, "MPLS-TP Security Framework",
              draft-ietf-mpls-tp-security-framework-05 (work in
              progress), October 2012.

              IEEE Standards Association, "IEEE Std 802.1AX-2008 IEEE
              Standard for Local and Metropolitan Area Networks - Link
              Aggregation", 2006, <


Villamizar                Expires May 16, 2013                  [Page 8]

Internet-Draft         MPLS-TP and MPLS Multipath          November 2012

              ITU-T, "Unified functional architecture of transport
              networks", 2007, <

   [RFC2475]  Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
              and W. Weiss, "An Architecture for Differentiated
              Services", RFC 2475, December 1998.

   [RFC3209]  Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
              and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
              Tunnels", RFC 3209, December 2001.

   [RFC4201]  Kompella, K., Rekhter, Y., and L. Berger, "Link Bundling
              in MPLS Traffic Engineering (TE)", RFC 4201, October 2005.

   [RFC5286]  Atlas, A. and A. Zinin, "Basic Specification for IP Fast
              Reroute: Loop-Free Alternates", RFC 5286, September 2008.

   [RFC5462]  Andersson, L. and R. Asati, "Multiprotocol Label Switching
              (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic
              Class" Field", RFC 5462, February 2009.

   [RFC5654]  Niven-Jenkins, B., Brungard, D., Betts, M., Sprecher, N.,
              and S. Ueno, "Requirements of an MPLS Transport Profile",
              RFC 5654, September 2009.

   [RFC5714]  Shand, M. and S. Bryant, "IP Fast Reroute Framework",
              RFC 5714, January 2010.

   [RFC5920]  Fang, L., "Security Framework for MPLS and GMPLS
              Networks", RFC 5920, July 2010.

Author's Address

   Curtis Villamizar (editor)
   Outer Cape Cod Network Consulting


Villamizar                Expires May 16, 2013                  [Page 9]