[Search] [pdf|bibtex] [Tracker] [WG] [Email] [Nits]

Versions: 00                                                            
Internet Engineering Task Force                 Dave Cooper, Global Crossing
INTERNET-DRAFT                                  Blaine Christian, UUNET
TEWG Working Group                              Daniel Awduche
Expiration: August 2001                                Movaz Communications
                                                Vijay Gill
                                                       Metromedia Fiber
                                                Alan Hannan, RoutingLoop
                                                Jim Boyle, Level 3

                                                February 2001

     Applicability Statement for Traffic Engineering with MPLS


Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet- Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at

   The list of Internet-Draft Shadow Directories can be accessed at


      This memo describes the applicability of Multiprotocol Label
      Switching (MPLS) to traffic engineering in IP networks. Special
      considerations for deployment of MPLS for traffic engineering in
      operational contexts are discussed and the limitations of the
      MPLS approach to traffic engineering are highlighted.

      This document is intended for the Internet informational track.

Cooper et. al.                                                          [Page 1]

Internet Draft  draft-many-tewg-te-applicability-00.txt           February 2001

1.  Introduction

    It is generally acknowledged that one of the most significant
    initial applications of Multiprotocol Label Switching (MPLS) is
    traffic engineering [1][2] in IP networks.  A significant
    community of IP service providers have found that traffic
    engineering of their traffic can have tactical and strategic value
    [2][3][4].  To support traffic engineering application, extensions
    have been specified for IS-IS and OSPF [5][6], and to signaling
    protocols RSVP and LDP [7][8].  The extensions for ISIS, OSPF and
    RSVP have all been developed and deployed in scale, and in a
    multi-vendor environment.

    This memo discusses the applicability of Traffic Engineering to
    Internet Service Provider networks.  It augments protocol and
    operational statements on applicability and concern made in
    relation to the use of RSVP-TE in particular [9].  Special
    considerations for deployment of MPLS in operational contexts are
    discussed and the limitations of this approach to traffic
    engineering are highlighted.

2.  Technical Overview of ISP traffic engineering

    Traffic Engineering in this context consists of mapping IP traffic
    flows onto the existing physical topology. Techniques used to
    accomplish this include but are not limited to:

     (1) IGP Cost (Metrics)

     (2) Explicit Routing using constrained virtual circuit switching
         techniques such as ATM or Frame Relay SPVCs.

     (3) Explicit Routing using constrained path setup techniques such
         as MPLS

    This document primarily concerns itself with MPLS techniques.
    Specifically, dealing with the ability to move traffic away from
    the shortest paths selected by the IGP and onto other paths to
    avoid congestion in the network. This is achieved by the use of
    explicitly signalled LSP-tunnels.  The explicit route to be used
    may be computed offline and configured down into the router, or
    characteristics of the LSP may be configured into the router,
    which will then use SPF techniques subject to the constraints of
    the characteristics.  These characteristics can include:

      o) Bandwidth required
      o) Priority
      o) Nodes to include or exclude in the path
      o) Affinities to include or exclude in the path

    Affinities are arbitrary, provider assigned attributes applied to
    links and carried in the Traffic Engineering extensions for the

Cooper et. al.                                                          [Page 2]

Internet Draft  draft-many-tewg-te-applicability-00.txt           February 2001

    IGPs.  They can be used to affect certain policies, or route
    preferences.  They are unique to MPLS.

3.  Applicability of Internet Traffic Engineering

    As mentioned in [2] and [7], traffic engineering with MPLS is
    appropriate to maintain explicitly routed paths in an IP network
    for traffic placement. LSP-tunnels can be used to forward subsets
    of traffic through paths that are independent of routes computed
    by conventional Interior Gateway Protocol (IGP) Shortest Path
    First (SPF) algorithms. This allows Network Operators significant
    flexibility in directing traffic flows and allows policies to be
    implemented that can result in the performance optimization of
    networks.  Examples of where use of MPLS for traffic engineering
    in ISP environments is applicable are given below.  The uses are
    certainly not restricted to these.

3.1 Avoidance of congested resources

    In order to lower the utilization on congested link(s), an
    operator may have a variety of approaches to routing a subset of
    traffic on that link onto less congested topology elements.

    Operators who do not make extensive use of LSP tunnels might
    create an LSP between two points known to be a source of the
    congestion and explicitly route this traffic along another path.
    Operators who do make extensive use of LSP tunnels, either for
    measurement or automated traffic control, might also choose to
    explicitly route a subset of the LSPs onto other paths.  This can
    be employed as a quick response when the bandwidth signalled by
    the LSPs does not accurately match the actual traffic on the LSPs,
    yet the provider determines that changing the bandwidth parameters
    is not time effective or of lasting impact.

    Approaches that entail measurement of traffic loads on LSPs and
    using these to configure explicit routes for LSPs (also known as
    offline traffic engineering [10]) or solely the bandwidths and
    other parameters such as bandwidth, priority and/or affinities
    (online traffic engineering) can be used strategically to react to
    periods of congestion in a network.  This congestion may occur as
    a result of equipment or facility failure, slower than expected
    provisioning cycles for new circuits, unexpected surges in traffic
    levels and even to effectively utilize a more geographically
    meshed topology due to size of available routing or switching
    equipment relative to demand.

3.2 Utilization of parallelized topologies

    Transoceanic connectivity often comes in numerous smaller
    circuits, such as NxDS3, NxSTM-1.  This can also be the case into
    bandwidth constrained cities for continental networks.
    Effectively distributing the load across these circuits can be
    attempted by employing MPLS (and often ATM) traffic engineering.

Cooper et. al.                                                          [Page 3]

Internet Draft  draft-many-tewg-te-applicability-00.txt           February 2001

    Approaches include solely using bandwidth signalling, explicitly
    configuring routes for LSP tunnels, and assigning affinities to
    parallel elements, and tying LSP tunnels to different affinities.
    A similar problem and solution approach involves networks which
    have a replicated topology, such as an NxOC3/12/48 topology.

3.3 Affecting policies for use of certain circuits

    It is sometimes desirable to get certain traffic on or off certain
    links in the topology for policy or engineering reasons.

    Suppose a global service provider has a flat IGP.  MPLS TE
    functionality, in the form of affinities, can be used to keep
    continental traffic from traversing trans-oceanic resources.

    Another example of using MPLS TE affinities to exclude certain
    traffic from a subset of circuits might be to keep inter-regional
    LSPs off of circuits that are reserved for intra-regional traffic.

4.  Implementation Considerations

4.1 Architectural and Operational

    When deploying traffic engineering in a service provider
    environment, the network impact of administrative policies and the
    selection of nodes that will serve as endpoints for LSP-tunnels
    should be carefully thought through and understood.  Furthermore,
    when devising a virtual topology for LSP-tunnels, special
    consideration should be given to the tradeoff between the
    operational complexity associated with a large number of
    LSP-tunnels and the control granularity that large numbers of
    LSP-tunnels allow.  Stated otherwise, a large number of
    LSP-tunnels allow greater control over the distribution of traffic
    across the network, but increases network operational
    complexity. In large networks, it may be advisable to start with a
    simple LSP-tunnel virtual topology and then introduce additional
    complexity based on observed or anticipated traffic flow patterns.

    Administrative policies should guide the amount of bandwidth to be
    allocated.  One may choose to set the bandwidth of a particular
    LSP to the measured peak, or a particular Percentile or Quantile
    of observed utilization over a determined period.  Sufficient over
    subscription (of LSPs) or under reporting bandwidth on actual
    links should be used to account for flows that exceed their normal
    limits on an event driven basis. Flows should be monitored for
    trends that indicate when a LSP should be resized for additional
    bandwidth required. Flows should be monitored dynamically for
    extreme variance from expected levels.  Should an unpoliced flow
    greatly exceed its subscription factor, efforts should be made to
    determine the cause and remedy the problem.  Policing flows is an
    effective method for containing congestion to flows that are in
    need of remediation.

    When creating LSPs, a hierarchical network approach can be used to

Cooper et. al.                                                          [Page 4]

Internet Draft  draft-many-tewg-te-applicability-00.txt           February 2001

    alleviate problems associated with full meshes.  In general,
    operational experience has shown that large macro flows (city
    pair) are long-lived and tractable, while smaller flows (edge to
    edge) are not. Using a TE Core to aggregate the edge of the
    network maps nicely into this hierarchy.

    Over aggregation of flows can result in a flow so large that
    precludes the constraint based routing entity from finding a path
    through a network.  Splitting the flow by using two or more
    parallel LSPs to map the flow onto can solve this problem at the
    expense of introducing more state.

    Failure scenarios should also be addressed when creating a flow
    over several links. It is of little value to establish a finely
    balanced set of flows over a set of links only to find that upon
    link failure the balance reacts poorly, or does not revert upon
    circuit restoration.

4.1 Network Management Aspects

    Networks planning to deploy MPLS for Traffic Engineering must
    consider the performance and fault management aspects of Network
    Management.  With the addition of the MPLS component to any
    infrastructure, complex operations are added which require
    constant monitoring to ensure they do not impede the performance
    of end-to-end traffic delivery.  In addition, traffic
    characteristics, such as latency across an LSP, must be assessed
    on a regular basis to ensure service-level guarantees are

    Attaining information on LSP behavior is critical in determining
    the volatility of a MPLS network.  When LSPs transition or path
    changes occur, packets may be dropped which results in a
    significant impact on network performance.  It should be the goal
    of any network deploying MPLS to minimize the volatility of LSPs
    and reduce the root causes that induce this volatility.
    Unfortunately, there are very few, if any, NMS systems deployed
    capable of providing such root cause analysis and must be
    accomplished using in-house code development.  This creates a
    significant constraint in deploying wide-scale MPLS networks.

    Performance of an MPLS network is also highly dependent on the
    configured values of bandwidth for each LSP.  Since congestion is
    a common cause of performance degradation, it is important to
    proactively avoid these situations. While MPLS was designed to
    minimize congestion on links by utilizing bandwidth reservations,
    it is still heavily reliant on user configurable data. If the LSP
    bandwidth value does not adequately represent the traffic flows of
    that LSP, over utilization may occur and cause significant
    congestion within a network.  Therefore it is imperative to build
    and maintain an adequate statistical modeling tool for determining
    LSP bandwidth size.  Failure to do so may result in sub optimal
    network performance.

Cooper et. al.                                                          [Page 5]

Internet Draft  draft-many-tewg-te-applicability-00.txt           February 2001

5.  Limitations

    Sufficiently understanding how this technology works and
    identifying problems with implementations can take significant
    engineering and testing resources.  Initial deployment of code and
    configurations to support traffic engineering can consume
    significant engineering, operation and system development
    resources.  Developing automated systems to create configurations
    for network elements can require significant software development
    and hardware resources.  Getting to a point where configurations
    for routers are updated in an automated fashion can be a time
    consuming process.  Tracking manual tweaks to router
    configurations, or problems associated with these can be
    an endless task.

    Certain architectural choices can lead to operational, protocol
    and router implementation scalability problems.  This is
    especially the case as the number of LSP tunnels or configuration
    data in a network increases, which can be exasperated by designs
    incorporating meshes, which create N^2 LSPs.  In these cases
    minimizing N through hierarchy, regionalization, or proper
    selection of tunnel termination points can affect the network's
    ability to scale.  Loss of scale in this sense can be via protocol
    instability, inability to change network configurations to
    accommodate growth, inability for router implementations to be
    updated, hold or properly process configurations or loss of
    ability to adequately manage the network.

    Although widely deployed, this is new technology when compared to
    standard fares such as ISIS, OSPF and BGP.  It also has more
    configuration and protocol options.  As such, implementations are
    less battle-hardened and automated testing of various
    configurations are not as thorough.  Multivendor environments are
    achievable, though not without additional reconciliation effort.

    Common approaches of traffic engineering in service provider
    environments switch the forwarding paradigm from connectionless to
    connection oriented.  Thus operational analysis of the network may
    be complicated in some regards (and improved in others).
    Inconsistencies in forwarding state result in dropped packets
    whereas with connectionless the packet will either loop and drop,
    or be misdirected onto another branch in the routing tree.

    Currently deployed MPLS TE approaches can be adversely affected by
    both internal and external router and link failures.  This can
    create a mismatch between the signalled capacity and the traffic
    an LSP tunnel carries.

    Many routers in service provider environments are already under
    stressful processing and operational loads just running IGP, BGP
    and IPC.  Enabling traffic engineering in an MPLS environment
    involves adding traffic engineering databases and processes,
    adding additional information to be carried by the routing

Cooper et. al.                                                          [Page 6]

Internet Draft  draft-many-tewg-te-applicability-00.txt           February 2001

    processes, and adding signalling state and processing to these
    network elements.  In some environments, this additional load may
    not be feasible.

    MPLS in general and MPLS-TE in particular is not a panacea for
    lack of network capacity, or lack of proper capacity planning and
    provisioning. MPLS-TE generally causes network traffic to traverse
    more network elements as well as incur greater latency. Generally
    this added inefficiency is done to prevent shortcomings in
    capacity planning execution or available resources path to avoid
    hot spots.  The ability of traffic engineering to accommodate more
    traffic on a given topology can also be characterized as a short
    term gain during periods of persistent traffic growth.  Failure to
    properly capacity plan and execute will lead to congestion, no
    matter what technological aides are employed.

6.  Conclusion

    The applicability of traffic engineering in Internet Service
    Provider environments has been discussed in this document. The
    focus has been on use of MPLS based approaches to achieve traffic
    engineering in this context.  The applicability of Traffic
    Engineering and associated management and deployment
    considerations are detailed, as well as limitations noted.

    The capability to monitor point to point traffic statistics
    between two routers and the capability to control the forwarding
    paths of subsets of traffic through a given network topology
    together make traffic engineering with MPLS applicable and useful
    for traffic engineering within service provider networks.

7.  Security Considerations

    This document does not introduce new security issues. When
    deployed in service provider networks, it is mandatory to ensure
    that only authorized entities are permitted to initiate
    establishment of LSP-tunnels.

8.  Acknowledgments

    The effectiveness of the MPLS protocols for traffic engineering in
    service provider networks is in large part due to the experience
    gained and foresight given by network engineers and developers
    familiar with traffic engineering with ATM in these environments.
    In particular, the authors wish to acknowledge the authors of RFC
    2702 for the clear articulation of the requirements, as well as
    the developers and testers of code in deployment today for keeping
    their focus.

9.  References

   [1] E. Rosen, A. Viswanathan, R. Callon, "A Proposed
   Architecture for MPLS", RFC3031

Cooper et. al.                                                          [Page 7]

Internet Draft  draft-many-tewg-te-applicability-00.txt           February 2001

   [2] D. Awduche, J. Malcolm, J. Agogbua, M. O'Dell, J. McManus,
   "Requirements for Traffic Engineering Over MPLS," RFC 2702,
   September 1999

   [3] X. Xiao, A. Hannan, B. Bailey, L. Ni, "Traffic Engineering with
   MPLS in the Internet", IEEE Transactions on Networking, March/April

   [4] V. Springer, C. Pierantozzi, J. Boyle, "Level3 MPLS Protocol
   Architecture", work in progress, August 2000.

   [5] T. Li, H. Smit "IS-IS extensions for Traffic Engineering", work
   in progress, September 2000.

   [6] D. Katz, D. Yeung "Traffic Engineering extensions to OSPF",
   work in progress, September 2000.

   [7] D. Awduche et. al. "RSVP-TE: Extensions to RSVP for LSP
   Tunnels", work in progress, August 2000.

   [8] B. Jamousi, "Constraint-Based LSP Setup using LDP", work in
   progress, July 2000.

   [9] D. Awduche, A. Hannan, X. Xiao "Applicability Statement for
   Extensions to RSVP for LSP-Tunnels", work in progress, April 2000.

   [10] D. Awduche, A. Chiu, A. Elwalid, I. Widjaja, X. Xiao, "A
   Framework for Internet Traffic Engineering", work in progress, July


   Dave Cooper                          Blaine Christian
   Global Crossing                      MCI/WorldCom/UUNET Campus
   960 Hamlin Court                     22001 Loudoun County Parkway
   Sunnyvale, CA 94089                  Room D1-2-737
   (916) 415-0437                       Ashburn, VA 20147
   email: dcooper@gblx.net              email: blaine@uu.net

   Vijay Gill                           Daniel Awduche
   Metromedia Fiber Network             Movaz Networks
   8075 Leesburg Pike                   7926 Jones Branch Drive, Suite 615
   Viena, VA 22182                      McLean, VA 22102
   (410) 262-0660                       (703) 847-7250
   email: vgill@mfnx.net                email: awduche@movaz.com

   Alan Hannan                          Jim Boyle
   RoutingLoop Intergalactic            Level3 Communications
   112 Falkirk Court                    1025 Eldorado Boulevard
   Sunnyvale, CA 94087                  Broomfield, CO 80021
   (408) 666-2326                       (720) 888-1192
   email: alan@routingloop.net          email: jboyle@level3.net

[Page 8]