Network Working Group                                          D. Walton
Internet-Draft                                          Cumulus Networks
Intended status: Standards Track                               A. Retana
Expires: February 9, 2017                                        E. Chen
                                                     Cisco Systems, Inc.
                                                              J. Scudder
                                                        Juniper Networks
                                                          August 8, 2016


               BGP Persistent Route Oscillation Solutions
                draft-ietf-idr-route-oscillation-stop-04

Abstract

   The routing information reduction by BGP Route Reflection or
   Confederation can result in persistent internal BGP route
   oscillations with certain routing setup and network topologies.  This
   document specifies two sets of additional paths that can be used to
   eliminate these route oscillations in a network.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on February 9, 2017.

Copyright Notice

   Copyright (c) 2016 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect



Walton, et al.          Expires February 9, 2017                [Page 1]


Internet-Draft          BGP Oscillation Solutions            August 2016


   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Requirements Language . . . . . . . . . . . . . . . . . . . .   3
   3.  Advertise All the Available Paths . . . . . . . . . . . . . .   3
   4.  Advertise the Group Best Paths  . . . . . . . . . . . . . . .   3
   5.  Route Reflection and Confederation  . . . . . . . . . . . . .   4
     5.1.  Route Reflection  . . . . . . . . . . . . . . . . . . . .   4
     5.2.  Confederation . . . . . . . . . . . . . . . . . . . . . .   5
   6.  Deployment Considerations . . . . . . . . . . . . . . . . . .   5
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   6
   8.  Security Considerations . . . . . . . . . . . . . . . . . . .   6
   9.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   6
   10. References  . . . . . . . . . . . . . . . . . . . . . . . . .   7
     10.1.  Normative References . . . . . . . . . . . . . . . . . .   7
     10.2.  Informative References . . . . . . . . . . . . . . . . .   7
   Appendix A.  Why the Group Best Paths Are Adequate? . . . . . . .   7
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   8

1.  Introduction

   As documented in [RFC3345], the routing information reduction by BGP
   Route Reflection [RFC4456] or BGP Confederation [RFC5065] can result
   in persistent IBGP route oscillations with certain routing setup and
   network topologies.  Except for a couple artificially engineered
   network topologies, the MULTI_EXIT_DISC attribute (MED) [RFC4271] has
   played a pivotal role in virtually all of the known persistent IBGP
   route oscillations.  For the sake of brevity, we use the term "MED-
   induced route oscillation" hereafter to refer to a persistent IBGP
   route oscillation in which the MED plays a role.

   In order to eliminate the MED-induced route oscillations and to
   achieve consistent routing in a network, a route reflector or a
   confederation ASBR needs to advertise more than just the best path
   for an address prefix.  Our goal is to identify the necessary set of
   paths for an address prefix that needs to be advertised by a route
   reflector or a confederation ASBR to prevent the condition.

   In this document we describe two sets of paths for an address prefix
   that can be advertised by a BGP route reflector or confederation ASBR
   to eliminate the MED-induced route oscillations in a network.  The
   first set involves all the available paths, and would achieve the
   same routing consistency as the full IBGP mesh.  The second set,



Walton, et al.          Expires February 9, 2017                [Page 2]


Internet-Draft          BGP Oscillation Solutions            August 2016


   which is a subset of the first one, involves the neighbor-AS based
   Group Best Paths, and would be sufficient to eliminate the MED-
   induced route oscillations (subject to certain commonly adopted
   topological constraints).

   These paths can be advertised using the mechanism described in ADD-
   PATH [RFC7911] for advertising multiple paths.  No other assumptions
   in functionality beyond the base BGP specification [RFC4271] are
   made.

2.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

3.  Advertise All the Available Paths

   Observe that in a network that maintains a full IBGP mesh all the BGP
   speakers have consistent and equivalent routing information.  Such a
   network is thus free of the MED-induced route oscillations and other
   routing inconsistencies such as forwarding loops.

   Therefore one approach is to allow a route reflector or a
   confederation ASBR to advertise all the available paths for an
   address prefix.  Clearly this approach would yield the same amount of
   routing information and achieve the same routing consistency as the
   full IBGP mesh in a network.

   This approach can be implemented using the mechanism described in
   ADD-PATH [RFC7911] for advertising multiple paths for certain
   prefixes.

   For the sake of scalability the advertisement of multiple paths
   should be limited to those prefixes which are affected by MED-induced
   route oscillation in a network carrying a large number of alternate
   paths.  A detailed description of how these oscillations can occur
   can be found in [RFC3345]; the description of how a node would
   locally detect such condition is outside the scope of this document.

4.  Advertise the Group Best Paths

   The term neighbor-AS for a route refers to the neighboring AS from
   which the route was received.  The calculation of the neighbor-AS is
   specified in Section 9.1.2.2 of [RFC4271], and Section 7.2 of
   [RFC5065].  By definition the MED is comparable only among routes
   with the same neighbor-AS.  Thus the route selection procedures
   specified in [RFC4271] would conceptually involve two steps: first



Walton, et al.          Expires February 9, 2017                [Page 3]


Internet-Draft          BGP Oscillation Solutions            August 2016


   organize the paths for an address prefix into groups according to
   their respective neighbor-AS's, and calculate the most preferred one
   (termed "Group Best Path") for each of the groups; Then calculate the
   overall best path among all the Group Best Paths.

   As a generally recommended ([RFC4456], [RFC5065]) and widely adopted
   practice, a route reflection cluster or a confederation sub-AS should
   be designed such that BGP routes from within the cluster (or
   confederation sub-AS) are preferred over routes from other clusters
   (or confederation sub-AS) when the decision is based on the IGP cost
   to the BGP NEXT_HOP.  This is typically done by setting IGP metrics
   for links within a cluster (or confederation sub-AS) to be much
   smaller than the IGP metrics for the links between the clusters (or
   confederation sub-AS).  This practice helps achieve consistent
   routing within a route reflection cluster or a confederation sub-AS.

   When the aforementioned practice for devising a route reflection
   cluster or confederation sub-AS is followed in a network, we claim
   that the advertisement of all the Group Best Paths by a route
   reflector or a confederation ASBR is sufficient to eliminate the MED-
   induced route oscillations in the network.  This claim is validated
   in Appendix A.

   Note that a Group Best Path for an address prefix can be identified
   by the combination of the address prefix and the neighbor-AS.  Thus
   this approach can be implemented using the mechanism described in
   ADD-PATH [RFC7911] for advertising multiple paths, and in this case
   the neighbor-AS of a path may be used as the path identifier of the
   path.

   It should be noted that the approach of advertising the Group Best
   Paths requires certain topological constraints to be satisfied in
   order to eliminate the MED-induced route oscillation.  Specific
   topological considerations are described in [RFC3345].

5.  Route Reflection and Confederation

   To allow a route reflector or a confederation ASBR to advertise
   either the Available Paths or Group Best Paths using the mechanism
   described in ADD-PATH [RFC7911], the following revisions are proposed
   for BGP route reflection and BGP Confederation.

5.1.  Route Reflection

   For a particular <AFI, SAFI> a route reflector MUST include the <AFI,
   SAFI> with the "Send/Receive" field set to 2 (send multiple paths) or
   3 (send/receive multiple paths) in the ADD-PATH Capability [RFC7911]
   advertised to an IBGP peer.  When the ADD-PATH Capability is also



Walton, et al.          Expires February 9, 2017                [Page 4]


Internet-Draft          BGP Oscillation Solutions            August 2016


   received from the IBGP peer with the "Send/Receive" field set to 1
   (receive multiple paths) or 3 (send/receive multiple paths) for the
   same <AFI, SAFI>, then the following procedures apply:

   If the peer is a route reflection client, the route reflector MUST
   advertise to the peer the Group Best Paths (or the Available Paths)
   received from its non-client IBGP peers.  The route reflector MAY
   also advertise to the peer the Group Best Paths (or the Available
   Paths) received from its clients.

   If the peer is a non-client, the route reflector MUST advertise to
   the peer the Group Best Paths (or the Available Paths) received from
   its clients.

5.2.  Confederation

   For a particular <AFI, SAFI> a confederation ASBR MUST include the
   <AFI, SAFI> with the "Send/Receive" field set to 2 (send multiple
   paths) or 3 (send/receive multiple paths) in the ADD-PATH Capability
   [RFC7911] advertised to an IBGP peer, and to a confederation external
   peer.  When the ADD-PATH Capability is also received from the IBGP
   peer or the confederation external peer with the "Send/Receive" field
   set to 1 (receive multiple paths) or 3 (send/receive multiple paths)
   for the same <AFI, SAFI>, then the following procedures apply:

   If the peer is internal, the confederation ASBR MUST advertise to the
   peer the Group Best Paths (or the Available Paths) received from its
   confederation external peers.

   If the peer is confederation external, the confederation ASBR MUST
   advertise to the peer the Group Best Paths (or the Available Paths)
   received from its IBGP peers.

6.  Deployment Considerations

   Some route oscillations, once detected, can be eliminated by simple
   configuration workarounds.  As carrying additional paths impacts the
   memory usage and routing convergence in a network, it is recommended
   that the impact be evaluated and the approach of using a
   configuration workaround be considered in deciding whether to deploy
   the proposed mechanism in a network.  In addition, the advertisement
   of multiple paths should be limited to those prefixes which are
   affected by MED-induced route oscillation.

   While the route reflectors or confederation ASBRs in a network need
   to advertise the Group Best Paths or Available Paths, the vast
   majority of the BGP speakers in the network only need to receive the




Walton, et al.          Expires February 9, 2017                [Page 5]


Internet-Draft          BGP Oscillation Solutions            August 2016


   Group Best Paths or Available Paths, which would involve only minor
   software changes.

   It should be emphasized that in order to eliminate the MED-induced
   route oscillations in a network using the approach of advertising the
   Group Best Paths, the recommended practice for devising a route
   reflection cluster or confederation sub-AS with respect to the IGP
   metrics ([RFC4456], [RFC5065]) should be followed.

   It is expected that the approach of advertising the Group Best Paths
   would be adequate to achieve consistent routing for the vast majority
   of the networks.  For a network that has large number of alternate
   paths, the approach should be a good choice as the number of paths
   advertised by a reflector or a confederation ASBR is bounded by the
   number of the neighbor-AS's for a particular address prefix.  The
   additional states for an address prefix would also be per neighbor-AS
   based rather than per path based.  The number of the neighbor-AS's
   for a particular address prefix is typically small because of the
   limited number of upstream providers for a customer and the nature of
   advertising only customer routes at the inter-exchange points.

   The approach of advertising the Group Best Paths, however, may still
   be inadequate for certain networks to avoid other routing
   inconsistencies such as forwarding loops.  The required topological
   constraints could also be operationally challenging.  In these cases
   the approach of advertising the Available Paths may be used, but
   should be limited to those prefixes which are affected by MED-induced
   route oscillation in a network carrying a large number of alternate
   paths.

7.  IANA Considerations

   This memo includes no request to IANA.

8.  Security Considerations

   This extension to BGP does not change the underlying security issues
   inherent in the existing BGP [RFC4271].

9.  Acknowledgements

   We would like to thank David Cook and Naiming Shen for their
   contributions to the design and development of the solutions.

   Many thanks to Tony Przygienda, Sue Hares, Jon Mitchell and Paul
   Kyzivat for their helpful suggestions.





Walton, et al.          Expires February 9, 2017                [Page 6]


Internet-Draft          BGP Oscillation Solutions            August 2016


10.  References

10.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <http://www.rfc-editor.org/info/rfc2119>.

   [RFC4271]  Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
              Border Gateway Protocol 4 (BGP-4)", RFC 4271,
              DOI 10.17487/RFC4271, January 2006,
              <http://www.rfc-editor.org/info/rfc4271>.

   [RFC4456]  Bates, T., Chen, E., and R. Chandra, "BGP Route
              Reflection: An Alternative to Full Mesh Internal BGP
              (IBGP)", RFC 4456, DOI 10.17487/RFC4456, April 2006,
              <http://www.rfc-editor.org/info/rfc4456>.

   [RFC5065]  Traina, P., McPherson, D., and J. Scudder, "Autonomous
              System Confederations for BGP", RFC 5065,
              DOI 10.17487/RFC5065, August 2007,
              <http://www.rfc-editor.org/info/rfc5065>.

   [RFC7911]  Walton, D., Retana, A., Chen, E., and J. Scudder,
              "Advertisement of Multiple Paths in BGP", RFC 7911,
              DOI 10.17487/RFC7911, July 2016,
              <http://www.rfc-editor.org/info/rfc7911>.

10.2.  Informative References

   [RFC3345]  McPherson, D., Gill, V., Walton, D., and A. Retana,
              "Border Gateway Protocol (BGP) Persistent Route
              Oscillation Condition", RFC 3345, DOI 10.17487/RFC3345,
              August 2002, <http://www.rfc-editor.org/info/rfc3345>.

Appendix A.  Why the Group Best Paths Are Adequate?

   It is assumed that the following common practice is followed.  A
   route reflection cluster or a confederation sub-AS should be designed
   such that the IGP metrics for links within a cluster (or
   confederation sub-AS) are much smaller than the IGP metrics for the
   links between the clusters (or confederation sub-AS).  This practice
   helps achieve consistent routing within a route reflection cluster or
   a confederation sub-AS.

   Observe that in a network that maintains full IBGP mesh only the
   paths that survive the (Local_Pref, AS-PATH Length, Origin, MED)



Walton, et al.          Expires February 9, 2017                [Page 7]


Internet-Draft          BGP Oscillation Solutions            August 2016


   comparisons [RFC4271] would contribute to the route selection in the
   network.

   Consider a route reflection cluster that sources one or more paths
   that would survive the (Local_Pref, AS-PATH Length, Origin, MED)
   comparisons among all the paths in the network.  One of these
   surviving paths would be selected as the Group Best Path by the route
   reflector in the cluster.  Due to the constrain on the IGP metrics as
   described previously, this path would remain as the Group Best Path
   and would be advertised to all other clusters even after a path is
   received from another cluster.

   On the other hand, when no path in a route reflection cluster would
   survive the (Local_Pref, AS-PATH Length, Origin, MED) comparisons
   among all the paths in the network, the Group Best Path (when exists)
   for a route reflector would be from another cluster.  Clearly the
   advertise of the Group Best Path by the route reflector to the
   clients only depends on the paths received from other clusters.

   Therefore there is no MED-induced route oscillation in the network as
   the advertisement of a Group Best Path to a peer does not depend on
   the paths received from that peer.

   The claim for the confederation can be validated similarly.

Authors' Addresses

   Daniel Walton
   Cumulus Networks
   140C S. Whisman Rd.
   Mountain View, CA  94041
   USA

   Email: dwalton@cumulusnetworks.com


   Alvaro Retana
   Cisco Systems, Inc.
   7025 Kit Creek Rd.
   Research Triangle Park, NC  27709
   USA

   Email: aretana@cisco.com








Walton, et al.          Expires February 9, 2017                [Page 8]


Internet-Draft          BGP Oscillation Solutions            August 2016


   Enke Chen
   Cisco Systems, Inc.
   170 W. Tasman Dr.
   San Jose, CA  95134
   USA

   Email: enkechen@cisco.com


   John Scudder
   Juniper Networks
   1194 N. Mathilda Ave
   Sunnyvale, CA  94089
   USA

   Email: jgs@juniper.net



































Walton, et al.          Expires February 9, 2017                [Page 9]