Skip to main content

A solution to the Hierarchical Route Reflector issue in RT Constraints

The information below is for an old version of the document.
Document Type
This is an older version of an Internet-Draft whose latest revision state is "Expired".
Authors SATYA R MOHANTY , Juan Alcaide , Mrinmoy Ghosh
Last updated 2023-10-23 (Latest revision 2023-03-07)
RFC stream (None)
Stream Stream state (No stream defined)
Consensus boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date (None)
Responsible AD (None)
Send notices to (None)
Network Working Group                                       S R. Mohanty
Internet-Draft                                                J. Alcaide
Intended status: Standards Track                                M. Ghosh
Expires: 25 April 2024                               Cisco Systems, Inc.
                                                         23 October 2023

 A solution to the Hierarchical Route Reflector issue in RT Constraints


   Route Target Constraints (RTC) is used to build a VPN route
   distribution graph such that routers only receive VPN routes
   corresponding to specified route-targets (RT) that they are
   interested in.  This is done by exchanging the route-targets as
   routes in the RTC address-family and a corresponding "RT filter" is
   installed that influences the VPN route advertisement.  In networks
   employing hierarchical Route Reflectors (RR) the use of RTC can lead
   to incorrect VPN route distribution and loss in connectivity as
   detailed in an earlier draft . Two solutions were provided to
   overcome the problem.

   This draft presents a method with suggested modifications to the RTC
   RFC in order to solve the hierarchical RR RTC problem in an efficient

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 25 April 2024.

Copyright Notice

   Copyright (c) 2023 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

Mohanty, et al.           Expires 25 April 2024                 [Page 1]
Internet-Draft       Hierarchical RR RT-Constraints         October 2023

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Requirements Language . . . . . . . . . . . . . . . . . . . .   2
   2.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   3.  RTC and RR Rules  . . . . . . . . . . . . . . . . . . . . . .   3
   4.  Problem Definition  . . . . . . . . . . . . . . . . . . . . .   3
   5.  Proposed Solution . . . . . . . . . . . . . . . . . . . . . .   6
     5.1.  Overwriting of Attributes . . . . . . . . . . . . . . . .   6
     5.2.  Receiver Acceptance Rule  . . . . . . . . . . . . . . . .   7
   6.  Conclusion  . . . . . . . . . . . . . . . . . . . . . . . . .   8
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   8
   8.  Operational Considerations  . . . . . . . . . . . . . . . . .   8
   9.  Security Considerations . . . . . . . . . . . . . . . . . . .   8
   10. Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   8
   11. Normative References  . . . . . . . . . . . . . . . . . . . .   8
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   9

1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   document are to be interpreted as described in [RFC2119].

2.  Introduction

   Hierarchical RR [RFC4456] deployments with VPN [RFC4364] working in
   conjunction with RTC [RFC4684] may result in sub-optimal and
   incorrect VPN route distribution that is nicely described in
   [I-D.ietf-idr-rtc-hierarchical-rr].  The root reason for this is the
   way the RR rules for RTC are defined in [RFC4684].  The authors of
   [I-D.ietf-idr-rtc-hierarchical-rr] furnish two solutions for the
   problem, one based on add-paths and the other based on diverse-paths
   constructs.  In this memo, we present another another solution to the
   very same problem.

Mohanty, et al.           Expires 25 April 2024                 [Page 2]
Internet-Draft       Hierarchical RR RT-Constraints         October 2023

3.  RTC and RR Rules

   When advertising RT membership NLRI to a route-reflector client,
   Section 3.2 of [RFC4684] advocates the advertising RR to set the
   ORIGINATOR_ID attribute [RFC4456] to its own router-id, and the Next-
   hop attribute to be set to the local address for that session.
   However, this creates the issue in hierarchical RR setups as
   explained in [I-D.ietf-idr-rtc-hierarchical-rr].  Fig. 1 represents
   the same Figure as in [I-D.ietf-idr-rtc-hierarchical-rr].  When RR-2
   and RR-3 advertise RT-1 to RR-1, the latter will choose one of the
   routes to be best and will advertise the same to RR-2 and RR-3
   respectively after setting the ORIGINATOR_ID and next-hop to itself.
   Note that RR-1 will also add its own CLUSTER_ID [RFC4456]to the
   CLUSTER_LIST but importantly not overwrite the CLUSTER_ID of the
   sender.  This leads to the issue explained in

4.  Problem Definition

   In the Fig 1, when RR-1 chooses the route from RR-2 as the best
   route, and formats the next-hop and ORIGINATOR_ID as explained above
   and then advertises the route to RR-2, RR-2 will drop the route
   reflected from RR-1 because of the CLUSTER_ID check.

Mohanty, et al.           Expires 25 April 2024                 [Page 3]
Internet-Draft       Hierarchical RR RT-Constraints         October 2023

                       |              +----+             |
                       |        Clu-1 |RR-1|             |
                       |             /+----+\            |
                       |            /        \           |
                       |         +----+    +----+        |
                       |  Clu-2  |RR-2|    |RR-3|  Clu-3 |
                       |         +-/--+    +/--\+        |
                       |          /        /    \        |
                       |     +----+    +----+    +----+  |
                       |     |PE-1|    |PE-2|    |PE-3|  |
                       |     +----+    +----+    +----+  |
                       |       |          |         |    |
                          RT-1 |     RT-1 |         | RT-1
                       +--------+   +--------+    +--------+
                       |  VPN-1 |   |  VPN-1 |    |  VPN-1 |
                       +--------+   +--------+    +--------+

                Figure 1 Hierarchical RR Setup with RTC

                                  Figure 1

   RR-2 will therefore not form the outbound filter of RT-1 towards RR-1
   which means that after convergence RR-2 will not advertise VPN routes
   to RR-1 anymore.  This leads to an incorrect VPN route distribution
   across the network.

   In the scenario of Fig 2.  CE-1 is multi-homed to PE-1 and PE-2 and
   wants to communicate with CE-2 which is behind PE-4.  As explained
   earlier, because RR-1 chooses RR-2 path as best in the RTC family,
   RR-1 is only receiving the VPN route from RR-3 (and not RR-2) in the
   steady state.

Mohanty, et al.           Expires 25 April 2024                 [Page 4]
Internet-Draft       Hierarchical RR RT-Constraints         October 2023

                     |              +----+    +-----+  |  +------------+
                     |        Clu-1 |RR-1| ---|PE-4 |- - -| VPN-1 (CE2)|
                     |             /+----+\   +-----+  |  +------------+
                     |            /        \           |
                     |         +----+    +----+        |
                     |  Clu-2  |RR-2|    |RR-3|  Clu-3 |
                     |         +-/--+    +--\-+        |
                     |          /            \         |
                     |     +----+           +----+     |
                     |     |PE-1|           |PE-2|     |
                     |     +----+           +--/-+     |
                     |         \             /         |
                                 \   RT-1  /
                                 +--------+ ----|
                                 |  VPN-1 (CE1) |

             Figure 2 Hierarchical RR Setup with RTC with dual-homed CE

                               Figure 2

   Notice that even though the link between between RR-3 and RR-1 comes
   down, The RR-2 PATH still remains as best in the RTC address-family
   at RR-1 and the VPN route advertisements to RR-1 from RR-2 still
   continue to be blocked.  Thus even though there is an alternative
   connectivity from CE-1 to PE-4 via PE-1, RR-2 and RR-1, the BGP VPN
   routes cannot be sent.  In fact CE-1 is completely cut-off from rest
   of the network.  Generalizing, it means that in a hierarchical RR
   with only a single first-level RR as its client, the solution is
   completely broken.  Notice that without RTC, RR-1 would have both VPN
   paths and the loss of connectivity to RR-3 would just result in local
   convergence at RR-1 subject to the time when the path from RR-2
   becomes best.

   The solutions presented in [I-D.ietf-idr-rtc-hierarchical-rr] are
   based on

   a.  Addpath, RR-1 will advertise both the paths from RR-2 and RR-3 to
       RR-2 and RR-3 so that each of the first level RRS will accept at
       least one of the routes and install the filter

Mohanty, et al.           Expires 25 April 2024                 [Page 5]
Internet-Draft       Hierarchical RR RT-Constraints         October 2023

   b.  When RR-1 will advertise the best-path to a client or non-client
       speaker, and that speaker is the one whose path is the best, the
       advertising router will use the most "diverse" path (different
       next-hop and ORIGINATOR_ID than the best-path) to accomplish the
       same goal, i.e. the path will be accepted at the receiving

   In the next section, we provide a solution, that does not require
   add-path and also improves upon [RFC4684] while solving this
   hierarchical RR issue in RTC.

5.  Proposed Solution

   A problem that [RFC4684] does not address is limiting the number for
   VPN routes advertised to AN RR when only one client advertises RTC
   routes.  Consider in Figure 1 that PE-1 is the only one advertising a
   RTC route to RR-2.  RR-2 will advertise back the route towards PE-1
   (with next-hop/ORIGINATOR_ID rewriting to avoid PE-1 discarding the
   route).  PE-1 will advertise VPN routes towards RR-1, which is
   unnecessary and wastes resources on RR-1.

   Receiver Acceptance Rule need to change.  We need to take care that
   when reflecting back a RTC route to the advertising client, this
   client does not discard the update.

   The following two solutions are proposed, one for the sender of RTC
   routes, one for the receiver of RTC route.  Only one need to be
   implemented.  Implementing both, one at the receiver and one at the
   sender, allows easier interoperability with non-compliant
   implementations.  If sender option is implemented, it will have
   preference over receiver option.

5.1.  Overwriting of Attributes

   This rule is to be used by the sender of RTC routes.

   When reflecting a RTC route from RR client to RR client, some
   attributes of the route may be overwritten.  This overwrite is
   particular for RTC address family and will not happen for other Afs.
   It allows propagation of RTC routes back to its originators (needed
   for RTC filtering information to be useful), but it makes the
   propagation more prone to loops.  Thus, RTC is expected to be used
   only in standard route-reflectors topology where loops are not

   When reflecting the (best-path) RTC route from RR client to RR
   client, the following rules will apply:

Mohanty, et al.           Expires 25 April 2024                 [Page 6]
Internet-Draft       Hierarchical RR RT-Constraints         October 2023

   *  When RTC route has CLUSTER_LIST, overwrite all CLUSTER_ID of
      CLUSTER_LIST to local CLUSTER_ID.  Note that when advertising that
      RTC route, the local CLUSTER_LIST will still be prepending per
      usual rules.

   *  ORIGINATOR_ID is set or overwritten with local router-id.

   *  NEXT_HOP is overwritten with local peering address (next-hop-

   *  A RTC route will be always advertised to the client we received it

   Note that the rules above only exposes RTC routes to routing loops
   (by overwriting attributes) in the client to client top to down
   direction (i.e. from client to client).  Thus, this draft restricts
   RFC4684 into disallowing attribute overwrite into non-client to
   client direction.

   RFC4684 is not explicit about it, but the underlying assumption is
   that a route received from a route-reflector-client MUST be reflected
   back to that client.  Hereby, this is made explicit.

   RFC4684 mandates to overwrite next-hop to self and the ORIGINATOR_ID
   to our local router-id when advertising RTC route a route reflector
   client (section 3.2, rule (i) ).  The proposed rules are an extension
   of this concept to take care of the case where the reflection happens
   at a higher level RR.

   The rule above will apply as well to a top level RR, guaranteeing
   that top level RR does not have unnecessary routes.

5.2.  Receiver Acceptance Rule

   This rule is to be used by the receiver of RTC routes.

   When receiving a RTC route, the following rules will apply:

   1.  CLUSTER_ID, ORIGINATOR_ID and NEXT_HOP checks will be considered,
       but instead of discarding the routes, the route will be kept in
       Adj-RIB-IN as a Received-only route.

   2.  A route in Received-only state will not be considered for best
       -path nor advertised to any peer

   3.  A route in Received-only state will be considered to install a
       VPN filter.

Mohanty, et al.           Expires 25 April 2024                 [Page 7]
Internet-Draft       Hierarchical RR RT-Constraints         October 2023

   The rules above apply also to just one level of RR, and it’s a
   solution not contemplated in RFC4684.

   The rules above will allow propagation of RTC routes in a different
   way than using the sender option rules (with sender option, non-
   client to client propagation will not be stopped).  But the creation
   of VPN filters will be the same in a standard RR topology.

6.  Conclusion

   With the procedures it is not necessary for the RR to know in which
   level it is operating.  The above rules are compatible.  We always
   advertise best-path for any rule and it is easily seen that RR-2 will
   accept the RT Constraint path advertised from RR-1 . Since the path
   is accepted, the RT Filter at RR-2 will pass the VPN routes, and the
   problem scenarios are resolved accordingly.

   With this specification in the RT-Constraint address-family, we solve
   both the incorrect and sub-optimal issues as mentioned above.  There
   is no need for add-paths.  We can also optimize over [RFC4684] on RTC
   advertisements based on diversity of ORIGINATOR_ID and CLUSTER_ID so
   that a higher level RR does not have to be populated with VPN routes
   with a specific RT if that RT is not present in other clusters.

7.  IANA Considerations


8.  Operational Considerations


9.  Security Considerations

   This document raises no new security issues for RT Constraints.

10.  Acknowledgements

   The authors would like to thank Swadesh Agrawal and M.  Mirza for
   useful discussions related to hierarchical RR RTC.

11.  Normative References

Mohanty, et al.           Expires 25 April 2024                 [Page 8]
Internet-Draft       Hierarchical RR RT-Constraints         October 2023

              Dong, J., Chen, M., and R. Raszuk, "Extensions to RT-
              Constrain in Hierarchical Route Reflection Scenarios",
              Work in Progress, Internet-Draft, draft-ietf-idr-rtc-
              hierarchical-rr-03, 3 July 2017,

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,

   [RFC4364]  Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
              Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February
              2006, <>.

   [RFC4456]  Bates, T., Chen, E., and R. Chandra, "BGP Route
              Reflection: An Alternative to Full Mesh Internal BGP
              (IBGP)", RFC 4456, DOI 10.17487/RFC4456, April 2006,

   [RFC4684]  Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk,
              R., Patel, K., and J. Guichard, "Constrained Route
              Distribution for Border Gateway Protocol/MultiProtocol
              Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual
              Private Networks (VPNs)", RFC 4684, DOI 10.17487/RFC4684,
              November 2006, <>.

Authors' Addresses

   Satya Ranjan Mohanty
   Cisco Systems, Inc.
   225 West Tasman Drive
   San Jose, CA 95134
   United States of America

   Juan Alcaide
   Cisco Systems, Inc.
   225 West Tasman Drive
   San Jose, CA 95134
   United States of America

Mohanty, et al.           Expires 25 April 2024                 [Page 9]
Internet-Draft       Hierarchical RR RT-Constraints         October 2023

   Mrinmoy Ghosh
   Cisco Systems, Inc.
   225 West Tasman Drive
   San Jose, CA 95134
   United States of America

Mohanty, et al.           Expires 25 April 2024                [Page 10]