BGP Color-Aware Routing (CAR)

1.2. Illustration

Here is a brief illustration of the salient properties of the BGP CAR solution.¶

+-------------+      +-------------+      +-------------+
|             |      |             |      |             | V/v with C1
|----+        |------|             |------|        +----|/
| E1 |        |      |             |      |        | E2 |\
|----+        |      |             |      |        +----| W/w with C2
|             |------|             |------|             |
|  Domain 1   |      |   Domain 2  |      |   Domain 3  |
+-------------+      +-------------+      +-------------+

All the nodes are part of an inter-domain network under a single authority and with a consistent color-to-intent mapping:¶

• C1 is mapped to "low-delay"¶

  • Flex-Algo FA1 is mapped to "low delay" and hence to C1¶

• C2 is mapped to "low-delay and avoid resource R"¶

  • Flex-Algo FA2 is mapped to "low delay and avoid resource R" and hence C2¶

E1 receives two service routes from E2:¶

• V/v with BGP Color-EC C1¶

• W/w with BGP Color-EC C2¶

E1 has the following color-aware paths:¶

• (E2, C1) provided by BGP CAR with the following per-domain support:¶

  • Domain1: over IGP FA1¶

  • Domain2: over SR Policy bound to color C1¶

  • Domain3: over IGP FA1¶

• (E2, C2) provided by SR Policy¶

E1 automatically steers traffic for the received service routes as follows:¶

• V/v via (E2, C1) provided by BGP CAR¶

• W/w via (E2, C2) provided by SR Policy¶

Illustrated Properties:¶

• Leverage of the BGP Color-EC¶

  • The service routes are colored with widely used BGP Color Extended-Community [RFC9012] to request intent¶

• (E, C) Automated Steering¶

  • V/v and W/w are automatically steered on the appropriate color-aware path¶

• Seamless co-existence of BGP CAR and SR Policy¶

  • V/v is steered on BGP CAR color-aware path¶

  • W/w is steered on SR Policy color-aware path¶

• Seamless interworking of BGP CAR and SR Policy¶

  • V/v is steered on a BGP CAR color-aware path that is itself resolved within domain 2 onto an SR Policy bound to the color of V/v¶

Other properties:¶

• MPLS data-plane: with 300k PE's and 5 colors, the BGP CAR solution ensures that no single node needs to support a data-plane scaling in the order of Remote PE * C (Section 5). This would otherwise exceed the MPLS data-plane.¶

• Control-Plane: a node should not install a (E, C) path if it's not participating in that color-aware path.¶

• Incongruent Color-Intent mapping: the solution supports the signaling of a BGP CAR route across different color domains (Section 2.8)¶

The key benefits of this model are:¶

• leverage of the BGP Color-EC [RFC9012] to color service routes¶

• the definition of the automated service steering: a C-colored service route V/v from E2 is steered onto a color-aware path (E2, C)¶

• the definition of the data model of a BGP CAR path: (E, C)¶

  • natural extension of BGP IP/LU data model (E)¶

  • consistent with SR Policy data model¶

• the definition of the recursive resolution of a BGP CAR route: a BGP CAR (E2, C) route via N is resolved onto the color-aware path (N, C) which may itself be provided by BGP CAR or via another color-aware routing solution (e.g., SR Policy, IGP Flex-Algo).¶

• Native support for multiple transport encapsulations (e.g., MPLS, SR, SRv6, IP)¶

2.1. Data Model

The BGP CAR data model is:¶

• NLRI Key: Falls into two categories, to accommodate the use-cases described in the introduction:¶

  • Type-1: Key is IP Prefix and Color (E, C). Color in NLRI key distinguishes a color-aware route for a common IP prefix, one per intent. Color also indicates the intent associated with the route.¶

  • Type-2: Key is IP Prefix (E). The unique IP prefix assigned for an intent (i.e IP Prefix == Intent or Color) distinguishes the color-aware route. Color is not necessary in NLRI key as a distinguisher.¶

• NLRI non-key encapsulation data: Data such as MPLS label stack, Label Index and SRv6 SID list associated with NLRI. Contained in TLVs as described in section 2.9.2.1 - 2.9.2.3.¶

• BGP Next Hop.¶

• AIGP Metric [RFC7311]: accumulates color/intent specific metric value for a CAR route across multiple BGP hops.¶

• Local-Color-Mapping Extended-Community (LCM-EC): Optional 32-bit Color value used to represent the intent associated with a CAR route:¶

  • when the CAR route propagates between different color domains.¶

  • when the CAR route has a unique IP prefix for an intent.¶

• BGP Color Extended-Community (Color-EC) [RFC9012]: Optional 32-bit Color value used to represent the intent associated with the BGP CAR next hop. It is used as per [RFC9256] for automated route resolution, when intent/color used for the next hop is different than the CAR route's intent/color.¶

The sections below describe the data model in detail. The sections that describe the protocol processing for CAR SAFI generally apply consistently to both route types (for instance, any operation based on color). The examples use (E, C) for simplicity.¶

2.4. BGP CAR Route Validation

A BGP CAR path (E, C) via next hop N with encapsulation T is valid if color-aware path (N, C) exists with encapsulation T available in data-plane.¶

A local policy may customize the validation process:¶

• The color constraint in the first check may be relaxed. If N is reachable via alternate color(s) or in the default routing table, the route may be considered valid.¶

• The data-plane availability constraint of T may be relaxed to use an alternate encapsulation.¶

• A performance-measurement verification may be added to ensure that the intent associated with C is met (e.g. delay < bound).¶

A path that is not valid MUST NOT be considered for BGP best path selection.¶

2.5. BGP CAR Route Resolution

A BGP color-aware route (E2, C1) with next hop N is automatically resolved over a color-aware route (N, C1) by default. The color-aware route (N, C1) is provided by color aware mechanisms such as IGP Flex-Algo [RFC9350], SR policy [RFC9256] Section 2.2, or recursively by BGP CAR. When multiple producers of (N, C1) are available, the default preference is: IGP Flex-Algo, SR Policy, BGP CAR.¶

Local policy SHOULD provide additional control:¶

• A BGP color-aware route (E2, C1) with next hop N may be resolved over a color-aware route (N, C2): i.e. the local policy maps the resolution of C1 over a different color C2.¶

  • For example, in a domain where resource R is known to not be present, the inter-domain intent C1="low delay and avoid R" may be resolved over an intra-domain path of intent C2="low delay".¶

  • Another example is: if no (N, C1) path is available and the user has allowed resolution to fallback to a C2 path.¶

• Route resolution may be driven by an egress node. In an SRv6 domain, an egress node selects and advertises an SRv6 SID from its locator for intent C1, with a BGP CAR route. In such a case, the ingress node resolves the received SRv6 SID over an IPv6 route for the intent-aware locator of the egress node for C1 or a summary route that covers the locator. This summary route may be provided by SRv6 Flex Algo or BGP CAR Type-2 route itself (e.g., Appendix C.2).¶

• Local policy may map the CAR route to traditional mechanisms that are unaware of color or that provide best effort, such as RSVP-TE, IGP/LDP, BGP LU/IP (e.g., Appendix A.3.2) for brownfield scenarios.¶

Route resolution via a different color C2 can be automated by attaching BGP Color-EC C2 to CAR route (E2, C1), leveraging Automated steering as described in Section 8.4 of Segment Routing Policy Architecture [RFC9256] for BGP CAR routes. This mechanism is illustrated in section B.2. This mechanism SHOULD be supported.¶

For CAR route resolution, Color-EC color if present takes precedence over the route's intent color (LCM-EC if present (Section 2.9.4), or else NLRI color).¶

Local policy takes precedence over the color based automated resolution specified above.¶

The color-aware route (N, C1) may be provided by BGP CAR itself in a hierarchical transport routing design. In such cases, based on the procedures described above, recursive resolution may occur over the same or different CAR route type. Section 7.1.2 describes a scenario where CAR Type-1 route resolves over CAR Type-2.¶

CAR Type-2 route is allowed to be without color for best effort. In this case, resolution is based on BGP next hop N, or when present, a best-effort SRv6 SID advertised by node N.¶

A BGP CAR route may recursively resolve over a BGP route that carries TEA and follows Section 6 of [RFC9012] for validation. In this case, procedures of section 8 of [RFC9012] apply to BGP CAR routes, using color precedence as specified above for resolution.¶

The procedures of [RFC 9012] Section 6 also apply to BGP CAR routes (AFI/SAFI = 1/83 or 2/83). For instance, a BGP CAR BR may advertise a BGP CAR route to an ingress BR or PE with a specific BGP next hop per color, with a TEA or Tunnel Encapsulation EC, as per Section 6 of [RFC9012].¶

BGP CAR resolution in one network domain is independent of resolution in another domain.¶

2.6. AIGP Metric Computation

The Accumulated IGP (AIGP) Attribute [RFC7311] is updated as the BGP CAR route propagates across the network.¶

The value set (or appropriately incremented) in the AIGP TLV corresponds to the metric associated with the underlying intent of the color. For example, when the color is associated with a low-latency path, the metric value is set based on the delay metric.¶

Information regarding the metric type used by the underlying intra-domain mechanism can also be used to set the metric value.¶

If BGP CAR routes traverse across a discontinuity in the transport path for a given intent, a penalty is added in accumulated IGP metric (value set by user policy). For instance, when color C1 path is not available, and route resolves via color C2 path (See Appendix A.3 for an example).¶

AIGP metric computation is recursive.¶

To avoid continuous IGP metric changes causing end to end BGP CAR route churn, an implementation should provide thresholds to trigger AIGP update.¶

Additional AIGP extensions may be defined to signal state for specific use-cases: MSD along the BGP CAR route advertisement, Minimum MTU along the BGP CAR advertisement. This is out of scope for this document.¶

2.7. Native MultiPath Capability

The (E, C) route definition inherently provides availability of redundant paths at every BGP hop, identical to BGP-LU or BGP IP. For instance, BGP CAR routes originated by two or more egress ABRs in a domain are advertised as multiple paths to ingress ABRs in the domain, where they become equal-cost or primary-backup paths. A failure of an egress ABR is detected and handled by ingress ABRs locally within the domain for faster convergence, without any necessity to propagate the event to upstream nodes for traffic restoration.¶

BGP ADD-PATH [RFC7911] SHOULD be enabled for BGP CAR to signal multiple next hops through a transport RR.¶

2.8. BGP CAR Signaling through different Color Domains

          [Color Domain 1   A]-----[B     Color Domain 2     E2]
          [C1=low-delay      ]     [C2=low-delay               ]

Let us assume a BGP CAR route (E2, C2) is signaled from B to A, two border routers of respectively domain 2 and domain 1. Let us assume that these two domains do not share the same color-to-intent mapping (i.e, they belong to different color domains). Low-delay in domain 2 is color C2, while it is C1 in domain 1 (C1 <> C2).¶

The BGP CAR solution seamlessly supports this rare scenario while maintaining the separation and independence of the administrative authority in different color domains.¶

The solution works as described below:¶

• Within domain 2, the BGP CAR route is (E2, C2) via E2.¶

• B signals to A the BGP CAR route as (E2, C2) via B with Local-Color-Mapping-Extended-Community (LCM-EC) of color C2.¶

• A is aware (as per classic peering agreement) of the intent-to-color mapping within domain 2 ("low-delay" in domain 2 is C2).¶

• A maps C2 in LCM-EC to C1 and signals within domain 1 the received BGP CAR route as (E2, C2) via A with LCM-EC(C1).¶

• The nodes within the receiving domain 1 use the local color encoded in the LCM-EC for next-hop resolution and service steering.¶

The following procedures apply at a color domain boundary for BGP CAR routes, performed by route policy at the sending and receiving peer:¶

• Use local policy to control which routes are advertised to or accepted from a peer in a different color domain.¶

• Attach LCM-EC if not present with the route. If LCM-EC is present, then update the value to re-map the color as needed.¶

  • This function may be done by the advertising BGP speaker or the receiving BGP speaker as determined by the operator peering agreement, and indicated by local policy on the BGP peers.¶

These procedures apply to both CAR route types, in addition to all procedures specified in earlier sections. LCM-EC is described in Section 2.9.¶

Salient properties:¶

• The NLRI never changes, even though the color-to-intent mapping changes¶

• E is globally unique, which makes E-C in that order unique¶

• In the vast majority of the cases, the color of the NLRI is used for resolution and steering¶

• In the rare case of color incongruence, the local color encoded in LCM-EC takes precedence¶

Operational consideratons are in Section 11. Further illustrations are provided in Appendix B.¶

2.9. Format and Encoding

BGP CAR leverages the BGP multi-protocol extensions [RFC4760] and uses the MP_REACH_NLRI and MP_UNREACH_NLRI attributes for route updates by using the SAFI value 83 along with AFI 1 for IPv4 prefixes and AFI 2 for IPv6 prefixes.¶

BGP speakers MUST use BGP Capabilities Advertisement to ensure support for processing of BGP CAR updates. This is done as specified in [RFC4760], by using capability code 1 (multi-protocol BGP), with AFI 1 and 2 (as required) and SAFI 83.¶

The Next Hop network address field in the MP_REACH_NLRI may either be an IPv4 address or an IPv6 address, independent of AFI. If the next hop length is 4, then the next hop is an IPv4 address. The next hop length may be 16 or 32 for an IPv6 next hop address, set as per section 3 of [RFC2545]. Processing of the Next Hop field is governed by standard BGP procedures as described in section 3 of [RFC4760].¶

The sub-sections below specify the generic encoding of the BGP CAR NLRI followed by the encoding for specific NLRI types introduced in this document.¶

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  NLRI Length  |  Key Length   |   NLRI Type   |              //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+              //
|                  Type-specific Key Fields                    //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           Type-specific Non-Key Fields (if applicable)       //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

where:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  NLRI Length  |  Key Length   |   NLRI Type   |Prefix Length  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|               IP Prefix (variable)                           //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|               Color (4 octets)                                |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Followed by optional TLVs encoded as below:

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     Type      |    Length     |    Value (variable)          //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

where:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|T|  Type     |    Length     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Followed by one (or more) Labels encoded as below:

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                 Label                 |Rsrv |S|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

where:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|T|   Type    |    Length     |    Reserved   |     Flags     ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~               |                 Label Index                   ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~               |
+-+-+-+-+-+-+-+-+

where:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|T|  Type     |    Length     |   SRv6 SID Info (variable)   //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

where:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  NLRI Length  |  Key Length   |   NLRI Type   |Prefix Length  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|               IP Prefix (variable)                           //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Followed by optional TLVs encoded as below:

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|T|   Type    |    Length     |    Value (variable)          //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

where:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     Type=0x3  | Sub-Type=0x1b |          Reserved             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                             Color                             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

where:

2.10. LCM-EC and BGP Color-EC usage

There are 2 distinct requirements to be supported as stated in [I-D.hr-spring-intentaware-routing-using-color]":¶

1. Domains with different intent granularity (section 6.3.1.9)¶

2. Network domains under different administration, i.e. color domains (section 6.3.1.10)¶

Requirement 1 is the case where within the same administrative or color domain, BGP CAR routes for N end-to-end intents may need to traverse across an intermediate transit domain where only M intents are available, N >= M. For example, consider a multi-domain network is designed as Access-Core-Access. The core may have the most granular N intents, whereas the access only has fewer M intents. So, the BGP next-hop resolution for a CAR route in the access domain must be via a color-aware path for one of these M intents. As the procedures describe in Section 2.5, and the example illustrates in Appendix B.2, BGP Color-EC is used to automate the CAR route resolution in this case.¶

For requirement 2, where CAR routes traverse across different color domains, LCM-EC is used to carry the local color mapping for the NLRI color in other color domains. The related procedures are described in Section 2.8, and an example is given in Appendix B.3.¶

Both LCM-EC and BGP Color-EC may be present at the same time with a BGP CAR route. For axample, a BGP CAR route (E, C1) from color domain D1, with LCM-EC C2 in color domain D2, may also carry Color-EC C3 and next hop N in a transit network domain within D2 where C2 is being resolved via an available intra-domain intent C3 (See the detailed example in the combination of Appendix B.2 and Appendix B.3).¶

In this case, as described in Section 2.5, default order of processing for resolution in presence of LCM-EC is local policy, then BGP Color-EC color, and finally LCM-EC color.¶

2.11. Error Handling

The error handling actions as described in [RFC7606] are applicable for handling of BGP update messages for BGP CAR SAFI. In general, as indicated in [RFC7606], the goal is to minimize the disruption of a session reset or 'AFI/SAFI disable' to the extent possible.¶

When the error determined allows for the router to skip the malformed NLRI(s) and continue processing of the rest of the update message, then it MUST handle such malformed NLRIs as 'Treat-as-withdraw'. In other cases, where the error in the NLRI encoding results in the inability to process the BGP update message, then the router SHOULD handle such malformed NLRIs as 'AFI/SAFI disable' when other AFI/SAFI besides BGP CAR are being advertised over the same session. Alternately, the router MUST perform 'session reset' when the session is only being used for BGP CAR SAFI.¶

The CAR NLRI definition encodes NLRI length and key length explicitly. The NLRI length MUST be relied upon to enable the beginning of the next NLRI field to be located. Key length MUST be relied upon to extract the key and perform 'treat-as-withdraw' for malformed information.¶

A sender MUST ensure that the NLRI and key lengths are number of actual bytes encoded in NLRI and key fields respectively, regardless of content being encoded.¶

Given NLRI length and Key length MUST be valid, failures in following checks result in 'AFI/SAFI disable' or 'session reset':¶

• Minimum NLRI length (must be atleast 2, as key length and NLRI type are required fields).¶

• Key Length MUST be at least two less than NLRI Length.¶

NLRI Type specific error handling:¶

• By default, a speaker SHOULD discard unrecognized or unsupported NLRI type and move to next NLRI.¶

• Key length and key errors of known NLRI type SHOULD result in discard of NLRI similar to unrecognized NLRI type.(This MUST be logged for trouble shooting).¶

Transparent propagation of unrecognized NLRI type:¶

• Key length allows unrecognized route types to transit through RR transparently without a software upgrade. The RR receiving unrecognized route types does not need to interpret the key portion of an NLRI and handles the NLRI as an opaque value of a specific length. An implementation SHOULD provide a knob that controls the RR unrecognized route type propagation behavior and possibly at the granularity of route type values allowed. This configuration knob gives the operator the ability to allow specific route types to be transparently passed through RRs based on client speaker support.¶

• In such a case RR may reflect NLRIs with NLRI type specific key length and field errors. Clients of such RR that consume the route for installation will perform the key error handling of known NLRI type or discard unrecognized type. This prevents propagation of routes with NLRI errors any further in network.¶

Type-Specific Non-Key TLV handling:¶

• Either the length of a TLV would cause the NLRI length to be exceeded when parsing the TLV, or fewer than 2 bytes remain when beginning to parse the TLV. In either of these cases, an error condition exists and the 'treat-as-withdraw' approach MUST be used.¶

• Type specific length constraints should be verified. The TLV MUST be discarded if there is an error. When discarded, an error may be logged for further analysis.¶

• If multiple instances of same type are encountered, all but the first instance MUST be discarded. When discarded, an error may be logged for further analysis.¶

• If a speaker that performs encapsulation to the BGP next hop does not receive at least one recognized forwarding information TLV with T bit unset (such as label or SRv6 SID), such NLRI is considered invalid and not eligible for best path selection. Treat-as-withdraw may be used, though it is recommended to keep the NLRI for debugging purposes.¶

4.1. (E, C) Subscription and Filtering

This section illustrate an (E, C) BGP subscription model that allows to filter the (E, C) routes learned by a BGP CAR node.¶

     E1-----------------A-------------------B-------------------E2
                                             <--- (E2, C1) ----
      -- F (E2, C1) -->   --- F (E2, C1) -->
                        |                   |
      <-- (E2, C1) ----   <--- (E2, C1) ----

• BGP CAR route (E2, C1) advertised by E2 is not unconditionally distributed beyond a certain point (e.g., peer B in the above example).¶

• E1 subscribes to (E2, C1) by advertising a filter route F (E2, C1) to its upstream peer A.¶

• If A has (E2, C1) in its BGP RIB, it will advertise (E2, C1) to E1.¶

• If A does not have (E2, C1), it will advertise F (E2, C1) to its peer B.¶

• B will advertise (E2, C1) to A, which will distribute it to E1.¶

E1 may trigger a subscription for BGP CAR route (E2, C1) as a result of receiving a C1-colored service route V/v from E2, for on-demand steering via (E2, C1).¶

On-demand subscription and filtering procedures are outside the scope of this document.¶

5.1. Ultra-Scale Reference Topology

                                        RD:V/v via E2
         +-----+              +-----+ vpn label:30030 +-----+
 ....... |S-RR1| <........... |S-RR2| <...............|S-RR3| <......
 :       +-----+              +-----+  Color C1       +-----+       :
 :                                                                  :
 :                                                                  :
 :                                                                  :
+:------------+--------------+--------------+--------------+--------:-+
|:            |              |              |              |        : |
|:            |              |              |              |        : |
|:          +---+          +---+          +---+          +---+      : |
|:          |121|          |231|          |341|          |451|      : |
|:          +---+          +---+          +---+          +---+      : |
|---+         |              |              |              |      +---|
| E1|         |              |              |              |      | E2|
|---+         |              |              |              |      +---|
|           +---+          +---+          +---+          +---+        |
|           |122|          |232|          |342|          |452|        |
|           +---+          +---+          +---+          +---+        |
|   Access    |   Metro      |   Core       |   Metro      | Access   |
|   domain 1  |   domain 2   |   domain 3   |   domain 4   | domain 5 |
+-------------+--------------+--------------+--------------+----------+
 iPE         iBRM          iBRC           eBRC           eBRM       ePE

The following description applies to the reference topology above:¶

• Independent ISIS/OSPF SR instance in each domain.¶

• Each domain has Flex Algo 128. Prefix SID for a node is SRGB 168000 plus node number.¶

• A BGP CAR route (E2, C1) is advertised by egress BRM node 451.The route is sourced locally from redistribution from IGP-FA 128.¶

• Not shown for simplicity, node 452 will also advertise (E2, C1).¶

• When a transport RR is used within the domain or across domains, ADD-PATH is enabled to advertise paths from both egress BRs to it's clients.¶

• Egress PE E2 advertises a VPN route RD:V/v with BGP Color extended community C1 that propagates via service RRs to ingress PE E1.¶

• E1 steers V/v prefix via color-aware path (E2,C1) and VPN label 30030.¶

5.2. Deployment model

                                        RD:V/v via E2
         +-----+              +-----+ vpn label:30030 +-----+
 ....... |S-RR1| <........... |S-RR2| <...............|S-RR3| <......
 :       +-----+              +-----+  Color C1       +-----+       :
 :                                                                  :
 :                                                                  :
 :                                                                  :
+:------------+--------------+--------------+--------------+--------:-+
|:            |              |              |              |        : |
|:            |   (E2,C1)    |   (E2,C1)    |   (E2,C1)    |        : |
|:          +---+ via 231  +---+ via 341  +---+ via 451  +---+      : |
|:(E2,C1)   |121|<---------|231|<---------|341|<---------|451|      : |
|: via 121 /+---+ L=168002 +---+ L=168002 +---+ L=168002 +---+      : |
|---+     /   |              |              |              |      +---|
| E1| <--/    |              |              |              |      | E2|
|---+ L=168002|              |              |              |      +---|
|           +---+          +---+          +---+          +---+        |
|           |122|          |232|          |342|          |452|        |
|           +---+          +---+          +---+          +---+        |
|   Access    |   Metro      |   Core       |   Metro      | Access   |
|   domain 1  |   domain 2   |   domain 3   |   domain 4   | domain 5 |
+-------------+--------------+--------------+--------------+----------+
 iPE         iBRM          iBRC           eBRC           eBRM      ePE


168121      168231        168341        168451
168002      168002        168002        168002         168002
 30030       30030         30030         30030          30030     30030

                               (E2,C1)
                      +-----+  via 451        +-----+
                      |T-RR1| <-------------- |T-RR2|
                    / +-----+  L=168002       +-----+\
                   /                                   \
+-------------+---/----------+--------------+-----------\--+----------+
|             |  /           |              |            \ |          |
|  (E2,C1)    | / (451,C1)   |   (451,C1)   |             \|          |
|  via 121  +---+ via 231  +---+ via 341  +---+          +---+        |
|  L=168002 |121| <======= |231| <========|341| <======= |451|        |
|         / +---+ L=168451 +---+ L=168451 +---+          +---+        |
|---+    /    |              |              |              |      +---|
| E1|<--/     |              |              |              |      | E2|
|---+         |              |              |              |      +---|
|           +---+          +---+          +---+          +---+        |
|           |122|          |232|          |342|          |452|        |
|           +---+          +---+          +---+          +---+        |
|   Access    |   Metro      |   Core       |   Metro      | Access   |
|   domain 1  |   domain 2   |   domain 3   |   domain 4   | domain 5 |
+-------------+--------------+--------------+--------------+----------+
 iPE         iBRM          iBRC           eBRC           eBRM      ePE

            168231        168341
168121      168451        168451        168451
168002      168002        168002        168002         168002
 30030       30030         30030         30030          30030     30030

                               (E2,C1)
                      +-----+  via 451        +-----+
                      |T-RR1| <-------------- |T-RR2|
                    / +-----+  L=168002       +-----+\
                   /                                   \
+-------------+---/----------+--------------+-----------\--+----------+
|             |  /           |              |            \ |          |
|  (E2,C1)    | / (451,C1)   |   (451,C1)   |             \|          |
|  via 451  +---+ via 231  +---+ via 341  +---+          +---+        |
|  L=168002/|121| <======= |231| <========|341| <======= |451|        |
|         / +---+ L=168451 +---+ L=168451 +---+          +---+        |
|---+ <--/  //|              |              |              |      +---|
| E1|      // |              |              |              |      | E2|
|---+ <===//  |              |              |              |      +---|
|  (451,C1) +---+          +---+          +---+          +---+        |
|  via 121  |122|          |232|          |342|          |452|        |
|  L=168451 +---+          +---+          +---+          +---+        |
|             |              |              |              |          |
|   Access    |   Metro      |   Core       |   Metro      | Access   |
|   domain 1  |   domain 2   |   domain 3   |   domain 4   | domain 5 |
+-------------+--------------+--------------+--------------+----------+
 iPE         iBRM           iBRC          eBRC           eBRM      ePE

168121      168231        168341
168451      168451        168451        168451
168002      168002        168002        168002         168002
 30030       30030         30030         30030          30030     30030

5.3. Scale Analysis

The following two tables summarize the logically analyzed scaling of the control-plane and data-plane for these three models:¶

     |        E1           |       121           |       231
-----+---------------------+---------------------+--------------------
FLAT | (E2,C) via (121,C)  | (E2,C) via (231,C)  | (E2,C) via (341,C)
-----+---------------------+---------------------+--------------------
H.NHS| (E2,C) via (121,C)  | (E2,C) via (451,C)  |
     |                     | (451,C) via (231,C) | (451,C) via (341,C)
-----+---------------------+---------------------+--------------------
H.NHU| (E2,C) via (451,C)  |                     |
     | (451,C) via (121,C) | (451,C) via (231,C) | (451,C) via (341,C)
-----+---------------------+---------------------+--------------------

     |        E1           |       121           |       231
-----+---------------------+---------------------+--------------------
FLAT | V ->   30030        | 168002 -> 168002    | 168002 -> 168002
     |        168002       |           168231    |           168341
     |        168121       |                     |
-----+---------------------+---------------------+--------------------
H.NHS| V ->   30030        | 168002 -> 168002    | 168451 -> 168451
     |        168002       |           168451    |           168341
     |        168121       |           168231    |
-----+---------------------+---------------------+--------------------
H.NHU| V ->   30030        | 168451 -> 168451    | 168451 -> 168451
     |        168002       |           168231    |           168341
     |        168451       |                     |
     |        168121       |                     |
-----+---------------------+---------------------+--------------------

• The flat model is the simplest design, with a single BGP transport level. It results in the minimum label/SID stack at each BGP hop. However, it significantly increases the scale impact on the core BRs (e.g. 341), whose FIB capacity and even MPLS label space may be exceeded.¶

  • 341's data-plane scales with (E2,C) where there may be 300k E's and 5 C's hence 1.5M entries > 1M MPLS data-plane.¶

• The hierarchical models avoid the need for core BRs to learn routes and install label forwarding entries for (E, C) routes.¶

  • Whether next hop is set to self or left unchanged at 121, 341's data-plane scales with (451,C) where there may be thousands of 451's and 5 C's. Therefore, this scaling is well under the 1 million MPLS labels data-plane limit.¶

  • They also aid faster convergence by allowing the PE routes to be distributed via out-of-band RRs that can be scaled independent of the transport BRs.¶

• The next-hop-self option at ingress BRM (e.g. 121) hides the hierarchical design from the ingress PE, keeping its outgoing label programming as simple as the flat model. However, the ingress BRM requires an additional BGP transport level recursion, which coupled with load-balancing adds data-plane complexity. It needs to support a swap and push operation. It also needs to install label forwarding entries for the egress PEs that are of interest to its local ingress PEs.¶

• With the next-hop-unchanged option at ingress BRM (e.g. 121), only an ingress PE needs to learn and install output label entries for egress (E, C) routes. The ingress BRM only installs label forwarding entries for the egress ABR (e.g. 451). However, the ingress PE needs an additional BGP transport level recursion and pushes a BGP VPN label and two BGP transport labels. It may also need to handle load-balancing for the egress ABRs. This is the most complex data-plane option for the ingress PE.¶

5.4. Scaling Benefits of the (E, C) BGP Subscription and Filtering

The (E, C) subscription scheme from Section 4.1 provides the following theoretical scaling benefits for the models in Section 5.2¶

• An ingress PE (E1) only learns (E, C) routes that it needs to install into data plane for service route automated steering.¶

• An ingress BRM (121) only learns (E, C) routes that it needs to install into data plane (for Next-Hop-Self), or that it needs to distribute towards it's ingress PEs (inline RR with Next-Hop-Unchanged).¶

• An ingress BRM or a transport RR only needs to distribute the necessary subset of (E, C) routes to each client (subscriber); this minimizes their processing load for generating updates.¶

• As a result, withdrawal of (E, C) routes when a remote node fails (E2), may also be faster, aiding better convergence.¶

5.5. Anycast SID

This section describes how Anycast SID complements and improves the scaling designs above.¶

7.2. Deployment Options For CAR SRv6 Locator Reachability Distribution and Forwarding

Since an SRv6 locator (or summary) is an IPv6 prefix, it will be installed into the IPv6 forwarding table on a BGP router (e.g., ABR or ASBR), for packet forwarding. With the use of IPv6 locator prefixes, there is no need to allocate and install per-PE SIDs on each BGP hop to forward packets.¶

A few options to forward packets for BGP SRv6 prefixes described in ([I-D.agrawal-spring-srv6-mpls-interworking] also apply to BGP CAR. These options are described in Section 7.2.1 and Section 7.2.2.¶

7.3. Operational Benefits of using CAR SAFI for SRv6 Locator Prefix Distribution

When reachability to an SRv6 SID is provided by distribution of a locator prefix via underlay routing, BGP IPv6 SAFI (AFI/SAFI=2/1) may also be used for inter-domain distribution of these IPv6 prefixes as described in [I-D.agrawal-spring-srv6-mpls-interworking] (Section 7.1.2) or [I-D.ietf-idr-cpr].¶

Using the BGP CAR SAFI provides the following operational benefits:¶

• CAR SAFI is a separate BGP SAFI used for underlay transport intent-aware routing. It avoids overloading of BGP IPv6 SAFI, which also carries Internet (service) prefixes. Using CAR SAFI provides:¶

  • Automatic separation of SRv6 locator (transport) routes from Internet (service) routes,¶

    • Preventing inadvertent leaking of routes.¶

    • Avoiding need to configure specific route filters for locator routes.¶

  • Priority handling of infrastructure routes over service (Internet) routes.¶

• CAR SAFI also supports inter-domain distribution of (E, C) routes sourced from SR-Policy, in addition to SRv6 locator IPv6 prefixes.¶

• CAR SAFI may also be used for best-effort routes in addition to intent-aware routes as described in the next section.¶

Note: If infrastructure routes such as SRv6 locator routes are carried in both BGP-IP [RFC 4271] / BGP-LU [RFC8277, RFC4798], and BGP CAR, Section 8 describes the path selection preference between them.¶

10.2. BGP CAR NLRI Types Registry

IANA is requested to create a "BGP CAR NLRI Types" registry under the "Border Gateway Protocol (BGP) Parameters" registry with this document as a reference. The registry is for assignment of the one octet sized code-points for BGP CAR NLRI types and populated with the values shown below:¶

    Type      NLRI Type                  Reference
-----------------------------------------------------------------
     0        Reserved               [This document]
     1        Color-Aware Route NLRI [This document]
     2        IP Prefix NLRI         [This document]
    3-255     Unassigned

Allocations within the registry are to be made under the "Specification Required" policy as specified in [RFC8126]) and in Section 10.4.¶

10.3. BGP CAR NLRI TLV Registry

IANA is requested to create a "BGP CAR NLRI TLV Types" registry under the "Border Gateway Protocol (BGP) Parameters" registry with this document as a reference. The registry is for assignment of the one octet sized code-points for BGP CAR NLRI non-key TLV types and populated with the values shown below:¶

    Type      NLRI TLV Type                  Reference
-----------------------------------------------------------------
     0        Reserved                   [This document]
     1        Label TLV                  [This document]
     2        Label Index TLV            [This document]
     3        SRv6 SID TLV               [This document]
    4-64      Unassigned

Allocations within the registry are to be made under the "Specification Required" policy as specified in [RFC8126]) and in Section 10.4.¶

10.4. Guidance for Designated Experts

In all cases of review by the Designated Expert (DE) described here, the DE is expected to ascertain the existence of suitable documentation (a specification) as described in [RFC8126] for BGP CAR NLRI Types Registry and BGP CAR NLRI TLV Registry.¶

The DE is also expected to check the clarity of purpose and use of the requested code points. Additionally, the DE must verify that any request for one of these code points has been made available for review and comment within the IETF: the DE will post the request to the IDR Working Group mailing list (or a successor mailing list designated by the IESG). If the request comes from within the IETF, it should be documented in an Internet-Draft. Lastly, the DE must ensure that any other request for a code point does not conflict with work that is active or already published within the IETF.¶

10.5. BGP Extended-Community Registry

IANA has assigned the sub-type 0x1b for "Local Color Mapping (LCM)" in the "Transitive Opaque Extended Community Sub-Types" registry located in the "Border Gateway Protocol (BGP) Extended Communities" registry group.¶

A.1. E2E BGP transport CAR intent realized using IGP Flex-Algo

                              RD:V/v via E2
          +-----+             vpn label: 30030       +-----+
   ...... |S-RR1| <..................................|S-RR2| <.......
   :      +-----+             Color C1               +-----+        :
   :                                                                :
   :                                                                :
   :                                                                :
+-:-----------------------+----------------------+------------------:--+
| :                       |                      |                  :  |
| :                       |                      |                  :  |
| :   (E2,C1) via 121     |   (E2,C1) via 231    | (E2,C1)via E2    :  |
| :   L=168002,AIGP=110 +---+ L=168002,AIGP=10 +---+ L=0x3,LI=8002  :  |
| : |-------------------|121|<-----------------|231|<-------------| :  |
| : V LI=8002           +---+ LI=8002          +---+              | :  |
|----+                    |                      |               +-----|
| E1 |                    |                      |               | E2  |
|----+(E2,C1) via 122     |   (E2,C1) via 232    |  (E2,C1)via E2+-----|
|   ^ L=168002,AIGP=210 +---+ L=168002,AIGP=20 +---+ L=0x3        |    |
|   |----------------   |122|<-----------------|232|<-------------|    |
|     LI=8002           +---+ LI=8002          +---+ LI=8002           |
|                         |                      |                     |
|         ISIS SR         |      ISIS SR         |     ISIS SR         |
|         FA 128          |      FA 128          |     FA 128          |
+-------------------------+----------------------+---------------------+
 iPE                     iABR                   eABR               ePE

         ---------direction of traffic-------->
+------+                  +------+
|168121|                  |168231|
+------+                  +------+
+------+                  +------+                 +------+
|168002|                  |168002|                 |168002|
+------+                  +------+                 +------+
+------+                  +------+                 +------+
|30030 |                  |30030 |                 |30030 |
+------+                  +------+                 +------+

Use case: Provide end to end intent for service flows.¶

• The following description applies to the reference topology above:¶

  • IGP FA 128 is running in each domain, and mapped to Color C1.¶

  • Egress PE E2 advertises a VPN route RD:V/v colored with Color-EC C1 to steer traffic to BGP transport CAR (E2, C1). VPN route propagates via service RRs to ingress PE E1.¶

  • BGP CAR route (E2, C1) with next hop, label index and label as shown above are advertised through border routers in each domain. When a RR is used in the domain, ADD-PATH is enabled to advertise multiple available paths.¶

  • On each BGP hop, (E2, C1) next hop is resolved over IGP FA 128 of the domain. AIGP attribute influences BGP CAR route best path decision as per [RFC7311]. BGP CAR label swap entry is installed that goes over FA 128 LSP to next hop providing intent in each IGP domain. Update AIGP metric to reflect FA 128 metric to next hop.¶

  • Ingress PE E1 learns CAR route (E2, C1). It steers colored VPN route RD:V/v into (E2, C1).¶

• Important:¶

  • IGP FA 128 top label provides intent within each domain.¶

  • BGP CAR label (e.g. 168002) carries end to end intent. Thus it stitches intent over intra domain FA 128.¶

A.3. BGP transport CAR intent realized in a section of the network

                              RD:1/8 via E2
          +-----+             vpn label: 30030       +-----+
   ...... |S-RR1| <..................................|S-RR2| <.......
   :      +-----+             Color C1               +-----+        :
   :                                                                :
   :                                                                :
   :                                                                :
+-:-----------------------+----------------------+------------------:--+
| :                       |                      |                  :  |
| :                       |                      |                  :  |
| :   (E2,C1) via 121     |  (E2,C1) via 231     | (E2,C1) via E2   :  |
| :   L=168002,AIGP=1110+---+L=168002,AIGP=1010+---+ L=0x3          :  |
| : |-------------------|121|<-----------------|231|<-------------| :  |
| : V LI=8002           +---+ LI=8002          +---+              | :  |
|----+                    |                      |               +-----|
| E1 |                    |                      |               | E2  |
|----+(E2,C1) via 122     |  (E2,C1) via 232     | (E2,C1) via E2+-----|
|   ^ L=168002,AIGP=1210+---+L=168002,AIGP=1020+---+ L=0x3        |    |
|   |----------------   |122|<-----------------|232|<-------------|    |
|     LI=8002           +---+ LI=8002          +---+                   |
|                         |                      |                     |
|         ISIS SR         |      ISIS SR         |     ISIS SR         |
|         Algo 0          |      Flex-Algo 128   |     Algo 0          |
|         Access          |      Core            |     Access          |
+-------------------------+----------------------+---------------------+
iPE                     iABR                    eABR                ePE

         ---------direction of traffic-------->
+------+                  +------+
|160121|                  |168231|
+------+                  +------+
+------+                  +------+                 +------+
|168002|                  |168002|                 |160002|
+------+                  +------+                 +------+
+------+                  +------+                 +------+
|30030 |                  |30030 |                 |30030 |
+------+                  +------+                 +------+

                     RD:1/8 via E2
          +-----+         vpn label: 30030           +-----+
   ...... |S-RR1| <..................................|S-RR2| <.......
   :      +-----+             Color C1               +-----+        :
   :                                                                :
   :                                                                :
   :                                                                :
 +-:-----------------------+----------------------+-----------------:-+
 | :                       |                      |                 : |
 | :                       |                      |                 : |
 | :   (E2,C1) via 121     |  (E2,C1) via 231     | (E2,C1) via E2  : |
 | :   L=242003,AIGP=1110+---+L=242002,AIGP=1010+---+ L=0x3         : |
 | : |-------------------|121|<-----------------|231|<-------------|: |
 | : V                   +---+ TE tunnel(231)   +---+              |: |
 |----+                    |                      |               +---|
 | E1 |                    |                      |               |E2 |
 |----+(E2,C1) via 122     |  (E2,C1) via 232     | (E2,C1) via E2+---|
 |   ^ L=242004,AIGP=1210+---+L=242001,AIGP=1020+---+ L=0x3        |  |
 |   |----------------   |122|<-----------------|232|<-------------|  |
 |                       +---+ TE tunnel(232)   +---+                 |
 |                         |                      |                   |
 |                         |                      |                   |
 |         ISIS/LDP        |      ISIS/RSVP-TE    |     ISIS/LDP      |
 |         Access 0        |      Core            |     Access 1      |
 +-------------------------+----------------------+-------------------+
  iPE                    iABR                   eABR               ePE

             ---------direction of traffic-------->
     +------+                  +------+
     |240121|                  |241231|
     +------+                  +------+
     +------+                  +------+                 +------+
     |242003|                  |242002|                 |240002|
     +------+                  +------+                 +------+
     +------+                  +------+                 +------+
     |30030 |                  |30030 |                 |30030 |
     +------+                  +------+                 +------+

A.7. Advertising BGP CAR routes for shared IP addresses

          +-------------+      +--------------+
          |             |      |         +----|
          |             |------|         | E2 |(IP1)
          |----+        |      |         +----|
          | E1 |        |      |  Domain 2    |
          |----+        |      +--------------+
          |             |      +--------------+
          |             |      |         +----|
          |  Domain 1   |------|         | E3 |(IP1)
          +-------------+      |         +----|
                               |  Domain 3    |
                               +--------------+

This example describes a case where a route for the same transport IP address is originated from multiple nodes in different network domains.¶

One use of this scenario is an Anycast transport service, where packet encapsulation (e.g., LSP) may terminate on any one among a set of nodes. All the nodes are capable of forwarding the inner payload, typically via an IP lookup in the global table for Internet routes.¶

A couple of variations of the use-case are described in the example below.¶

One node is shown in each domain, but there will be multiple nodes in practice for redundancy.¶

Example-1: Anycast with forwarding to nearest¶

• Both E2 (in egress domain 2) and E3 (in egress domain 3) advertise Anycast (shared) IP (IP1, C1) with same label L1.¶

• An ingress PE E1 receives by default the best path(s) for (IP1, C1) propagated through BGP hops across the network.¶

• The paths to (IP1, C1) from E2 and E3 may merge at a common node along the path to E1, forming equal cost multipaths or active-backup paths at that node.¶

• Service route V/v is advertised from egress domains D2 and D3 with color C1 and next hop IP1.¶

• Traffic for V/v steered at E1 via (IP1, C1) is forwarded to either E2 or E3 (or both) as determined by routing along the network (nodes in the path).¶

Example-2: Anycast with egress domain visibility at ingress PE¶

• E2 advertises (IP1, C1) and E3 advertises (IP1, C2) CAR routes for the Anycast IP IP1. C1 and C2 are colors assigned to distinguish the egress domains originating the routes to IP1.¶

• An ingress PE E1 receives the best path(s) propagated through BGP hops across the network for both (IP1, C1) and (IP1, C2).¶

• The CAR routes (IP1, C1) and (IP1, C2) do not get merged at any intermediate node, providing E1 control over path selection and load-balancing of traffic across these two routes. Each route may itself provide multipathing or Anycast to a set of egress nodes.¶

• Service route V/v advertised from egress domains D2 and D3 with colors C1 and C2 respectively, but with same next hop IP1.¶

• E1 will resolve and steer V/v path from D2 via (IP1, C1) and path from D3 via (IP2, C2). E1 will load-balance traffic to V/v across the two paths as determined by a local load-balancing policy.¶

• Traffic for colored service routes steered at E1 is forwarded to either E2 or E3 (or load-balanced across both) as determined by E1.¶

In above example, D2 and D3 belonged to the same color or administrative domain. If D2 and D3 belong to different color domains, the domains will coordinate the assignment of colors with shared IP IP1 so that they do not cause conflicts. For instance, in Example-1:¶

• D2 and D3 may both use C1 for the same intent when they originate CAR route for IP1.¶

  • In this case, neither D2 nor D3 will reuse C1 for some other intent.¶

• Alternatively, D2 may use C2 and D3 may use C3 for originating a CAR route for IP1 for the same intent.¶

  • In this case, D2 will not use C3 for originating CAR route for IP1 for some other intent. Similarly, D3 will not use C2 for originating CAR route for IP1 for some other intent.¶

B.2. Single color domain containing network domains with N:M color distribution

• Certain network domains may not be enabled for some of the colors used for end-to-end intents, but may still be required to provide transit for routes of those colors.¶

• When a (E, C1) route traverses a domain where color C1 is not available, the operator may decide to use a different intent of color C2 that is available in that domain to resolve the next hop and establish a path through the domain.¶

  • The next hop resolution may occur via paths of any intra-domain protocol or even via paths provided by BGP CAR.¶

  • The next hop resolution color C2 may be defined as a local policy at ingress or transit nodes of the domain.¶

  • It may also be automatically signaled from egress border nodes by attaching a Color-EC with value C2 to the BGP CAR routes.¶

• Hence, routes of N end-to-end colors may be resolved over paths from a smaller set of M colors in a transit domain, while preserving the original color-awareness end-to-end.¶

• Any ingress PE that installs a service (VPN) route with a color C1, must have C1 enabled locally to install IP routes to (E, C1) and resolve the service route's next hop.¶

• A degenerate variation of this scenario is where a transit domain does not support any color. Appendix A.3 describes an example of this case.¶

Illustration for N end to end intents over fewer M intra domain intents:¶

                     RD:V/v via E2 Color-EC: 100
                     RD:W/w via E2 Color-EC: 200
          +-----+    RD:X/x via E2 Color-EC: 300     +-----+
   ...... |S-RR1| <..................................|S-RR2| <........
  :       +-----+    RD:Y/y via E2 Color-EC: 400     +-----+          :
  :                                                                   :
  :                                                                   :
  :                                                                   :
+-:---------------------+---------------------+----------------------:-+
| :                     |                     |                      : |
|                       |                     |                        |
|     (E2,100) via 121  |   (E2,100) via 231  |     (E2,100) via E2    |
|      Color-EC: 1,10   |    Color-EC: 1,10   |      Color-EC: 1,10    |
|                       |                     |                        |
|     (E2,200) via 121  |   (E2,200) via 231  |     (E2,200) via E2    |
|      Color-EC: 1,20   |    Color-EC: 1,20   |      Color-EC: 1,20    |
|                     <---                  <----                      |
|     (E2,300) via 121  |   (E2,300) via 231  |     (E2,300) via E2    |
|      Color-EC: 2,30   |    Color-EC: 2,30   |      Color-EC: 2,30    |
|                       |                     |                        |
|     (E2,400) via 121  |   (E2,400) via 231  |     (E2,400) via E2    |
|      Color-EC: 2,40   |    Color-EC: 2,40   |      Color-EC: 2,40    |
|                       |                     |                        |
|                     +===+                 +===+                      |
|====+                |   |-------C10-------|   |                +=====|
|    |-------C1-------|   |-------C20-------|   |-------C1-------|     |
| E1 |                |121|                 |231|                | E2  |
|    |-------C2-------|   |-------C30-------|   |-------C2-------|     |
|====+                |   |-------C40-------|   |                +=====|
|                     +===+                 +===+                      |
|       C1=FA132        |      C10=FA128      |       C1=FA132         |
|       C2=FA133        |      C20=FA129      |       C2=FA133         |
|                       |      C30=FA130      |                        |
|                       |      C40=FA131      |                        |
|                       |                     |                        |
|        ISIS SR        |      ISIS SR        |     ISIS SR            |
|        ACCESS         |       CORE          |     ACCESS             |
+-----------------------+---------------------+------------------------+
 iPE                  iABR                  eABR                    ePE

• The following description applies to the reference topology above:¶

  • Core domain provides 4 intra domain intents as described below:¶

    • FA128 mapped to C10,¶

    • FA129 mapped to C20,¶

    • FA130 mapped to C30, and¶

    • FA131 mapped to C40.¶

  • Access domain provides following 2 intra domain intents:¶

    • FA132 mapped to C1, and¶

    • FA133 mapped to C2¶

  • Operator defines following 4 BGP CAR end to end intents as below:¶

    • CAR color C100 that resolves on C1 in access and C10 in core domain,¶

    • CAR color C200 that resolves on C1 in access and C20 in core domain,¶

    • CAR color C300 that resolves on C2 in access and C30 in core domain, and¶

    • CAR color C400 that resolves on C2 in access and C40 in core domain.¶

  • E2 may originate BGP CAR routes with multiple BGP Color-ECs as shown above. At each hop, CAR route's next hop is resolved over the available intra-domain color. For example (E2,C100) with BGP color ECs C1, C10 resolves over C1 at ABR 231, C10 at ABR 121, and C1 at E1.¶

  • Egress PE E2 advertises a VPN route RD:V/v colored with BGP Color-EC C100 to steer traffic through FA 132 in access and FA 128 in core. It also advertises another VPN route RD:W/w colored with BGP Color-EC C200 to steer traffic through FA 132 in access and FA 129 in core.¶

• Important:¶

  • End-to-end (BGP CAR) colors can be decoupled from intra-domain transport colors.¶

  • Each end-to-end BGP CAR color is a combination of various intra-domain colors or intents.¶

  • Combination can be expressed by local policy at ABRs or by attaching multiple BGP Color-ECs at origination point of BGP CAR route.¶

  • Service traffic is steered into suitable CAR color to use the most granular intent in a domain multiple hops away from ingress PE.¶

  • Consistent reuse of standard color based resolution mechanism at both service and transport layers.¶

B.3. Multiple color domains

When the routes are distributed between domains with different color-to-intent mapping schemes, both N:N and N:M cases are possible. Although an N:M mapping is more likely to occur.¶

Reference topology:¶

   D1 ----- D2 ----- D3
   C1       C2       C3

• C1 in D1 maps to C2 in D2 and to C3 in D3.¶

• BGP CAR is enabled in all three color domains.¶

The reference topology above is used to elaborate on the design described in Section 2.8¶

When the route originates in color domain D1 and gets advertised to a different color domain D2, following procedures apply:¶

• The NLRI of the BGP CAR route is preserved end to end i.e. route is (E, C1).¶

• A BR of D1 attaches LCM-EC with value C1 when advertising to a BR in D2.¶

• A BR in D2 receiving (E, C1) maps C1 in received LCM-EC to local color, say C2.¶

  • A BR in D2 may receive (E, C1) from multiple D1 BRs which provide equal cost or primary/backup paths.¶

• Within D2, this LCM-EC value of C2 is used instead of the Color in CAR route NLRI (E, C1). This applies to all procedures described in the earlier section for a single color domain, such as next-hop resolution and service steering.¶

• A colored service route V/v originated in color domain D1 with next hop E and Color-EC C1 will also have its color extended-community value re-mapped to C2, typically at a service RR.¶

• On an ingress PE in D2, V/v will resolve via C2.¶

• When a BR in D2 advertises the route to a BR in D3, the same process repeats.¶

C.1. BGP CAR SRv6 locator reachability hop by hop distribution

                           RD:V/v via E2
          +-----+          SRv6SID=B:C11:2:DT4::     +-----+
   ...... |S-RR1| <..................................|S-RR2| <.....
  :       +-----+                                    +-----+       :
  :                                                                :
  :                                                                :
  :             AS2                                         AS1    :
+-:------------------------------------+            +--------------:--+
| :                                    |            |              :  |
| :                 B:C11::/32 via IP1 |            |              :  |
| :          +-----+ LCM=C1, AIGP=10   |            |              :  |
| :          | TRR |<..............    |            |              :  |
| :          +-----+<..........     :  |            |              :  |
| :             :    B:C11::/32 :   :  |            |              :  |
| :             :       via IP2 :   :  |            |              :  |
| :             : LCM=C1,AIGP=10:   :  |            |              :  |
| :   ......... :               :   :  | B:C11::/32 |              :  |
| : :           :               :   :  | via 231    |           +-----|
| : :           :               :   :  |  LCM=C1    |           | E2  |
  : :    +---+  :   +---+       :   :  |  AIGP=10   |           +-----|
| : :    |P11|<.:..>|P13|       :  +----+        +---+             :  |
| : :    +---+  :   +---+       :  | 121|-----IP1|231|             :  |
| V V           :               :  +----+  eBGP  +---+             :  |
|----+          :               :      |            |           +-----|
| E1 |   +---+  :   +---+       :      |            |           | En  |
|----+   |P12|<.:..>|P14|       :      |            |           +-----|
|        +---+      +---+       :  +----+  eBGP  +---+                |
|        IPv6 FIB:              ...| 122|-----IP2|232|                |
|        B:C11::/32 via IP1        +----+        +---+                |
|                   via IP2            | B:C11::/32 |                 |
|                                      | via 232    |                 |
|                                      | LCM=C1     |                 |
|                                      | AIGP=10    |                 |
|         ISISv6                       |            |     ISISv6      |
|  FA 128 (B:C12::/32)                 |            |FA128(B:C11::/32)|
|  FA 0   (B:02::/32)                  |            |FA0  (B:01::/32) |
+--------------------------------------+            +-----------------+
iPE                                  ASBR          ASBR             ePE

The topology above is an example to illustrate the BGP CAR SRv6 locator prefix route based design (Routed Service SID: Section 7.1.1), with hop by hop IPv6 routing within and between domains.¶

• Multi-AS network with eBGP CAR session between ASBRs.¶

• Transport RR (TRR) peers with P, BR and PE clients within an AS to propagate CAR prefixes. AddPath is enabled to propagate multiple paths.¶

• ISIS (IGP) Flex-Algo 128 for SRv6 is running in each AS (AS may consist of multiple IGP domains), where the following steps apply:¶

  • Prefix B:C11::/32 summarizes Flex-Algo 128 block in AS1 for the given intent. Node locators in the egress domain are sub-allocated from the block for the given intent.¶

  • Similarly, Prefix B:C12::/32 summarizes Flex-Algo 128 block in AS2.¶

  • Per Flex-Algo external subnets for eBGP next hops IP1 and IP2 are distributed in ISIS within AS2.¶

• BGP CAR prefix route B:C11::/32 with LCM C1 is originated by AS1 BRs 231 and 232 on eBGP sessions to AS2 BRs 121 and 122.¶

• ASBR 121 and 122 propagate the route in AS2 to all the P, ABRs and PEs through transport RR.¶

• Every router in AS2 resolves BGP CAR prefix B:C11::/32 next hops IP1 and IP2 in ISISv6 Flex-Algo 128 and programs B:C11::/32 prefix in global IPv6 forwarding table.¶

• AIGP attribute influences BGP CAR route best path decision.¶

• Egress PE E2 advertises a VPN route RD:V/v with SRv6 service SID B:C11:2:DT4::. Service SID is allocated by E2 from its locator of color C1 intent.¶

• Ingress PE E1 learns (via service RRs S-RR1 and S-RR2) VPN route RD:V/v with SRv6 SID B:C11:2:DT4::.¶

• Service traffic encapsulated with SRv6 Service SID B:C11:2:DT4:: is natively steered hop by hop along IPv6 routed path to B:C11::/32 provided by BGP CAR in AS2.¶

• Encapsulated service traffic is natively steered along IPv6 routed path to B:C11::/32 provided by ISISv6 Flex-Algo 128 in AS1.¶

• Design applies to multiple ASNs. BGP next hop is rewritten across a eBGP hop.¶

Important:¶

• No tunneling/encapsulation on Ingress PE and BRs for BGP CAR provided transport.¶

• Uses longest prefix match of SRv6 service SID to BGP CAR IP prefix. No mapping to labels/SIDs, instead use of simple IP based forwarding.¶

Packet forwarding¶

@E1:  IPv4 VRF V/v => H.Encaps.red <B:C11:2:DT4::> => forward based on
                                                        B:C11::/32
@P*:  IPv6 table: B:C11::/32 => forward to interface, NH
@121: IPv6 Table: B:C11::/32 => forward to interface, NH
@231: IPv6 table: B:C11:2::/48 :: => forward via ISISv6 FA path to E2
@231: IPv6 Table B:C11:2::/48 => forward via ISISv6 FA path to E2
@E2:  My SID table B:C11:2:DT4:: =>pop the outer header and lookup the
                                                   inner DA in the VRF

C.2. BGP CAR SRv6 locator reachability distribution with encapsulation

                           RD:V/v via E2
          +-----+          SRv6SID=B:C11:2:DT4::     +-----+
   ...... |S-RR1| <..................................|S-RR2| <.......
   :      +-----+                                    +-----+        :
   :                                                                :
   :                                                                :
   :                                                                :
+-:-----------------------+----------------------+------------------:--+
| :                       |                      |                  :  |
| :                       |                      |                  :  |
| :  B:C11::/32 via 121   |  B:C11::/32 via 231  |                  :  |
| :  SID=B:C13:121:END::  |  SID=B:C12:231:END:: |                  :  |
| :  LCM=C1,AIGP=110    +---+LCM=C1 AIGP=10    +---+                :  |
| : |-------------------|121|<-----------------|231|<-------------| :  |
| : V                   +---+                  +---+              | :  |
|----+                    |                      |               +-----|
| E1 |                    |                      |               | E2  |
|----+                    |                      |               +-----|
|   ^                     |                      |                  :  |
|   |                     |                      |                  :  |
|   |                     |                      |               +-----|
|   |                     |                      |               | En  |
|   |                     |                      |               +-----|
|   |                   +---+                  +---+              |    |
|   |----------------   |122|<-----------------|232|<-------------|    |
|                       +---+                  +---+                   |
|    B:C11::/32 via 122   |  B:C11::/32 via 232  |                     |
|    SID=B:C13:122:END::  |  SID=B:C12:232:END:: |                     |
|    LCM=C1 AIGP=120      |  LCM=C1 AIGP=20      |                     |
|                         |                      |                     |
|         ISISv6          |      ISISv6          |     ISISv6          |
|  FA 128 (B:C13::/32)    | FA 128 (B:C12::/32)  |  FA128 (B:C11::/32) |
|  FA 0   (B:03::/32)     | FA 0   (B:02::/32)   |  FA1 0 (B:01::/32)  |
+-------------------------+----------------------+---------------------+
 iPE                    iABR                    eABR                ePE

The topology above is an example to illustrate the BGP CAR SRv6 locator prefix route based design (Routed Service SID: Section 7.1.1), with intra-domain encapsulation. The example shown is iBGP, but also applies to eBGP (multi-AS).¶

• IGP Flex-Algo 128 is running in each domain, where¶

  • Prefix B:C11::/32 summarizes Flex-Algo 128 block in egress domain for the given intent. Node locators in the egress domain are sub-allocated from the block.¶

  • Prefix B:C12::/32 summarizes FA128 block in transit domain.¶

  • Prefix B:C13::/32 summarizes FA128 block in ingress domain.¶

• BGP CAR route B:C11::/32 is originated by ABRs 231 and 232 with LCM C1. Along the propagation path, border routers set next-hop-self and appropriately update the intra-domain encapsulation information for the C1 intent. For example, 231 and 121 signal SRv6 SID of END behavior [RFC8986] allocated from their respective locators for the C1 intent. (Note, IGP Flex-Algo is shown for intra-domain path, but SR-Policy may also provide the path as shown in Appendix C.3).¶

• AIGP attribute influences BGP CAR route best path decision.¶

• Egress PE E2 advertises a VPN route RD:V/v with SRv6 service SID B:C11:2:DT4::. Service SID is allocated by E2 from its locator of color C1 intent.¶

• Ingress PE E1 learns CAR route B:C11::/32 and VPN route RD:V/v with SRv6 SID B:C11:2:DT4::.¶

• Traffic encapsulated with SRv6 Service SID B:C11:2:DT4:: is steered along IPv6 routed path provided by BGP CAR IP prefix route to locator B:C11::/32.¶

Important¶

• Uses longest prefix match of SRv6 service SID to BGP CAR prefix. No mapping labels/SIDs, instead simple IP based forwarding.¶

• Originating domain PE locators of the given intent can be summarized on transit BGP hops eliminating per PE state on border routers.¶

Packet forwarding¶

@E1:   IPv4 VRF V/v => H.Encaps.red <B:C13:121:END::, B:C11:2:DT4::>
@121: My SID table: B:C13:121:END:: => Update DA with B:C11:2:DT4::
@121: IPv6 Table: B:C11::/32 => H.Encaps.red <B:C12:231:END::>
@231: My SID table: B:C12:231:END:: => Remove IPv6 header; Inner DA B:C11:2:DT4::
@231: IPv6 Table B:C11:2::/48 => forward via ISISv6 FA path to E2
@E2: My SID table B:C11:2:DT4:: =>pop the outer header and lookup the
                                   inner DA in the VRF

C.3. BGP CAR (E, C) route distribution for steering non-routed service SID

                          RD:V/v via E2
         +-----+          SRv6SID: B:01:2:DT4::     +-----+
  ...... |S-RR1| <..................................|S-RR2| <.......
  :      +-----+             Color C2               +-----+        :
  :                                                                :
  :                  +-----+ (E2,C2) via 231                       :
  : -----------------| TRR |-------------------|                   :
  :|                 +-----+  SID=B:C21:2:B6:: |                   :
+-:|---------------------+---------------------|+------------------:--+
| :|                     |                     ||                  :  |
| :|                     |                     ||                  :  |
| :|  B:C21::/32 via 121 |  B:C21::/32 via 231 ||SR policy(E2,C2)  :  |
| :|  LCM=C2,AIGP=110    |  LCM=C2 AIGP=10     ||BSID=B:C21:2:B6:: :  |
| :|                   +---+                  +---+                :  |
| :|-------------------|121|<-----------------|231|<-------------| :  |
| :V SR policy(121,C2) +---+SR policy(231,C2) +---+              | :  |
|----+                   |                      |               +-----|
| E1 |                   |                      |               | E2  |
|----+                   |                      |               +-----|
|  ^ SR policy(122,C2) +---+SR policy(232,C2) +---+              |    |
|  |----------------   |122|<-----------------|232|<-------------|    |
|    B:C21::/32 via 121+---+B:C21::/32 via 232+---+ SR policy(E2,C2)  |
|    LCM=C2,AIGP=120     |   LCM=C2 AIGP=20     |   BSID=B:C21:2:B6:: |
|                        |                      |                     |
|        ISISv6          |      ISISv6          |     ISISv6          |
|     FA 0 (B:03::/32)   |   FA 0 (B:02::/32)   |   FA 0(B:01::/32)   |
+------------------------+----------------------+---------------------+
 iPE                    iABR                   eABR                ePE

The topology above is an example to illustrate the BGP CAR (E, C) route based design (Section 7.1.2). The example is iBGP, but design also applies to eBGP (multi-AS).¶

• SR policy (E2, C2) provides given intent in egress domain.¶

  • SR policy (E2, C2) with segments <B:01:z:END::, B:01:2:END::> where z is the node id in egress domain.¶

• Egress ABRs 231 and 232 redistribute SR policy into BGP CAR NLRI Type-1 (E2,C2) to other domains, with SRv6 SID of End.B6 behavior. This route is propagated to ingress PEs through transport RR (TRR) or inline with next hop unchanged.¶

• The ABRs also advertise BGP CAR prefix route (B:C21::/32) summarizing locator part of SRv6 SIDs for SR policies of given intent to different PEs in egress domain. BGP CAR prefix route propagates through border routers. At each BGP hop, BGP CAR prefix next-hop resolution triggers intra-domain transit SR policy (C2, CAR next hop). For example:¶

  • SR policy (231, C2) with segments <B:02:y:END::, B:02:231:END::>, and¶

  • SR policy (121, C2) with segments <B:03:x:END::, B:03:121:END::>,¶

  • where x and y are node ids within the respective domains.¶

• Egress PE E2 advertises a VPN route RD:V/v with Color-EC C2.¶

• Ingress PE E1 steers VPN route from E2 onto BGP CAR route (E2, C2) that results in H.Encaps.red of SRv6 transport SID B:C21:2:B6:: and SRv6 service SID as last segment in IPv6 header.¶

• IPv6 destination B:C21:2:B6:: match on CAR prefix B:C21::/32 that steers the packet into intra domain (intent-aware) SR Policy on ingress PE E1 and ABR 121.¶

• IPv6 packet destination B:C21:2:B6:: lookup in mySID table on ABR 231 or 232 results in END.B6 behavior (i.e. push of policy segments to E2).¶

Important¶

• Ingress PE steers services via (E,C) CAR route as per [RFC9256].¶

• In data plane (E,C) resolution results in IPv6 header destination being SRv6 SID of END.B6 behavior whose locator is of given intent on originating ABRs.¶

• CAR IP prefix route along the transit path provides simple LPM IPv6 forwarding along the transit BGP hops.¶

• CAR NLRI Type-2 prefix summarizes binding SIDs of all SR policies on originating ABR of a given intent to different PEs in egress domain. This eliminates per PE state on transit routers.¶

Packet forwarding¶

@E1:   IPv4 VRF V/v => H.Encaps.red <B:C21:2:B6::, B:0:E2:DT4::>
                       H.Encaps.red <SR policy (C2,121) sid list>
@121: My SID table: B:03:121:END:: => Remove outer IPv6 header; Inner DA B:C21:2:B6::
@121: IPv6 Table: B:C21::/32 => H.Encaps.red <SR Policy (C2,231) sid
                                                                list>
@231: My SID table: B:02:231:END:: => Remove outer IPv6 header; Inner DA B:C21:2:B6::

@231: MySIDtable B:C21:2:B6:: =>  H.Encaps.red <SR Policy (C2,E2) sid
                                                                list>
@E2: IPv6 Table B:0:2:DT4:: =>pop the outer header and lookup the
                                inner DA in the VRF