Internet-Draft Consistent inter-domain consistency October 2023
Cheng, et al. Expires 25 April 2024 [Page]
Workgroup:
IDR
Internet-Draft:
draft-cheng-idr-inter-domain-consistency-00
Published:
Intended Status:
Standards Track
Expires:
Authors:
W. Cheng
China Mobile
M. Liu
Huawei Technologies
M. Huang
Huawei Technologies
S. Yue
China Mobile

Maintain consistency of inter-domain routing and forwarding

Abstract

The AS_PATH attribute of BGP UPDATE records the AS number that it has passed through. Ideally, the traffic to the destination IP prefixes in NLRI of BGP UPDATE should be reversely forwarded along the AS path recorded in the AS_PATH. However, due to other factors such traffic redirection, traffic engineering, and route aggregation, the actual forwarding AS path of the packet is usually different from the propagation path of BGP UPDATE. In other words, inter-domain routing and forwarding are usually inconsistent, which results in that inter-domain forwarding has security risks such as traffic black hole, loop, detour and the malicious AS, etc. Consistent inter-domain routing and forwarding greatly contributes to enhanced inter-domain forwarding security and visibility, which facilitates the long-term evolution of the Internet

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 25 April 2024.

1. The inconsistency of inter-domain routing and forwarding

The path from the origin AS to the destination AS is determined by BGP protocol and non-BGP factors. As a result, the actual forwarding AS path of the packet is usually different with the expected AS path which is the same as the AS_PATH attribute in BGP UPDATE. In other words, inter-AS routing and forwarding are inconsistent.

The inconsistency means that inter-domain routing is a black box. The origin AS, maybe any AS, does not see the actual forwarding AS path from the route, which cause these paths may not comply with some local defined rules, e.g., including an AS that it does not prefer, violating the valley-free principle.

 +-----+              +-----+              +-----+
 | AS1 |--------------| ASx |--------------| AS2 |
 +-----+              +-----+              +-----+
          BGP Path            Non-BGP Path
Figure 1: The complete forwarding AS path

As shown in the figure, the comlpete forwarding AS path from AS1 to AS2 consists fo BGP paths and non-BGP paths.

2. Reasons for the inconsistency

Inter-domain routing and forwarding inconsistency may be caused by traffic redirection, traffic engineering protocols, and route aggregation, etc.

2.1. Inter-domain traffic Redirection

Due to load-balancing, anti-DDoS, etc., policy-based routing (PBR) or BGP Flowspec may be configured to redirect traffic to a new next-hop AS which is different with the next-hop AS determined by AS_PATH in BGP UPDATE.

The subsequent forwarding AS path is determined by the next-hop AS, which is unknown to the origin AS and the upstream AS. The complete forwarding AS path consists of two parts: the AS path before redirection and the AS path after redirection. This path is not verified by BGP, which may violate the valley-free principle or even contain duplicate ASes, causing traffic blackholes or loops.

2.2. Traffic engineering protocols

Due to load-balancing, network optimization, etc., operators may utilize other protocols such as MPLS，SR to steer traffic to the specified AS path which is different from the AS_PATH in BGP UPDATE.

The complete forwarding AS path is determined by BGP and TE protocols. It may include multiple segments, for example, the first segment is an AS path determined by BGP that starts with the origin AS, the second segment is a TE path and the last segment is still the AS path determined by BGP that ends with the destination AS.

The complete forwarding path is actually transparent to the origin AS and is not verified by its filters, which may incur similar risks as redirection.

2.3. Route Aggregation

To minimize routing tables, route aggregation is widely used in IP networks.

BGP route aggregation causes the ordered AS_Sequence, to be converted to the unordered AS_SET, which are different types of AS_PATH. However, AS_SET does not represent the AS forwarding path of the data packet, which can also lead to the inconsistency between inter-domain routing and forwarding.

2.4. Other factors

Other factors, such as route convergence and route redistribution, may also contribute to the inconsistency between routing and forwarding.

During route convergence, non-simultaneous addition and deletion of routes by multiple ASes may also cause a short-term inconsistency between routing and forwarding. However, as routing completes convergence, eventually routing and forwarding will be consistent. This short-term inconsistency is caused by the flaws in the routing protocol's own design and is beyond the scope of this draft.

Redistributing BGP routes into IGP can result in the loss of AS_PATH. Before advertised to the other AS, the route should be redistributed from IGP into BGP. The other AS that receive the route by BGP considers the destination AS of the route to be the redistribution AS. The complete AS path that the origin AS wisher to obtain is actually the path from it to the redistribution AS. The forwarding AS path before redistribution are outside the scope of this draft for now.

This draft focuses on the factors that contribute to the chronic inconsistency of inter-domain routing and forwarding: inter-domain redirection, TE protocols and route aggregation. There factors are inevitable obstacles to the consistency.

3. The risk of the inconsistency

The inconsistency between inter-domain routing and forwarding result in the origin AS, possibly all ASes, not seeing the complete actual AS forwarding path and not knowing whether the forwarding is secure. Specifically, the inconsistency between the expected AS path and the actual AS path leads to the following risks:

  • Traffic blackhole: there is no corresponding route for the next-hop AS, resulting in a traffic black hole. For example, an AS redirects traffic that should be forwarded to the provider AS to another unrelated customer AS, which also violates the valley-free principle.

  • Loop: the forwarding AS path that has not been checked by BGP may lead to loops, e.g., the AS path before redirection and the AS path after redirection may contain the same AS, which is a risk that cannot be circumvented by the current BGP protocol itself.

  • Detour: the complete forwarding AS path is composed of multiple AS paths from different protocol, which may lead to unnecessary lengthening of AS_PATH.

  • Malicious AS: The actual forwarding path is not visible to the origin AS, which may cause its packets to pass through some ASes it does not expect.

  • Non-optimal route: The AS may prefer some ASes that may not be included in the actual pata to select a non-optimal route.

4. Consistent inter-domain routing and forwarding

During packet transmission from the origin AS to the destination AS, partial AS path may be controlled by redirection or non-BGP protocols, such MPLS, SR, etc. Therefore, to get the complete forwarding AS path by BGP, there are two steps:

  • Obtaining deviation AS paths that results from local non-BGP factors.

  • Advertising the deviation AS path and the steered flow by BGP

4.1. Obtaining deviation AS paths

4.1.1. Obtaining redirection deviation AS paths

A router redirecting traffic to a new next-hop AS generates a deviation AS path. In order to get the direction deviation AS path, the router first acquires the next-hop AS and the destination prefix of the redirection rule. The router then looks up the AS_PATH in Adj_RIBs_In from the next-hop AS by the destination prefix. The AS_PATH is the forwarding AS path after redirection, i.e., the redirection deviation AS path.

Redirection rules usually specify only the next-hop address or the outgoing interface, so it is also necessary to obtain the AS to which the interface is connected. In the prior art, one way is that there will be relevant configuration in the BGP peer that specifies the AS to which the remote peer belongs and the outgoing interface used for the connection. This interface is connected to the AS to which the peer belongs (except for the loopback interface). Additional configuration to specify the AS to which the interface is connected is also a viable approach.

All BGP routes sent by remote peers are stored in Adj-RIBs-in. The feasible redirection AS path should be among these unpreferred BGP routes. The new next-hop AS that was redirected to has no routes to the destination prefix of the redirection rule, which may result in the traffic black hole.

If the redirection rule does not have the destination prefix field, the traffic routed to any destination prefix may be redirected. Thus all AS_PATHs in Adj-RIBs-In are deviation AS paths resulting from the redirection rule. Typically such redirection rules that do not include the destination prefix field and whose next-hop AS lacks reachability are not recommended.

4.1.2. Obtaining deviation AS path from other protocols

Some TE protocols that may direct the specific flow into pre-planned AS paths. Their destination prefix of the steered traffic is obtained in a similar way as in Section 4.1.1.

In the egress router of the TE path, the Adj-RIBs-In is then looked up by the destination prefix to obtain the subsequent AS path. The complete forwarding path, therefore, is stitched together from the TE Path and other AS Paths controlled by BGP.

4.2. Advertising the deviation path 

                                              +-----+
                                            / | AS3 | \
                                           /  +-----+  \
                                          /             \
                                         /               \
+-----+            +-----+           +-----+          +-----+
| ASx |------------| AS1 |-----------| AS2 |----------| AS4 |
+-----+ <--------  +-----+ <-------  +-----+ <------  +-----+
   BGP UPDATE:              BGP UPDATE:          BGP UPDATE:
    AS_PATH:[AS1,AS2,AS4]    AS_PATH:[AS2,AS4]   AS_PATH:[AS4]
    D_PATH:[AS1,AS2,AS3,AS4] D_PATH:[AS2,AS3,AS4]
Figure 2: Advertising the deviation path

As shown in the figure, AS4 advertises a BGP UPDATE to ASx along the AS path ([AS1,AS2,AS4]). AS2 is a redirection AS which redirect the packet routing to AS4 to AS3. As a result, the actual forwarding path of packets from AS1 to AS2 is [AS1, AS2, AS3, AS4]. So AS2 should add D_PATH to BGP UPDATE. D_PATH refers to the actual forwarding AS path through which the packet sent to AS4.

The AS that generates the deviation AS path is obliged to advertise it to other ASes. For example, the AS that is configured with the redirection rule advertises the redirection deviation path to the origin AS. The origin AS then combines the AS path to the redirection AS and the redirection deviation AS path to get the complete forwarding AS path which is verified to be optimal or secure using the local AS_PATH filter. The path that passes through malicious ASes will not be selected by the origin AS.

However, only the flow that matches the redirection rule will go through the deviation path, and the other missed flow is still forwarded along the original path planned by BGP. Therefor, the redirection AS should jointly advertise the deviation path and the specific flow to other ASes.

In order to maintain the consistency of inter-domain routing and forwarding, the deviation AS path should be propagated to other ASes in the form of route advertisement. In other words, the deviation AS path and attributes of the specific flow should be included in BGP UPDATE, so that utilizing the BGP protocol, other ASes can see that some traffic will go through the AS path controlled by non-BGP factors. The other ASes can perform TE and QoS according to the real forwarding AS path of the flow, which is helpful to reduce the risk of unreachability and the potential damage incurred by malicious ASes.

5. Benefits

Since the birth of the Internet, inter-AS TE has been a very complex and difficult task. One of the major reasons is that inter-domain routing and forwarding are inconsistent. The AS_PATH attrtibute in BGP UPDATE only represents the propagation path of the route, which may not correspond to the actual forwarding path.

The community has designed many complex protocols to plan the forwarding path and guarantee SLA, while routing protocols are usually utilized to get the basic reachability. If inter-domain routing and forwarding keep consistent, TE and routing security will be significantly simplified. Moreover, the ability of the origin AS to plan the forwarding AS path of its own path opens the black box of the Internet. Problems that have plagued the Internet for decades, such as visualization and troubleshooting, may be solved.

8. Security Considerations

This document does not introduce any new security considerations.

Authors' Addresses

Weiqiang Cheng
China Mobile
Beijing
China
Mingxing Liu
Huawei Technologies
Beijing
China
Mingqing Huang
Huawei Technologies
Beijing
China
Shengnan Yue
China Mobile
Beijing
China