Skip to main content

Domain Path (D-PATH) for Ethernet VPN (EVPN) Interconnect Networks
draft-sr-bess-evpn-dpath-04

Document Type Active Internet-Draft (individual)
Authors Jorge Rabadan , Senthil Sathappan , Mallika Gautam , Patrice Brissette , Wen Lin , Bin Wen
Last updated 2023-10-23
RFC stream (None)
Intended RFC status (None)
Formats
Stream Stream state (No stream defined)
Consensus boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-sr-bess-evpn-dpath-04
BESS Workgroup                                           J. Rabadan, Ed.
Internet-Draft                                              S. Sathappan
Intended status: Standards Track                               M. Gautam
Expires: 25 April 2024                                             Nokia
                                                            P. Brissette
                                                           Cisco Systems
                                                                  W. Lin
                                                                 Juniper
                                                                  B. Wen
                                                                 Comcast
                                                         23 October 2023

   Domain Path (D-PATH) for Ethernet VPN (EVPN) Interconnect Networks
                      draft-sr-bess-evpn-dpath-04

Abstract

   The BGP Domain PATH (D-PATH) attribute is defined for Inter-Subnet
   Forwarding (ISF) BGP Sub-Address Families that advertise IP prefixes.
   When used along with EVPN IP Prefix routes or IP-VPN routes, it
   identifies the domain(s) through which the routes have passed and
   that information can be used by the receiver BGP speakers to detect
   routing loops or influence the BGP best path selection.  This
   document extends the use of D-PATH so that it can also be used along
   with other EVPN route types.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 25 April 2024.

Copyright Notice

   Copyright (c) 2023 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

Rabadan, et al.           Expires 25 April 2024                 [Page 1]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  D-PATH to Prevent Loops for EVPN Routes . . . . . . . . .   3
     1.2.  Add Path Visibility and Influence Best Path Selection for
           EVPN Routes . . . . . . . . . . . . . . . . . . . . . . .   4
   2.  Conventions used in this document . . . . . . . . . . . . . .   5
   3.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   5
   4.  Use of Domain Path Attribute (D-PATH) with EVPN routes  . . .   6
     4.1.  D-PATH and Best Path Selection for MAC/IP Advertisement
           routes  . . . . . . . . . . . . . . . . . . . . . . . . .   9
     4.2.  D-PATH and Best Path Selection for Ethernet A-D per EVI
           routes  . . . . . . . . . . . . . . . . . . . . . . . . .   9
     4.3.  D-PATH and Best Path Selection for Inclusive Multicast
           Ethernet Tag routes . . . . . . . . . . . . . . . . . . .  10
     4.4.  Loop Detection  . . . . . . . . . . . . . . . . . . . . .  10
     4.5.  Error Handling  . . . . . . . . . . . . . . . . . . . . .  11
   5.  Use-Case Examples . . . . . . . . . . . . . . . . . . . . . .  11
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .  13
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  13
   8.  Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  13
   9.  Contributors  . . . . . . . . . . . . . . . . . . . . . . . .  13
   10. References  . . . . . . . . . . . . . . . . . . . . . . . . .  14
     10.1.  Normative References . . . . . . . . . . . . . . . . . .  14
     10.2.  Informative References . . . . . . . . . . . . . . . . .  14
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  15

1.  Introduction

   The BGP Domain PATH (D-PATH) attribute
   [I-D.ietf-bess-evpn-ipvpn-interworking] is defined for Inter-Subnet
   Forwarding (ISF) BGP Sub-Address Families that advertise IP prefixes.
   When used along with EVPN IP Prefix routes or IP-VPN routes, it
   identifies the domain(s) through which the routes have passed and
   that information can be used by the receiver BGP speakers to detect
   routing loops or influence the BGP best path selection.  This
   document extends the use of D-PATH so that it can also be used along
   with other EVPN route types.

Rabadan, et al.           Expires 25 April 2024                 [Page 2]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

   The D-PATH attribute can be used to prevent control plane loops for
   EVPN routes, or to provide full path visibility of all the EVPN
   Interconnect Gateways through which a route has gone and modify the
   best path selection based on it.  Some use cases in which D-PATH can
   be used along with (non-IP Prefix) EVPN routes follow, but the use
   cases are not limited to the ones described in this section.

1.1.  D-PATH to Prevent Loops for EVPN Routes

   Figure 1 illustrates an EVPN Interconnect case where EVPN MAC/IP
   Advertisement routes can be looped indefinitely.  The three Gateways
   (GW1, GW2 and GW3) and PE1 in the diagram are attached to the same
   EVPN Broadcast Domain (BD1).  However, BD1 is extended throughout
   three different domains that are interconnected by the Gateways,
   which follow [RFC9014] procedures.  Suppose a host with MAC address
   M1 is learned on GW1 and GW1 advertises an EVPN MAC/IP Advertisement
   route for M1 into Domain-1 and Domain-2.  When the route gets
   imported by GW2 and GW3 and later exported into Domain-3, GW2 and GW3
   may redistribute each other's route for M1 back into Domain-1 and
   Domain-2, respectively, creating a loop.  D-PATH can be used by the
   Gateways when redistributing the route between Domains, to identify
   the Domains through which the route for M1 has gone.  When GW1
   receives an EVPN MAC/IP Advertisement route for M1 that contains a
   D-PATH with a domain-id locally assigned, GW1 identifies the route as
   "looped".

             +----------------+ GW2
             |   EVPN        +-------+
             |   Domain-1    | +---+ |
             |               | |BD1| |---------------+
             |               | +---+ |               |
        GW1  |               +-------+               |    PE1
      +-------+               |     |    EVPN       +-------+
      | +---+ |---------------+     |    Domain-3   | +---+ |
      | |BD1| |                     |               | |BD1| |
      | +---+ |---------------+     |               | +---+ |
      +---|---+               | GW3 |               +---|---+
          |  |   EVPN        +-------+               |  |
      M1--+  |   Domain-2    | +---+ |               |  +--M2
             |               | |BD1| |---------------+
             |               | +---+ |
             |               +-------+
             +----------------+

                      Figure 1: Loops for EVPN routes

Rabadan, et al.           Expires 25 April 2024                 [Page 3]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

   Similar examples are possible with EVPN VPWS services on the Gateways
   and PEs, where loop prevention for the redistributed A-D per EVI
   routes is needed.  D-PATH provides the end to end path visibility
   that is required to prevent the loop.

1.2.  Add Path Visibility and Influence Best Path Selection for EVPN
      Routes

   Figure 2 illustrates another [RFC9014] EVPN Interconnect case where,
   in addition to using D-PATH to prevent EVPN MAC/IP Advertisement
   route loops when redistributing routes between domains, the D-PATH
   attribute can also influence the best path selection for the routes.
   For example, if all the Gateways in the diagram are attached to the
   same BD1, an EVPN MAC/IP Advertisement route for MAC address M1
   advertised by GW1 is advertised into Domain-1 and Domain-4.  Two
   routes for M1 will arrive at GW3 with different route distinguishers
   and BGP Next Hops.  If D-PATH is used by all the Gateways, the two
   routes arriving at GW3 will have a different sequence of domain-ids
   in the D-PATH attribute.  GW3 can use the length of the D-PATH as a
   way of influencing the selection (i.e., the shortest D-PATH route is
   selected).  D-PATH improves the path visibility of the route since it
   provides information about all the Domains through which the route
   has passed.

           +----------+ GW11  +----------+  GW2  +----------+
           | EVPN     +-------+ EVPN     +-------+ EVPN     |
           | Domain-1 | +---+ | Domain-2 | +---+ | Domain-3 |
           |          | |BD1| |          | |BD1| |          |
           |          | +---+ |          | +---+ |          |
      GW1  |          +-------+          +-------+          |  GW3
    +-------+         |       |          |       |         +-------+
    | +---+ |---------+       +----------+       +---------| +---+ |
    | |BD1| |                                              | |BD1| |
    | +---+ |---------+       +----------------------------| +---+ |
    +---|---+         | GW12  |                            +---|---+
        |  | EVPN     +-------+      EVPN                   |  |
    M1--+  | Domain-4 | +---+ |      Domain-5               |  +--M2
           |          | |BD1| |                             |
           |          | +---+ |                             |
           |          +-------+                             |
           +----------+       +-----------------------------+

          Figure 2: Influence Best Path Selection for EVPN routes

Rabadan, et al.           Expires 25 April 2024                 [Page 4]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

2.  Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

3.  Terminology

   This section summarizes the terminology that is used throughout the
   rest of the document.

   *  AC: Attachment Circuit or logical interface associated to a given
      Broadcast Domain.  To determine the AC on which a packet arrived,
      the PE examines the combination of a physical port and VLAN tags
      (where the VLAN tags can be individual c-tags, s-tags or ranges of
      both).

   *  BD and BT: a Broadcast Domain and Bridge Table, as defined in
      [I-D.ietf-bess-rfc7432bis].  A BD is a group of PEs attached to
      the same EVPN layer-2 multipoint service.  A BT is the
      instantiation of a Broadcast Domain in a PE.  When there is a
      single Broadcast Domain in a given EVI, the MAC-VRF in each PE
      contains a single BT.  When there are multiple BTs within the same
      MAC-VRF, each BT is associated to a different Ethernet Tag. The
      EVPN routes specific to a BT indicate which Ethernet Tag the route
      corresponds to.  This document does not distinguish between a
      "Broadcast Domain" and a "Bridge Table", and will use the terms
      interchangeably (or will use the acronym "BD" to refer to either).
      Also, the way the BDs are grouped into MAC-VRFs depending on the
      service model (VLAN-based versus VLAN Bundle or VLAN-aware Bundle)
      is not relevant to the procedures specified in this document.

   *  BUM: Broadcast, Unknown unicast and Multicast traffic.

   *  ES and ESI: Ethernet Segment and Ethernet Segment Identifier, as
      defined in [I-D.ietf-bess-rfc7432bis].

   *  EVPN Domain: two PEs are in the same EVPN Domain if they are
      attached to the same service and the packets between them do not
      require a data path lookup of the inner frame (e.g., in the BD of
      a MAC-VRF) in any intermediate router.  An EVPN Domain Gateway PE
      is always configured with multiple Domain identifiers (EVPN
      Domain-ID) in the MAC-VRF or VPWS that connects those EVPN
      Domains, each EVPN Domain-ID representing an EVPN Domain.

Rabadan, et al.           Expires 25 April 2024                 [Page 5]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

      Example: Figure 3 illustrates an example where PE1 and PE2 belong
      to different EVPN Domains since packets between them (for flows
      between hosts with MAC addresses M1 and M2) require a MAC lookup
      in two of the gateways that are connecting the three EVPN Domains.
      E.g., if frames from M1 to M2 go through PE1, GW1, GW3 and PE2, a
      MAC lookup is performed at GW1 and GW3.

                              GW1------------GW3
                            +------+       +------+
              +-------------| BD1  |       | BD1  |-------------+
             PE1            +------+       +------+            PE2
           +------+            |              |             +------+
        M1-| BD1  |   EVPN     |     EVPN     |     EVPN    |  BD1 |-M2
           +------+           GW2            GW4            +---+--+
              |             +------+       +------+             |
              +-------------| BD1  |       | BD1  |-------------+
                            +------+       +------+
                               +--------------+
               EVPN Domain 1     EVPN Domain 2  EVPN Domain 3
              <---------------> <------------> <---------------->

                  Figure 3: EVPN Domain Interconnect Example

   *  EVPN Domain Gateway: a PE where a service (BD or VPWS instance) is
      instantiated and is attached to two or more EVPN Domains.  This
      document uses the term "EVPN Domain Gateway" or simply "Gateway".
      An example of EVPN Domain Gateway is a PE following the procedures
      in section 4.4 or section 4.6 of [RFC9014].  In the example in
      Figure 3, GW1 and GW2 connect EVPN Domains 1 and 2, whereas GW3
      and GW4 connect EVPN Domains 2 and 3.  GW1 and GW2 import the MAC/
      IP Advertisement route for M1 coming from the EVPN Domain 1 into
      the MAC-VRF for BD1, and redistribute it into EVPN Domain 2.
      Likewise, GW3 and GW4 import the route into their MAC-VRF and re-
      advertise it into EVPN Domain 3.

   *  MAC-VRF: a MAC Virtual Routing and Forwarding table, as defined in
      [I-D.ietf-bess-rfc7432bis].  It is also the instantiation of an
      EVI (EVPN Instance) in a PE.

4.  Use of Domain Path Attribute (D-PATH) with EVPN routes

   This document extends the use of the D-PATH attribute specified in
   [I-D.ietf-bess-evpn-ipvpn-interworking] so that D-PATH can be
   advertised and processed along with the following EVPN route types:

   *  EVPN MAC/IP Advertisement routes that are not used for Inter-
      Subnet Forwarding (ISF).  Note that if the EVPN MAC/IP
      Advertisement route is used for Inter-Subnet Forwarding as in

Rabadan, et al.           Expires 25 April 2024                 [Page 6]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

      [RFC9135], the procedures for the D-PATH advertisement and
      processing are described in
      [I-D.ietf-bess-evpn-ipvpn-interworking].

   *  EVPN A-D per EVI routes that are used for EVPN VPWS [RFC8214].
      Advertised A-D per EVI routes not used for EVPN VPWS SHOULD NOT
      contain D-PATH.

   *  EVPN Inclusive Multicast Ethernet Tag routes
      [I-D.ietf-bess-rfc7432bis].

   As discussed, the use of D-PATH with EVPN IP Prefix routes is
   specified in [I-D.ietf-bess-evpn-ipvpn-interworking].  When used
   along with EVPN routes other than IP Prefix routes, the D-PATH
   attribute is characterized as follows:

   1.  D-PATH is composed of a sequence of Domain segments following the
       format specified in [I-D.ietf-bess-evpn-ipvpn-interworking] where
       each Domain is represented as <DOMAIN-ID:ISF_SAFI_TYPE>.  In this
       specification, DOMAIN-ID is an EVPN Domain identifier configured
       in an EVPN Domain Gateway and ISF_SAFI_TYPE is set to either 70
       (EVPN) or 0 (local route).  To simplify the explanation, this
       document represents the domains for EVPN routes as <Domain-
       ID:TYPE>.

   2.  D-PATH identifies the sequence of EVPN Domains the route has gone
       through, with the last <Domain-ID:TYPE> entry (rightmost)
       identifying the first PE or EVPN Domain Gateway that added the
       D-PATH attribute.

   3.  For non-Inter Subnet Forwarding EVPN MAC/IP Advertisement routes
       or EVPN A-D per EVI routes [I-D.ietf-bess-rfc7432bis], D-PATH
       SHOULD be added/modified by a EVPN Domain Gateway that
       redistributes the route between EVPN Domains and MAY be added by
       a PE or EVPN Domain Gateway that originates the route, as
       follows:

       a.  An EVPN Domain Gateway that connects two EVPN Domains "X" and
           "Y", and receives a route on a EVPN Domain identified by a
           Domain-ID "X" SHOULD append a domain <X:EVPN> to the existing
           (or newly added) D-PATH attribute when redistributing the
           route to EVPN Domain "Y".  The route is redistributed if it
           is first imported in a MAC-VRF (or VPWS instance), the MAC
           (or Ethernet Tag) is active, and policy allows the re-export
           of the route to a BGP neighbor.

Rabadan, et al.           Expires 25 April 2024                 [Page 7]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

       b.  An EVPN Domain Gateway MAY also add the D-PATH attribute for
           locally learned MACs or MAC/IP pairs.  In this case, the
           domain added would be <A:0>, where "A" is the Domain-ID
           configured on the Gateway's MAC-VRF that is specific to local
           routes (MAC/IP learned via local AC), and "0" is the TYPE of
           the EVPN Domain and indicates that the route is locally
           originated and not redistributed after receiving it from a
           BGP-EVPN neighbor.  The EVPN Domain-ID for local routes MAY
           be shared by all the EVPN Domain Gateways of the same
           redundancy group for local routes, or each EVPN Domain
           Gateway of the redundancy group can use its own Domain-ID.

       c.  A PE that is connected to a single EVPN Domain (therefore the
           PE is not an EVPN Domain Gateway) MAY add the D-PATH with a
           domain <B:0>, where "B" is the Domain-ID configured on the
           PE's MAC-VRF (or VPWS) for locally learned MAC/IPs (or
           Ethernet Tag IDs for VPWS). "0" is the TYPE that indicates
           the route is not re-advertised, but originated in the PE.

   4.  For EVPN Inclusive Multicast Ethernet Tag routes, an EVPN Domain
       Gateway must not redistribute routes between Domains as specified
       in [RFC9014].  An EVPN Domain Gateway originates an EVPN
       Inclusive Multicast Ethernet Tag route per Domain to which the
       Gateway is attached, so that BUM traffic can be attracted from
       one Domain to the rest.  Therefore, only the above point 3.b.
       applies to EVPN Domain Gateways.  That is, an EVPN Domain Gateway
       MAY add a <A:0> D-PATH attribute for the Inclusive Multicast
       Ethernet Tag routes generated for the EVPN Domains, where "A" is
       the configured local Domain-ID, and "0" is the TYPE that
       indicates the route is locally originated and not redistributed
       across EVPN Domains.  When two or more EVPN Domain Gateways of
       the same redundancy group connect two EVPN Domains "X" and "Y"
       and D-PATH is used for EVPN Inclusive Multicast Ethernet Tag
       routes, it is RECOMMENDED to add the D-PATH attribute with the
       same local Domain-ID and only on "X" or "Y" but not on both
       Domains.  In this case, all Gateways under the same redundancy
       group MUST choose the same Domain in which the D-PATH attribute
       is used.

   5.  On received EVPN routes, D-PATH is processed and used for loop
       detection (Section 4.4) as well as to influence the best path
       selection (Section 4.1, Section 4.2 and Section 4.3).

Rabadan, et al.           Expires 25 April 2024                 [Page 8]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

4.1.  D-PATH and Best Path Selection for MAC/IP Advertisement routes

   When two (or more) MAC/IP Advertisement routes with the same route
   key (and same or different RDs) are received, a best path selection
   algorithm is used to select and install only one route.  The best
   path selection for MAC/IP Advertisement routes is specified in
   [I-D.ietf-bess-rfc7432bis], in section 7.13.1, and this document
   modifies the algorithm by including the D-PATH comparison across EVPN
   MAC/IP Advertisement routes after tie-breaking rule 5 in
   [I-D.ietf-bess-rfc7432bis] section 7.13.1, which removes from
   consideration routes that are not tied for higher degree of
   preference.

   If none of the tie-breaking rules up to (and including) rule 5
   produces a single route, the router compares the D-PATH attribute in
   the remaining candidate routes:

   1.  The routes with the shortest D-PATH are preferred, hence routes
       not tied for the shortest D-PATH are removed.  Routes without
       D-PATH are considered zero-length D-PATH.

   2.  Then routes with the numerically lowest left-most Domain-ID are
       preferred, hence routes not tied for the numerically lowest left-
       most Domain-ID are removed from consideration.

   If the steps above do not produce a single route, then the rest of
   the rules in [I-D.ietf-bess-rfc7432bis] follow.

4.2.  D-PATH and Best Path Selection for Ethernet A-D per EVI routes

   When two (or more) EVPN A-D per EVI routes with the same route key
   (and same or different RDs) are received for a Virtual Private Wire
   Service (VPWS), a best path selection algorithm is used.  The best
   path selection for EVPN A-D per EVI routes is specified in
   [I-D.ietf-bess-rfc7432bis], in section 7.13.2, and this document
   modifies the algorithm by including the D-PATH comparison across EVPN
   A-D per EVI routes in the same way Section 4.1 does it for EVPN MAC/
   IP Advertisement routes.  That is, rules 1 and 2 of Section 4.1 are
   interleaved between rules 5 and 6 of [I-D.ietf-bess-rfc7432bis].

Rabadan, et al.           Expires 25 April 2024                 [Page 9]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

4.3.  D-PATH and Best Path Selection for Inclusive Multicast Ethernet
      Tag routes

   When two (or more) EVPN Inclusive Multicast Ethernet Tag routes with
   the same route key (and same or different RDs) are received for a
   MAC-VRF, a best path selection algorithm is used and only one of them
   is programmed.  The selection algorithm follows
   [I-D.ietf-bess-rfc7432bis] the same D-PATH comparison steps as in
   Section 4.1 interleaved between rules 5 and 6 of
   [I-D.ietf-bess-rfc7432bis].

4.4.  Loop Detection

   An EVPN route received by a PE with a D-PATH attribute that contains
   one or more of its locally associated Domain-IDs for the MAC-VRF or
   VPWS instance is considered to be a looped route.  A looped route
   MUST NOT be redistributed to a different domain and SHOULD be flagged
   as "looped".

   EVPN Inclusive Multicast Ethernet Tag looped routes MUST NOT be
   installed, where "install" in this document means "create forwarding
   state".  An EVPN MAC/IP Advertisement looped route or an A-D per EVI
   looped route (in EVPN VPWS services) MAY be installed if selected as
   the best route.

   For instance, in the example of Figure 3, assuming PE1 advertises
   M1's MAC/IP and does not add the D-PATH attribute, the EVPN Domain
   Gateway GW1 receives two MAC/IP Advertisement routes for M1's MAC/IP:

   *  A MAC/IP Advertisement route with next hop PE1 and no D-PATH.

   *  A MAC/IP Advertisement route with next hop GW2 and
      D-PATH={length=1,<6500:1:EVPN>}, assuming that the Domain-ID for
      EVPN Domain 1 is 6500:1.

   In this case, EVPN Domain Gateway GW1 flags the MAC/IP Advertisement
   route with D-PATH as "looped", and does not install the MAC in the
   BD, and does not redistribute the route back to EVPN Domain 1 (since
   the route includes one of GW1's Domain-IDs).  In case the MAC/IP
   Advertisement route with next hop PE1 is withdrawn, GW1 may install
   the route with next hop GW2 and D-PATH <6500:1:EVPN>; this may help
   speed up convergence in case of failures.

   The procedures described in this section, based on D-PATH, can be
   used along with the Ethernet Segment Identifier of the received
   routes as a way to detect looped routes on EVPN domain gateways
   attached to an Interconnect Ethernet Segment as in [RFC9014].  An
   EVPN domain gateway MUST NOT redistribute a received EVPN MAC/IP

Rabadan, et al.           Expires 25 April 2024                [Page 10]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

   route or EVPN A-D per EVI route with an Ethernet Segment Identifier
   value that matches the value of a local Ethernet Segment,
   irrespective of the D-PATH Domain-IDs.

4.5.  Error Handling

   The error handling for the D-PATH attribute is described in
   [I-D.ietf-bess-evpn-ipvpn-interworking].  This document extends the
   use of D-PATH to non-Inter Subnet Forwarding (non-ISF) EVPN routes.

5.  Use-Case Examples

   This section illustrates the use of D-PATH in EVPN routes with
   examples.

   Figure 4 and Figure 5 illustrate an integrated interconnect solution
   for an EVPN BD, as described in section 4.4 and section 4.6 of
   [RFC9014].  GW1 and GW2 are EVPN Domain Gateways connecting two EVPN
   Domains identified by D-PATH domain {1:1:EVPN} and {1:2:EVPN},
   respectively.  Received Ethernet A-D routes, ES routes, and Inclusive
   Multicast routes from the routers in one EVPN Domain are consumed and
   processed by GW1 and GW2, but not redistributed to the other EVPN
   Domain.  However, MAC/IP Advertisement routes received by GW1 and GW2
   in one EVPN Domain are processed and, if installed, redistributed
   into the other EVPN Domain.

            +----EVPN Domain-1---+      +----EVPN Domain-2--+
            |     1:1:EVPN       | GW1  |    1:2:EVPN       |
            |                   +---------+                 |
            |                   | +-----+ |                 |
            |                   | | BD1 | X <-+             |
           PE1                  | +-----+ |   |            PE2
        +---------+             +---------+   |         +---------+
        | +-----+ |              |      |     |         | +-----+ |
   M1-----| BD1 | |              |      |     |         | | BD1 |-----M2
        | +-----+ |  ------->    |      |     |         | +-----+ |
        +---------+ (RT2)M1/IP1  |      |     |         +---------+
            |                   +---------+   |             |
            |                   | +-----+ |   |(RT2)M1/IP1  |
            |                   | | BD1 | | --+ <1:1:EVPN>  |
            |                   | +-----+ |                 |
            |                   +---------+                 |
            |                     | GW2 |                   |
            +---------------------+     +-------------------+

                        Figure 4: EVPN Interconnect

Rabadan, et al.           Expires 25 April 2024                [Page 11]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

   Consider the example of Figure 4, where PE1 advertises a MAC/IP
   Advertisement route for M1/IP1.  The route is processed and installed
   by GW1 and GW2 in BD1, and both redistribute the routes into the EVPN
   Domain-2.  By using D-PATH in GW1 and GW2, when the route is received
   on PE2, PE2 has the visibility of the EVPN Domains through which the
   route has gone, and can also use the D-PATH for best path selection.
   In addition, GW1 and GW2 can compare the D-PATH of the incoming
   routes with their local list of EVPN Domain-IDs, and detect looped
   routes if any of the local EVPN Domain-IDs matches a domain in the
   received D-PATH.  This procedure prevents the redistribution of the
   route back into EVPN Domain-1.  For example, when GW1 receives the
   MAC/IP Advertisement route for M1/IP1 with D-PATH <1:1:EVPN>, GW1
   identifies the route as looped and it does not redistribute it back
   to Domain-1.  The M1/IP1 route with Next Hop PE1 is installed.  If
   M1/IP1 with Next Hop PE1 is withdrawn, GW1 MAY install the route M1/
   IP1 with Next Hop GW2, as specified in Section 4.4.

   The example of Figure 5 illustrates how GW1 and GW2 can also have
   local ACs in BD1 and learn local MAC (or MAC/IP) addresses from
   devices connected to the ACs.

            +----EVPN Domain-1---+      +----EVPN Domain-2--+
            |     1:1:EVPN       | GW1  |    1:2:EVPN       |
            |                   +---------+                 |
            |                   | +-----+ |                 |
            |              +-->X| | BD1 | |X<--+            |
           PE1             |    | +-----+ |    |           PE2
        +---------+        |    +---------+    |        +---------+
        | +-----+ |        |     |      |      |        | +-----+ |
   M1-----| BD1 | |        |     |      |      +--->    | | BD1 |-----M2
        | +-----+ |        |     |      |      |        | +-----+ |
        +---------+        |     |      |      |        +---------+
            |              |     | GW2  |      |            |
            |          <---+--  +---------+ (RT2)M3/IP3     |
            |       (RT2)M3/IP3 | +-----+ |  {1:3:0}        |
            |        {1:3:0}    | | BD1 | |    |            |
            |                   | +-----+ | ---+            |
            |                   +----|----+                 |
            |                     |  |  |                   |
            +---------------------+  |  +-------------------+
                                     +
                                     M3

                    Figure 5: EVPN Interconnect local AC

   Assuming GW2 learns M3/IP3 via local AC, GW2 advertises a MAC/IP
   Advertisement route for M3/IP3 into each of the EVPN Domains that GW2
   is connected to.  As described in Section 4, GW2 can advertise these

Rabadan, et al.           Expires 25 April 2024                [Page 12]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

   two MAC/IP Advertisement routes with a configured EVPN Domain-ID for
   local MAC/IPs routes that can be shared with GW1.  Consider this EVPN
   Domain-ID is 1:3 and it is configured on both, GW1 and GW2.  When GW2
   advertises the route into each EVPN Domain, it adds the D-PATH
   attribute with a domain {1:3:0}. These routes are flagged by GW1 as
   "looped" since 1:3 is configured as a local EVPN Domain-ID in GW1.
   In addition, PE1 and PE2 receive the routes with the D-PATH and they
   have the visibility of the origin of the routes, in this case local
   EVPN Domain Gateway routes.  This information can be used to
   influence the best path selection in case of multiple routes for M3/
   IP3 are received on PE1 or PE2 for BD1.

   As an alternative solution to configuring the same EVPN Domain-ID for
   local routes on both EVPN Domain Gateways, GW2 can be configured with
   EVPN Domain-ID 1:3 for local routes, and GW1 can use a different EVPN
   Domain-ID, e.g., 1:4.  In this case, GW2 advertises the route for M3/
   IP3 into each EVPN Domain as before, but now GW1 does not flag the
   route as "looped" since 1:3 is not on the list of GW1's local EVPN
   Domain-IDs.  GW1 receives the routes from both EVPN Domains, and GW1
   selects the route from e.g., EVPN Domain-1.  GW1 then installs the
   route in its BD and redistributes the route into EVPN Domain-2 with
   D-PATH {1:1:EVPN, 1:3:0}. When PE2 receives two routes for M3/IP3,
   one from GW2 with D-PATH {1:3:0} and another from GW1 with D-PATH
   {1:1:EVPN, 1:3:0}, PE2 uses best path selection and choose to send
   its traffic to GW2.  Also GW2 receives the route for M3/IP3 from GW1
   and mark it as "looped" since that route conveys its own EVPN Domain-
   IDs 1:1 and 1:3.

   In a nutshell, the use of D-PATH in MAC/IP Advertisement routes helps
   prevent loops and influences the best path selection so that PEs
   choose the shortest paths to the destination PEs.

6.  Security Considerations

   Most of the considerations included in
   [I-D.ietf-bess-evpn-ipvpn-interworking] apply to this document.

7.  IANA Considerations

   None.

8.  Acknowledgments

9.  Contributors

   In addition to the authors included on the front page, the following
   people contributed significantly:

Rabadan, et al.           Expires 25 April 2024                [Page 13]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

   Vinod Prabhu, Nokia

10.  References

10.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [I-D.ietf-bess-evpn-ipvpn-interworking]
              Rabadan, J., Sajassi, A., Rosen, E. C., Drake, J., Lin,
              W., Uttaro, J., and A. Simpson, "EVPN Interworking with
              IPVPN", Work in Progress, Internet-Draft, draft-ietf-bess-
              evpn-ipvpn-interworking-09, 9 October 2023,
              <https://datatracker.ietf.org/doc/html/draft-ietf-bess-
              evpn-ipvpn-interworking-09>.

   [I-D.ietf-bess-rfc7432bis]
              Sajassi, A., Burdet, L. A., Drake, J., and J. Rabadan,
              "BGP MPLS-Based Ethernet VPN", Work in Progress, Internet-
              Draft, draft-ietf-bess-rfc7432bis-07, 13 March 2023,
              <https://datatracker.ietf.org/doc/html/draft-ietf-bess-
              rfc7432bis-07>.

   [RFC9014]  Rabadan, J., Ed., Sathappan, S., Henderickx, W., Sajassi,
              A., and J. Drake, "Interconnect Solution for Ethernet VPN
              (EVPN) Overlay Networks", RFC 9014, DOI 10.17487/RFC9014,
              May 2021, <https://www.rfc-editor.org/info/rfc9014>.

   [RFC8214]  Boutros, S., Sajassi, A., Salam, S., Drake, J., and J.
              Rabadan, "Virtual Private Wire Service Support in Ethernet
              VPN", RFC 8214, DOI 10.17487/RFC8214, August 2017,
              <https://www.rfc-editor.org/info/rfc8214>.

10.2.  Informative References

   [RFC9135]  Sajassi, A., Salam, S., Thoria, S., Drake, J., and J.
              Rabadan, "Integrated Routing and Bridging in Ethernet VPN
              (EVPN)", RFC 9135, DOI 10.17487/RFC9135, October 2021,
              <https://www.rfc-editor.org/info/rfc9135>.

Rabadan, et al.           Expires 25 April 2024                [Page 14]
Internet-Draft           D-PATH for Layer2 EVPN             October 2023

Authors' Addresses

   J. Rabadan (editor)
   Nokia
   520 Almanor Avenue
   Sunnyvale, CA 94085
   United States of America
   Email: jorge.rabadan@nokia.com

   S. Sathappan
   Nokia
   520 Almanor Avenue
   Sunnyvale, CA 94085
   United States of America
   Email: senthil.sathappan@nokia.com

   M. Gautam
   Nokia
   520 Almanor Avenue
   Sunnyvale, CA 94085
   United States of America
   Email: mallika.gautam@nokia.com

   P. Brissette
   Cisco Systems
   Canada
   Email: pbrisset@cisco.com

   W. Lin
   Juniper
   United States of America
   Email: wlin@juniper.net

   B. Wen
   Comcast
   United States of America
   Email: bin_wen@comcast.com

Rabadan, et al.           Expires 25 April 2024                [Page 15]