BESS WG                                                          Y. Wang
Internet-Draft                                                  Z. Zhang
Intended status: Standards Track                         ZTE Corporation
Expires: September 17, 2020                               March 16, 2020


              ARP/ND Synching And IP Aliasing without IRB
            draft-wang-bess-evpn-arp-nd-synch-without-irb-03

Abstract

   This draft proposes an extension to [RFC7432] to do ARP synchronizing
   and IP aliasing for Layer 3 routes that is needed for EVPN signalled
   L3VPN to build a complete IP ECMP.  The phrase "EVPN signalled L3VPN"
   means that there may be no MAC-VRF or IRB interface in the use case.
   When there are no MAC-VRF or IRB interface, EVPN signalled L3VPN is
   also called as pure L3VPN instance.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 17, 2020.

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of




Wang & Zhang           Expires September 17, 2020               [Page 1]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   4
   2.  ARP/ND Synching and IP Aliasing . . . . . . . . . . . . . . .   4
     2.1.  Constructing MAC/IP Advertisement Route . . . . . . . . .   5
     2.2.  Constructing EAD/IP-VRF Route . . . . . . . . . . . . . .   6
     2.3.  Constructing EAD/ES Route . . . . . . . . . . . . . . . .   6
   3.  Fast Convergence for Routed Traffic . . . . . . . . . . . . .   7
   4.  Determining Reach-ability to Unicast IP Addresses . . . . . .   7
   5.  Forwarding Unicast Packets  . . . . . . . . . . . . . . . . .   7
   6.  EVPN signalled L3VPN  . . . . . . . . . . . . . . . . . . . .   8
     6.1.  RT-5E Advertisement on Distributed L3 GW  . . . . . . . .   8
     6.2.  Centerlized RT-5G Advertisement for Distributed L3
           Forwarding  . . . . . . . . . . . . . . . . . . . . . . .   8
       6.2.1.  Centerlized CE-BGP  . . . . . . . . . . . . . . . . .   9
       6.2.2.  RT-2E Advertisement from PE1/PE2 to PE3 . . . . . . .  10
       6.2.3.  RT-5G Advertisement from PE3 to PE1/PE2 . . . . . . .  10
       6.2.4.  RT-2E Advertisement between PE1 and PE2 . . . . . . .  11
       6.2.5.  Egress ESI Link Protection between PE1 and PE2  . . .  11
       6.2.6.  Comparing with Distributed RT-5G Advertisement  . . .  11
       6.2.7.  Mass-Withdraw by EAD/ES Route . . . . . . . . . . . .  11
     6.3.  RT-5L Advertisement . . . . . . . . . . . . . . . . . . .  12
   7.  Load Balancing of Unicast Packets . . . . . . . . . . . . . .  12
   8.  Security Considerations . . . . . . . . . . . . . . . . . . .  12
   9.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  12
   10. Normative References  . . . . . . . . . . . . . . . . . . . .  13
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  13

1.  Introduction

   [I-D.sajassi-bess-evpn-ip-aliasing] proposes an extension to
   [RFC7432] to do aliasing for Layer 3 routes that is needed for
   symmetric IRB to build a complete IP ECMP.  But typically there may
   be both IRB interfaces(to do EVPN IRB per-MAC-VRF basis) and VRF-
   interfaces in the same IP-VRF instance.  It is necessary to apply the
   EVPN control-plane to the VRF-interfaces in order to support both
   such situations and the EVPN signalled L3VPN use case where maybe no
   IRB interfaces will be found in the IP-VRF instances.









Wang & Zhang           Expires September 17, 2020               [Page 2]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


                                      +---------+
                   +-------------+    |         |
                   |             |    |         |
                  /|    PE1      |----|         |   +-------------+
                 / |             |    |  MPLS/  |   |             |
            LAG /  +-------------+    |  VxLAN/ |   |     PE3     |---N3
   N1---SW1=====                      |  NVGRE/ |   |             |
       /        \  +-------------+    |  SRv6   |---|             |
     N2          \ |             |    |         |   +-------------+
                  \|     PE2     |----|         |
                   |             |    |         |
                   +-------------+    |         |
                                      |         |
                                      |         |
                                      +---------+

        Figure 1: ARP/ND Synchronizing and IP Aliasing without IRB

   There three CE node named N1/N2/N3 in the above network.  N1/N2/N3
   may be a host or a IP router.  When N1/N2/N3 is a host, it is also
   called H1/H2/H3 in this document.  When N1/N2/N3 is a router, it is
   also called R1/R2/R3 in this document.

   Consider a pair of multi-homed PEs PE1 and PE2.  Let there be two
   hosts H1 and H2 attached to them via a L2 switch SW1.  Consider
   another PE PE3 and a host H3 attached to it.  The H1 and H2 represent
   subnet SN1 and the H3 represents subnet SN2.

   Note that it is different from [I-D.sajassi-bess-evpn-ip-aliasing] in
   the following aspects: There is no MAC-VRF or IRB interface on
   PE1/PE2/PE3.  And it is the IP-VRFs that are called as EVPN instance
   instead.  Such EVPN instance can be called pure L3 EVPN instance or
   L3 EVI for short.  The anycast gateway of H1/H2 is configured on a
   sub-interface on PE1/PE2.

   Note that the communication between H1 and H2 won't pass through any
   of the multi-homed PEs.  So it is not necessary for PE1/PE2 keeping a
   Broadcast domain and its IRB for SN1.

   Note that the SW1 multi-homing PE1 and PE2 via a LAG interface which
   maybe load-balance traffic to the PEs.

   This draft proposes an extension to do ARP/ND synchronizing and IP
   aliasing for Layer 3 routes that is needed for L3 EVI to build a
   complete IP ECMP.






Wang & Zhang           Expires September 17, 2020               [Page 3]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


1.1.  Terminology

   Most of the terminology used in this documents comes from [RFC7432]
   and [I-D.sajassi-bess-evpn-ip-aliasing] except for the following:

   VRF Interface: A interface that connects to a CE for an IP-VRF but is
   not an IRB interface.

   L3 EVI: An EVPN instance spanning the Provider Edge (PE) devices
   participating in that EVPN which contains VRF Interfaces and maybe
   contains IRB interfaces.

   EAD/IP-VRF: Ethernet Auto-Discovery route per IP-VRF, which
   differentiates itself from IP-EAD/EVI route by the MPLS label field.

   CE-BGP: The BGP session between PE and CE.  Note that CE-BGP route
   doesn't have a RD or Route-Target.

   RMAC: Router's MAC, which is signaled in the Router's MAC extended
   community.

   RT-2E: A MAC/IP Advertisement Route with a non-reserved ESI.

   RT-5E: An EVPN Prefix Advertisement Route with a non-reserved ESI.

   RT-5G: An EVPN Prefix Advertisement Route with a zero ESI and a non-
   zero GW-IP.

   RT-5L: An EVPN Prefix Advertisement Route with both zero ESI and zero
   GW-IP.

2.  ARP/ND Synching and IP Aliasing

   Host IP and MAC routes are learnt by PEs on the access side via a
   control plane protocol like ARP.  In case where a CE is multihomed to
   multiple PE nodes using a LAG and is running in All-Active Redundancy
   Mode, the Host IP will be learnt and advertised in the MAC/IP
   Advertisement only by the PE that receives the ARP packet.  The MAC/
   IP Advertisement with non-zero ESI will be received by both PE2 and
   PE3.

   As a result, after PE2 receives the MAC/IP Advertisement and imports
   it to the L3 EVI, PE2 installs an ARP entry to the VRF interface
   whose subnet matches the IP Address from the MAC/IP Advertisement.
   Such ARP entry is called remote synched ARP Entry in this document.






Wang & Zhang           Expires September 17, 2020               [Page 4]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


   Note that the PEs follow [I-D.sajassi-bess-evpn-ip-aliasing] to
   achieve the ESI load balance except for the constructing of MAC/IP
   Advertisement Route and IP-EAD/EVI route.

   When PE3 load balance the traffic towards the multihomed Ethernet
   Segment, both PE1 and PE2 would have been prepared with corresponding
   ARP entry yet because of the ARP synching procedures.

   It is important to explain that typically there may be both IRB
   interface and VRF interface in an IP-VRF instance, which is called as
   the "VRF interface in EVPN IRB" use-case in this document.  But each
   IRB/VRF interface is independent to each other in EVPN control plane.
   So the use-case here is constrained to a pure L3 EVPN schema, Because
   it is enough to describe all the control-plane updates for both the
   pure L3 EVPN use-case and the "VRF interface in EVPN IRB" use-case.

   In current EVPN control-plane for "VRF interface in EVPN IRB" use-
   case, the VRF interface is considered as "external link" and it just
   inter-operates with the EVPN control-plane.  But in this document it
   is assumed to be better if the EVPN control-plane directly applied to
   the VRF interface.

2.1.  Constructing MAC/IP Advertisement Route

   This draft introduces a new usage/construction of MAC/IP
   Advertisement route to enable Aliasing for IP addresses in pure L3
   EVPN use-cases.  The usage/construction of this route remains similar
   to that described in RFC 7432 with a few notable exceptions as below.

   * The Route-Distinguisher should be set to the corresponding L3VPN
   context.

   *  The Ethernet Tag should be set to 0.

   * The MAC/IP Advertisement SHOULD carry one or more IP VRF Route-
   Target (RT) attributes.

   * The ESI SHOULD be set to the ESI of the VRF interface from which
   the ARP entry is learned.

   Note that the ESI is not used to install remote synched ARP entries
   to corresponding VRF interfaces on PE1/PE2.  It is only used to load
   balance traffic on PE3.

   * The MPLS Label1 should be set to implicit-null in MPLS/SRv6
   encapsulation.  For VXLAN encapsulation, the MPLS label1 should be
   set to 0 instead.




Wang & Zhang           Expires September 17, 2020               [Page 5]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


   Note that there may be no MAC-VRF here, and this is outside the scope
   of RFC 7432.

   * The MPLS Label2 should be set to the local label of the IP-VRF in
   MPLS or VXLAN EVPN.  But it should be set to implicit-null in SRv6
   EVPN.

   Note that the label may be VNI label or MPLS label.

   Note that in SRv6 EVPN an SRv6 L3 Service TLV MAY also be advertised
   along with the route following [I-D.dawra-bess-srv6-services].  But
   SRv6 L2 Service TLV won't be advertiseed along with the route.
   Because that no MAC-VRF exists in the use case.

   * The RMAC Extended Community attribute SHOULD be carried in VXLAN
   EVPN.

2.2.  Constructing EAD/IP-VRF Route

   Note that the MAC/IP Advertisement is used for two reasons.  It is
   used between PE1 and PE2 to synch the ARP entries to each other.  It
   is used between PE1/PE2 and PE3 to achieve the load balance to ES
   adjacent PEs.

   The usage/construction of this route is similar to the IP-EAD/EVI
   route described in [I-D.sajassi-bess-evpn-ip-aliasing] with a few
   notable exceptions as below.

   * The MPLS Label should be set to the local label of the IP-VRF in
   MPLS EVPN or VXLAN EVPN.  But it should be set to implicit-null in
   SRv6 EVPN.

   Note that there may be no MAC-VRF here, and this is outside the scope
   of [RFC7432] .

   Note that in SRv6 EVPN an L3 Service SID MAY also be advertised along
   with the route following [I-D.dawra-bess-srv6-services].

   Such Ethernet Auto-Discovery route is called Ethernet Auto-Discvoery
   route per IP-VRF which is abbreviated as EAD/IP-VRF in this document.

2.3.  Constructing EAD/ES Route

   The usage/construction of this route remains similar to that
   described in section 3.1.1. of [I-D.sajassi-bess-evpn-ip-aliasing]
   with a few notable exceptions as explained as below.

   There may be no MAC-VRF RTs in the EAD/ES Route.



Wang & Zhang           Expires September 17, 2020               [Page 6]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


3.  Fast Convergence for Routed Traffic

   The procedures for Fast Convergence do not change from
   [I-D.sajassi-bess-evpn-ip-aliasing] except for a few notable
   exceptions as explained as below.

   The local ARP entries and remote synced ARP entries is installed/
   learned on a VRF interface rather than an IRB interface.

   There is no MAC entry.

4.  Determining Reach-ability to Unicast IP Addresses

   The procedures for local/remote host learning and MAC/IP
   Advertisement route constructing are described above.  The procedures
   for Route Resolution do not change from [I-D.sajassi-bess-evpn-ip-
   aliasing].

5.  Forwarding Unicast Packets

   Because of the nature of the MPLS label or SRv6 SID for IP-VRF
   instance, when these EAD/IP-VRF routes are referred in IP-VRF routing
   and forwarding procedures, the inner ethernet headers are absent on
   the corresponding packets transported following these EAD/IP-VRF
   routes.

   Note that in [I-D.sajassi-bess-evpn-ip-aliasing] the inner ethernet
   header need to be included in the packets which are sent from IP-VRF
   following IP-EAD/EVI routes, becuase that the MPLS label of such IP-
   EAD/EVI route is for MAC-VRF, not for IP-VRF.  and the inner
   destination MAC of these packets is following the "Router's MAC"
   extended community of MAC/IP advertisement routes with non-zero ESI.
   But in many use cases, the RMAC is not the same as the IRB
   interface's own MAC and the RMAC is not the same among different PEs.
   For example, the MAC address of the two IRB interfaces with anycast
   GW-IP address will be the same, but these two IRB interfaces lies on
   two different GW node and their "Router's MAC" is typically not the
   same.  In these use cases, it is recommanded to use EAD/IP-VRF route
   instead, even if there is indeed a MAC-VRF instance.

   Note that [I-D.sajassi-bess-evpn-ip-aliasing] suggests to carry RMAC
   in the Ethernet A-D per EVI routes in its version 01.  This can be
   used to solve the problem.

   Note that in EVPN IRB use cases, the EAD/IP-VRF route is more
   accordant with the symmetric IRB concept in the sense of data-plane
   behavior for unicast packets than the IP-EAD/EVI route of
   [I-D.sajassi-bess-evpn-ip-aliasing].



Wang & Zhang           Expires September 17, 2020               [Page 7]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


   Note that from the viewpoint of the route receiver It is impossible
   to distinguish the EAD/IP-VRF route from the IP-EAD/EVI route.  So
   the receiver has to configure how to interprete the remote EAD/EVI
   route.  If it is interpreted as EAD/IP-VRF route, the corresponding
   transported unicast packets will not be inserted with an ethernet
   header.  But if it is interpreted as IP-EAD/EVI route, they will.
   Note that we will rely on such configuration only in MPLS/SRv6 EVPN,
   it is not needed in VXLAN EVPN.

6.  EVPN signalled L3VPN

   EVPN signalled L3VPN can be deployed without EVPN IRB like what MPLS/
   BGP VPNs have done for a long time, but it is optional for it to be
   combined with EVPN IRB.  The EVPN siganlled L3VPN without EVPN IRB is
   not well defined yet, so we take the non-IRB usecase as an example.
   But the following routes and procedures can be used in EVPN IRB
   usecase too.  Note that in EVPN IRB usecase, the IRB interfaces are
   VRF-interface too.

6.1.  RT-5E Advertisement on Distributed L3 GW

   Given that PE1/PE2 can receive remote synced ARP entries from each
   other by RT-2 route following section 2.1.  So it is not necessary
   for PE1/PE2 to advertise per-host IP prefixes by RT-2 routes.  It is
   recommended that PE1/PE2 advertise an RT-5 route per subnet to PE3
   instead.  The ESI of these RT-5E routes can be set to the ESI of the
   corresponding VRF interface.  If the VRF interface fails, these
   subnets will achieve more faster convergency on PE3 by the withdraw
   of the corresponding EAD/IP-VRF route.

   Note that N1/N2 may be a host or a router, when it is a router, those
   subnets will be the subnets behind it.  When N1 and N2 are hosts,
   those subnets will be the subnets of N1 and N2 which are from
   different subnets.

6.2.  Centerlized RT-5G Advertisement for Distributed L3 Forwarding

   When N1/N2/N3 is a router, it is called R1/R2/R3 in the following
   figure.  Note that figure 1 only illustrates the physical ethernet
   links, but figure 2 illustrates the logical L3 adjacencies betweent
   PE and CE as the following.










Wang & Zhang           Expires September 17, 2020               [Page 8]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


                        PE2
   +----+         +---------------+
   |    | 20.2    | 20.1 +------+ |  ------>
   | R2 |===+------------|      | |  RT-2E
   |    |   |     |      |IPVRF1| |  20.2                 PE3
   +----+   |  +---------|      | |  ESI1          +---------------+
   Prefix2  |  |  | 10.1 +------+ |                |               |
            |  |  +---------------+                | +-----------+ |
            |  |          ^                        | | IPVRF1    | |
            |  |          | RT-2E       <--------  | |           |----R3
            |  |  ESI1    | 10.2         RT-5E     | | 3.3.3.3   | |
            |  |          | ESI1         Prefix1   | +-----------+ |
            |  |          |              ESI1      |   ^           |
            |  |  +---------------+                |   |           |
   Prefix1  |  |  | 20.1 +------+ |                +---|-----------+
   +----+   +--|---------|      | |                    |
   |    |      |  |      |IPVRF1| |                    |
   | R1 |======+---------|      | |  ------>           |
   |    | 10.2    | 10.1 +------+ |  RT-2E             |
   +----+         +---------------+  10.2              | CE-BGP
     |                  PE1          ESI1              | Prefix1
     |                                                 | NH=10.2
     |                      CE-BGP                     |
     +------------------------>------------------------+

                 Figure 2: Centerlized RT-5G Advertisement

   Note that R1/R2 should establish CE-BGP session with both PE1 and PE2
   in case of one of them fails, PE1 and PE2 will advertise RT-5E route
   to PE3 for their prefixes learned from CE-BGP independently.  If R1/
   R2 prefers to establish a single CE-BGP session, it can establish the
   CE-BGP session with PE3 instead.  This CE-BGP session can be called
   the centerlized CE-BGP session.  But when we use centerlized CE-BGP
   session, we should use RT-5G route instead.

   Note that we just use centerlized CE-BGP session to do route
   advertisement, but we still expect a distributed Layer 3 forwarding
   framework.

6.2.1.  Centerlized CE-BGP

   The CE-BGP session between R1 and PE3 is established between 10.2 and
   3.3.3.3.  The CE-BGP session between R2 and PE3 is established
   between 20.2 and 3.3.3.3.  The IP address 10.2/20.2 is called the
   uplink interface address of R1/R2 in this document.  The IP address
   3.3.3.3 is called the centerlized loopback address of IPVRF1 in this
   document.  The IP address 10.1/20.1 is called the downlink interface
   address of PE1/PE2 in this document.



Wang & Zhang           Expires September 17, 2020               [Page 9]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


   Note that the downlink interface is a Layer 3 link and it needn't
   attach an BD.

   R1 advertises a BGP route for a prefix (say "Prefix1") behind it to
   PE3 via that CE-BGP session.  The nexthop for Prefix1 is R1's uplink
   interface address (say 10.2).

   The route advertisement of R2 is similar to the above advertisement.

   Note that the packets from R1/R2 to the centerlized loopback address
   may be routed following the default route on R1/R2.

6.2.2.  RT-2E Advertisement from PE1/PE2 to PE3

   When PE1 learns the ARP entry of 10.2, it advertises a RT-2E route to
   PE3.  The ESI value of the RT-2E route is ESI1, which is the ESI of
   PE1's downlink interface.  The RT-2E route is constructed following
   section 2.1.

   Note that in [RFC7432], when the ESI is single-active, the MAC
   forwarding only use the label and MPLS nexthop of the RT-2E route.
   But in IP forwarding we assume that the ESI is preferred even if the
   ESI is single-active.  This is similar to
   [I-D.ietf-bess-evpn-prefix-advertisement].  The ESI usage in IP
   forwarding is out of the [RFC7432]'s scope.

   The RT-2E route advertisement of PE2 is similar to the above
   advertisement.

6.2.3.  RT-5G Advertisement from PE3 to PE1/PE2

   When PE3 receives the prefix1 from the CE-BGP session.  The nexthop
   for Prefix1 is 10.2, and the ESI for 10.2 is ESI1.  So PE3 advertises
   a RT-5G route to PE1/PE2 for Prefix1.  The GW-IP value of the RT-5G
   route for Prefix1 is 10.2.

   Note that PE3 can load-balance packets for Prefix1 via the EAD/IP-VRF
   routes from PE1/PE2.  Because ESI1 is the ESI for Prefix1's GW-IP.

   The RT-5 route advertisement and packet forwarding for Prefix2 is
   similar to the above.

   Note that the centerlized loopback address is advertised by PE3 via
   RT-5L route.  The nexthop of the RT-5L route is PE3, and the GW-IP
   value of the RT-5L route is zero.  The label of the RT-5L route is
   IPVRF1's label on PE3.  The RMAC of the RT-5L route is PE3's MAC when
   the encapsulation is VXLAN.




Wang & Zhang           Expires September 17, 2020              [Page 10]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


6.2.4.  RT-2E Advertisement between PE1 and PE2

   The RT-2E routes advertisement between PE1 and PE2 is used to sync
   these ARP entries to each other in order to avoid ARP missing.  The
   ESI Value of these two RT-2E routes is ESI1.

   Note that the ARP entry for 10.2 will be learned on PE1 only, and
   20.2 will be learned on PE2 only.  Note that the two downlink
   interfaces on PE1/PE2 are sub-interfaces of the same physical
   interface.  So they have the same ESI.

6.2.5.  Egress ESI Link Protection between PE1 and PE2

   The EAD/IP-VRF routes between PE1 and PE2 is used to do egress link
   protection.  The egress link protection follows the second approach
   of the [RFC8679] section 6.

   Note that although the ARP entry for 10.2 on PE2 is synced from PE1
   via RT-2E route.  The ARP entry on PE2 is installed to forward
   packets directly to the corresponding downlink interface primarily.
   The bypass tunnel following the EAD/IP-VRF route is only activated
   when the downlink interface fails.

6.2.6.  Comparing with Distributed RT-5G Advertisement

   When R1/R2 establish CE-BGP sessions with both PE1 and PE2, The RT-5G
   routes can be used by PE1/PE2.  But when R1 only establish just a
   single CE-BGP session with PE1, there will be some trouble when PE1
   fails.  Even if PE2/PE3 applies a delayed deletion when PE1 fails,
   the delay cann't be long enough when PE1 never comes up again.

   Note that when there is only a single CE-BGP session, the RT-5E
   advertisement will face the same fact.  In fact it is even worse when
   R1 uses different subnets to connect to PE1 and PE2 as described in
   [I-D.sajassi-bess-evpn-ip-aliasing].  Because that RT-5E can only
   sync the prefixes, it can't sync the nexthops, so the ARP entry for
   the other uplink interface that connects PE2 and R1 will not be
   resolved.

   Note that when R1 uses different subnets to connect to PE1 and PE2 ,
   it is not necessary to configure a BD for the two subnets connecting
   PE and CE as described in [I-D.sajassi-bess-evpn-ip-aliasing].

6.2.7.  Mass-Withdraw by EAD/ES Route

   We can assume that R1 and R2 are attached to different IP-VRFs(say
   IPVRF1 and IPVRF2 respectively), and the physical interface of the
   downlink interfaces on PE1 fails, PE1 will withdraw the EAD/ES route



Wang & Zhang           Expires September 17, 2020              [Page 11]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


   of ESI1, so PE3 will re-route 10.2 for Prefix1 and 20.2 for Prefix2.
   Then data packets for Prefix1 and Prefix2 will be sent to PE2
   instead.

6.3.  RT-5L Advertisement

   When R1/R2 establish CE-BGP sessions with both PE1 and PE2, it is
   enough for PE1/PE2 to advertise RT-5L routes to PE3.  There is no
   need for RT-5G or RT-5E advertisement on PE1/PE2 in that usecase.

   Note that when R1/R2 establish CE-BGP sessions with both PE1 and PE2,
   the downlink interface addresses on PE1 and PE2 may be different IP
   addresses of the same subnet.

   Note that when centerlized CE-BGP session is used, the prefixes from
   R3 and the local loopback addresses on PE3 are advertised to PE1/PE2
   using RT-5L too.

7.  Load Balancing of Unicast Packets

   It is similar to [I-D.sajassi-bess-evpn-ip-aliasing] except for a few
   notable exceptions as explained in section 6.1.3 and the following.

   Note that when the encapsulation is VXLAN, PE3 will encapsulate the
   RMAC of the RT-2E route for corresponding GW-IP address.  And the
   RMAC of PE1 should have the same value with the RMAC of PE2.  This
   can be achieved by configuration.  When a IP packet is encapsulated
   with a VNI label according to an EAD/IP-VRF route, the packet SHOULD
   be encapsulated with a Destination-MAC according to the RMAC of the
   same EAD/IP-VRF route.

   Note that PE1/PE2 just do egress link protection following EAD/IP-VRF
   and EAD/ES route.  Even if ESI1 is configured as all-active ESI, PE1/
   PE2 will not load-balance between local downlink interface and the
   bypass tunnel.  The downlink interfaces will always have more higher
   priority than the bypass tunnel.

8.  Security Considerations

   This document does not introduce any new security considerations
   other than already discussed in [RFC7432] and [RFC8365].

9.  IANA Considerations

   There is no IANA consideration.






Wang & Zhang           Expires September 17, 2020              [Page 12]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


10.  Normative References

   [I-D.dawra-bess-srv6-services]
              Dawra, G., Filsfils, C., Brissette, P., Agrawal, S.,
              Leddy, J., daniel.voyer@bell.ca, d.,
              daniel.bernier@bell.ca, d., Steinberg, D., Raszuk, R.,
              Decraene, B., Matsushima, S., Zhuang, S., and J. Rabadan,
              "SRv6 BGP based Overlay services", draft-dawra-bess-
              srv6-services-02 (work in progress), July 2019.

   [I-D.ietf-bess-evpn-prefix-advertisement]
              Rabadan, J., Henderickx, W., Drake, J., Lin, W., and A.
              Sajassi, "IP Prefix Advertisement in EVPN", draft-ietf-
              bess-evpn-prefix-advertisement-11 (work in progress), May
              2018.

   [I-D.sajassi-bess-evpn-ip-aliasing]
              Sajassi, A., Badoni, G., Warade, P., Pasupula, S., Drake,
              J., and J. Rabadan, "L3 Aliasing and Mass Withdrawal
              Support for EVPN", draft-sajassi-bess-evpn-ip-aliasing-01
              (work in progress), March 2020.

   [RFC7432]  Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
              Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
              Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
              2015, <https://www.rfc-editor.org/info/rfc7432>.

   [RFC8365]  Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R.,
              Uttaro, J., and W. Henderickx, "A Network Virtualization
              Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365,
              DOI 10.17487/RFC8365, March 2018,
              <https://www.rfc-editor.org/info/rfc8365>.

   [RFC8679]  Shen, Y., Jeganathan, M., Decraene, B., Gredler, H.,
              Michel, C., and H. Chen, "MPLS Egress Protection
              Framework", RFC 8679, DOI 10.17487/RFC8679, December 2019,
              <https://www.rfc-editor.org/info/rfc8679>.

Authors' Addresses

   Yubao(Bob) Wang
   ZTE Corporation
   No. 50 Software Ave, Yuhuatai Distinct
   Nanjing
   China

   Email: yubao.wang2008@hotmail.com




Wang & Zhang           Expires September 17, 2020              [Page 13]


Internet-Draft          EVPN ARP/ND Synch no IRB              March 2020


   Zheng(Sandy) Zhang
   ZTE Corporation
   No. 50 Software Ave, Yuhuatai Distinct
   Nanjing
   China

   Email: zzhang_ietf@hotmail.com












































Wang & Zhang           Expires September 17, 2020              [Page 14]