L2VPN Workgroup                                               J. Rabadan
Internet Draft                                              S. Sathappan
Intended status: Standards Track                           W. Henderickx
                                                         S. Palislamovic
                                                          Alcatel-Lucent

                                                                F. Balus
                                                          Nuage Networks


Expires: August 18, 2014                               February 14, 2014



      Data Center Interconnect Solution for EVPN Overlay networks
              draft-rabadan-l2vpn-dci-evpn-overlay-01.txt

Abstract

   This document describes how Network Virtualization Overlay networks
   (NVO3) can be connected to a Wide Area Network (WAN) in order to
   extend the layer-2 connectivity required for some tenants. The
   solution will analyze the interaction between NVO3 networks running
   EVPN and other L2VPN technologies used in the WAN, such as VPLS/PBB-
   VPLS or EVPN/PBB-EVPN, and will propose a solution for the
   interworking between both.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html




Rabadan et al.          Expires August 18, 2014                 [Page 1]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


   This Internet-Draft will expire on August 18, 2014.

Copyright Notice

   Copyright (c) 2014 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.


Table of Contents

   1. Introduction  . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2. Decoupled DCI solution for EVPN overlay networks  . . . . . . .  3
     2.1. Interconnect requirements . . . . . . . . . . . . . . . . .  4
     2.2. VLAN-based hand-off . . . . . . . . . . . . . . . . . . . .  5
     2.3. Pseudowire-based hand-off . . . . . . . . . . . . . . . . .  5
     2.4. Multi-homing solution . . . . . . . . . . . . . . . . . . .  6
     2.5. Data Center Gateway Optimizations . . . . . . . . . . . . .  7
       2.5.1 Use of the Unknown MAC route to reduce unknown
             flooding . . . . . . . . . . . . . . . . . . . . . . . .  7
       2.5.2. MAC address advertisement control . . . . . . . . . . .  7
       2.5.3. ARP flooding control  . . . . . . . . . . . . . . . . .  8
   3. Integrated DCI solution for EVPN overlay networks . . . . . . .  8
     3.1. Interconnect requirements . . . . . . . . . . . . . . . . .  9
     3.2. VPLS DCI for EVPN-Overlay networks  . . . . . . . . . . . . 10
       3.2.1. Control/Data Plane setup procedures on the DC GWs . . . 10
       3.2.2. Multi-homing procedures on the DC GWs . . . . . . . . . 11
     3.3. PBB-VPLS DCI for EVPN-Overlay networks  . . . . . . . . . . 11
       3.3.1. Control/Data Plane setup procedures on the DC GWs . . . 11
       3.3.2. Multi-homing procedures on the DC GWs . . . . . . . . . 12
     3.4. EVPN-MPLS DCI for EVPN-Overlay networks . . . . . . . . . . 12
       3.4.1. Control Plane setup procedures on the DC GWs  . . . . . 12
       3.4.2. Data Plane setup procedures on the DC GWs . . . . . . . 14
       3.4.3. Multi-homing procedures on the DC GWs . . . . . . . . . 14
       3.4.4. Impact on MAC Mobility procedures . . . . . . . . . . . 15
       3.4.5. Data Center Gateway optimizations . . . . . . . . . . . 16
       3.4.6. Benefits of the EVPN-MPLS DCI solution  . . . . . . . . 16
     3.5. PBB-EVPN DCI for EVPN-Overlay networks  . . . . . . . . . . 17
       3.5.1. Control/Data Plane setup procedures on the DC GWs . . . 17



Rabadan et al.          Expires August 18, 2014                 [Page 2]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


       3.5.2. Multi-homing procedures on the DC GWs . . . . . . . . . 18
       3.5.3. Impact on MAC Mobility procedures . . . . . . . . . . . 18
       3.5.4. Data Center Gateway optimizations . . . . . . . . . . . 18
   5. Conventions and Terminology . . . . . . . . . . . . . . . . . . 18
   6. Security Considerations . . . . . . . . . . . . . . . . . . . . 19
   7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 19
   8. References  . . . . . . . . . . . . . . . . . . . . . . . . . . 19
     8.1. Normative References  . . . . . . . . . . . . . . . . . . . 19
     8.2. Informative References  . . . . . . . . . . . . . . . . . . 19
   9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 20
   10. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 20



1. Introduction

   [EVPN-Overlays] discusses the use of EVPN as the control plane for
   Network Virtualization Overlay (NVO) networks, where VXLAN, NVGRE or
   MPLS over GRE can be used as possible data plane encapsulation
   options.

   While this model provides a scalable and efficient multi-tenant
   solution within the Data Center, it might not be easily extended to
   the WAN in some cases due to the requirements and existing deployed
   technologies. For instance, a Service Provider might have an already
   deployed (PBB-)VPLS or (PBB-)EVPN network that must be used to
   interconnect Data Centers and WAN VPN users.

   This document describes a Data Center Interconnect (DCI) solution for
   E-VPN overlay Data Center networks, assuming that the Data Center
   Gateway (DC GW) and the WAN Edge functions can be decoupled in two
   separate systems or integrated into the same system. The former
   option will be referred as "Decoupled DCI solution" throughout the
   document whereas the latter one will be referred as "Integrated DCI
   solution".

2. Decoupled DCI solution for EVPN overlay networks

   This section describes the interconnect solution when the DC GW and
   WAN Edge functions implemented in different systems. Figure 1 depicts
   the reference model described in this section.










Rabadan et al.          Expires August 18, 2014                 [Page 3]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


                                   +--+
                                   |CE|
                                   +--+
                                     |
                                  +----+
                             +----| PE |----+
           +---------+       |    +----+    |       +---------+
   +----+  |        +---+  +----+        +----+  +---+        |  +----+
   |NVE1|--|        |DC |  |WAN |        |WAN |  |DC |        |--|NVE3|
   +----+  |        |GW1|--|Edge|        |Edge|--|GW3|        |  +----+
           |        +---+  +----+        +----+  +---+        |
           | DC1      |      |     WAN      |      |    DC2   |
           |        +---+  +----+        +----+  +---+        |
           |        |DC |  |WAN |        |WAN |  |DC |        |
   +----+  |        |GW2|--|Edge|        |Edge|--|GW4|        |  +----+
   |NVE2|--|        +---+  +----+        +----+  +---+        |--|NVE4|
   +----+  +---------+       |              |       +---------+  +----+
                             +--------------+

   |<-EVPN-Overlay-->|<-VLAN->|<-WAN L2VPN->|<--PW-->|<--EVPN-Overlay->|
                      hand-off               hand-off

                         Figure 1 Decouple DCI model

   The following section describes the interconnect requirements that
   make Service Providers select this model and the requirements of the
   solution itself.

2.1. Interconnect requirements

   The proposed Interconnect architecture will be normally deployed in
   networks where the EVPN-Overlay provider and WAN providers are
   different entities and a clear demarcation is needed. The solution
   must observe the following requirements:

   o A simple connectivity hand-off must be provided between the EVPN-
      Overlay network provider and the WAN provider so that QoS and
      security enforcement are easily accomplished.

   o The solution must be independent of the L2VPN technology deployed
      in the WAN.

   o Multi-homing between DC GW and WAN Edge routers is required. Per-
      service load balancing MUST be supported. Per-flow load balancing
      MAY be supported but it is not a strong requirement since a
      deterministic path per service is usually required for an easy QoS
      and security enforcement.




Rabadan et al.          Expires August 18, 2014                 [Page 4]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


   o Ethernet OAM and Connectivity Fault Management (CFM) functions must
      be supported between the EVPN-Overlay network and the WAN network.

   o The following optimizations MAY be supported at the DC GW:
      + Unknown flooding reduction for the unicast traffic sourced from
        the DC Network Virtualization Edge devices (NVEs).
      + Control of the WAN MAC addresses advertised to the DC.
      + ARP flooding control for the requests coming from the WAN.

2.2. VLAN-based hand-off

   In this option, the hand-off between the DC GWs and the WAN Edge
   routers is based on 802.1Q VLANs. This is illustrated in Figure 1,
   between the DC GWs in DC1 and the WAN Edge routers. Each EVPN
   Instance (EVI) in the DC GW is connected to a different VPLS/EVI
   instance in the WAN Edge router by using a different C-TAG VLAN ID or
   a different combination of S-TAG/C-TAG VLAN IDs that matches at both
   sides. In this use-case, the WAN Edge router becomes a VPLS/EVPN PE
   with regular Attachment Circuits.

   This option provides the best possible demarcation between the DC and
   WAN providers and it does not require control plane interaction
   between both providers. The disadvantages of this model are the
   provisioning overhead and the reduced scalability (limited to the
   VLAN-ID space).

   In this model, the DC GW acts as a regular Network Virtualization
   Edge (NVE) towards the DC. Its control plane, data plane procedures
   and interactions are described in [EVPN-Overlays].

   The WAN Edge router acts as a (PBB-)VPLS or (PBB-)EVPN PE. Its
   functions are described in [RFC4761][RFC4762][RFC6074] or [EVPN][PBB-
   EVPN].

2.3. Pseudowire-based hand-off

   If MPLS can be enabled between the DC GW and the WAN Edge router, a
   more scalable DCI solution can be deployed. In this option the hand-
   off between both routers is based on FEC128-based pseudowires or,
   alternatively, FEC129-based pseudowires for a greater level of
   network automation. Note that this model still provides a clear
   demarcation boundary between DC and WAN, and security/QoS policies
   may be applied on a per pseudowire basis. The PW-based hand-off
   interconnect is illustrated in Figure 1, between the DC2 DC GWs and
   the WAN Edge routers.

   In this model, besides the usual MPLS procedures between DC GW and
   WAN Edge router, the DC GW MUST support an interworking function in



Rabadan et al.          Expires August 18, 2014                 [Page 5]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


   each EVI that requires extension to the WAN:

   o If a FEC128-based pseudowire is used between the EVI (DC GW) and
      the VSI (WAN Edge), the provisioning of the VCID for such
      pseudowire MUST be supported on the EVI and must match the VCID
      used in the peer VSI at the WAN Edge router.

   o If BGP Auto-discovery [RFC6074] and FEC129-based pseudowires are
      used between the DC GW EVI and the WAN Edge VSI, the provisioning
      of the VPLS-ID MUST be supported on the EVI and must match the
      VPLS-ID used in the WAN Edge VSI.

2.4. Multi-homing solution

   As already discussed, single-active multi-homing, i.e. per-service
   load-balancing multi-homing MUST be supported in this type of
   interconnect. All-active multi-homing may be considered in future
   revisions of this document.

   The DC GWs will be provisioned with a unique ESI per WAN interconnect
   and the hand-off attachment circuits or pseudowires between the DC GW
   and the WAN Edge router will be assigned to such ESI. The ESI will be
   administratively configured on the DC GWs according to the procedures
   in [EVPN] and its use assumes that the DC GWs are connected to a
   single DC and to a single WAN domain. Multi-homing for cases where
   the DC GWs are connected to more than one DC and/or more than one WAN
   domain is for further study. This ESI will be referred as "DCI-ESI"
   hereafter.

   The solution (on the DC GWs) MUST follow the single-active multi-
   homing procedures as described in [EVPN-Overlays] for the provisioned
   DCI-ESI, i.e. Ethernet A-D routes per ESI and per EVI will be
   advertised to the DC NVEs. The MAC addresses learnt (in the data
   plane) on the hand-off links will be advertised with the DCI-ESI
   encoded in the ESI field.

   The use of OAM is recommended between the DC GWs and the WAN Edge
   routers:

   o If the DCI solution is based on a VLAN hand-off, 802.1ag/Y.1731
      Ethernet-CFM can be used by the non-DF DC GW so that the peer WAN
      Edge router do not send any traffic to the DC GW for that
      particular EVI.

   o If the VPLS DCI solution is based on a pseudowire hand-off, the LDP
      PW Status bits TLV can be used by the non-DF to signal "Standby
      status" to the WAN Edge router for that particular EVI.




Rabadan et al.          Expires August 18, 2014                 [Page 6]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


2.5. Data Center Gateway Optimizations

      The following features MAY be supported on the DC GW in order to
      optimize the control plane and data plane in the DC.

2.5.1 Use of the Unknown MAC route to reduce unknown flooding

      The use of EVPN, as the control plane of Network Virtualization
      Networks in the DC, brings a significant number of benefits as
      described in [EVPN-Overlays]. There are however some potential
      issues that SHOULD be addressed when the DC EVIs are connected to
      the WAN VPN instances.

      The first issue is the additional unknown unicast flooding created
      in the DC due to the unknown MACs existing beyond the DC GW. In
      virtualized DCs where all the MAC addresses are learnt in the
      control/management plane, unknown unicast flooding is
      significantly reduced. This is no longer true if the DC GW is
      connected to a layer-2 domain with data plane learning.

      The solution suggested in this document is based on the use of an
      "Unknown MAC route" that is advertised by the Designated Forwarder
      DC GW. The Unknown MAC route is a regular EVPN MAC/IP
      Advertisement route where the MAC Address Length is set to 48 and
      the MAC address to 00:00:00:00:00:00 (IP length is set to 0).

      If this procedure is used, when an EVI is created in the DC GWs
      and the Designated Forwarder (DF) is elected, the DF will send the
      Unknown MAC route. The NVEs supporting this concept will prune
      their unknown unicast flooding list and will only send the unknown
      unicast packets to the owner of the Unknown MAC route. Note that
      the DCI-ESI will be encoded in the ESI field of the NLRI so that
      regular multi-homing procedures can be applied to this unknown MAC
      too (e.g. backup-path).

2.5.2. MAC address advertisement control

      Another issue derived from the EVI interconnect to the WAN layer-2
      domain is the potential massive MAC advertisement into the DC. All
      the MAC addresses learnt from the WAN on the hand-off attachment
      circuits or pseudowires must be advertised by BGP EVPN. Even if
      optimized BGP techniques like RT-constraint are used, the amount
      of MAC addresses to advertise or withdraw (in case of failure)
      from the DC GWs can be difficult to control and overwhelming for
      the DC network, especially when the NVEs reside in the
      hypervisors.

      This document proposes the addition of administrative options so



Rabadan et al.          Expires August 18, 2014                 [Page 7]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


      that the user can enable/disable the advertisement of MAC
      addresses learnt from the WAN as well as the advertisement of the
      Unknown MAC route from the DF DC GW. In cases where all the DC MAC
      addresses are learnt in the control/management plane, the DC GW
      may disable the advertisement of WAN MAC addresses. Any frame with
      unknown destination MAC will be exclusively sent to the Unknown
      MAC route owner(s).

2.5.3. ARP flooding control

      Another optimization mechanism, naturally provided by EVPN in the
      DC GWs, is the Proxy ARP function. The DC GWs SHOULD build a Proxy
      ARP cache table as per [EVPN]. When the active DC GW receives an
      ARP request coming from the WAN, the DC GW does a Proxy ARP table
      lookup and replies to the ARP request as long as the information
      is available in its table.

      This mechanism is specially recommended on the DC GWs since it
      protects the DC network from external ARP-flooding.

3. Integrated DCI solution for EVPN overlay networks

      When the DC and the WAN are operated by the same administrative
      entity, the Service Provider can decide to integrate the DC GW and
      WAN Edge PE functions in the same router for obvious CAPEX and
      OPEX saving reasons. This is illustrated in Figure 2. Note that
      this model does not provide an explicit demarcation link between
      DC and WAN anymore. ACLs or QoS policies between DC and WAN are
      not required.






















Rabadan et al.          Expires August 18, 2014                 [Page 8]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


                                +--+
                                |CE|
                                +--+
                                  |
                               +----+
                          +----| PE |----+
              +---------+ |    +----+    | +---------+
      +----+  |        +---+            +---+        |  +----+
      |NVE1|--|        |DC |            |DC |        |--|NVE3|
      +----+  |        |GW1|            |GW3|        |  +----+
              |        +---+            +---+        |
              |   DC1    |       WAN      |    DC2   |
              |        +---+            +---+        |
              |        |DC |            |DC |        |
      +----+  |        |GW2|            |GW4|        |  +----+
      |NVE2|--|        +---+            +---+        |--|NVE4|
      +----+  +---------+ |              | +---------+  +----+
                          +--------------+

      |<--EVPN-Overlay--->|<-----VPLS--->|<---EVPN-Overlay-->|
                          |<--PBB-VPLS-->|
        DCI options ->    |<-EVPN-MPLS-->|
                          |<--PBB-EVPN-->|

                         Figure 2 Integrated DCI model

3.1. Interconnect requirements

   The solution must observe the following requirements:

   o The DC GW function must provide control plane and data plane
      interworking between the EVPN-overlay network and the L2VPN
      technology supported in the WAN, i.e. (PBB-)VPLS or (PBB-)EVPN, as
      depicted in Figure 2.

   o Multi-homing MUST be supported. Single-active multi-homing with
      per-service load balancing MUST be implemented. All-active multi-
      homing, i.e. per-flow load-balancing, MUST be implemented as long
      as the technology deployed in the WAN supports it.

   o If EVPN is deployed in the WAN, the MAC Mobility, Static MAC
      protection and other procedures (e.g. proxy-arp) described in
      [EVPN] must be supported end-to-end.

   o Any type of inclusive multicast tree MUST be independently
      supported in the WAN as per [EVPN], and in the DC as per [EVPN-
      Overlays].




Rabadan et al.          Expires August 18, 2014                 [Page 9]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


3.2. VPLS DCI for EVPN-Overlay networks

3.2.1. Control/Data Plane setup procedures on the DC GWs

   Regular MPLS tunnels and TLDP/BGP sessions will be setup to the WAN
   PEs and RRs as per [RFC4761][RFC4762][RFC6074] and overlay tunnels
   and EVPN will be setup as per [EVPN-Overlays]. Note that different
   route-targets for the DC and for the WAN are normally required. A
   single type-1 RD per service can be used.

   In order to support multi-homing, the DC GWs will be provisioned with
   a DCI-ESI (see section 2.4), that will be unique per interconnection.
   Note that Ethernet Segment is a system wide assigned value, as
   opposed to the Ethernet Segments defined in [EVPN]. All the [EVPN]
   procedures are still followed for the DCI-ESI, e.g. any MAC address
   learnt from the WAN will be advertised to the DC with the DCI-ESI in
   the ESI field.

   A MAC-VRF per EVI will be created in each DC GW. The MAC-VRF will
   have two different types of tunnel bindings instantiated in two
   different split-horizon-groups:

      o VPLS pseudowires will be instantiated in the "WAN
      split-horizon-group".

      o Overlay tunnel bindings (e.g. VXLAN, NVGRE) will be instantiated
      in the "DC split-horizon-group".

   Attachment circuits are also supported on the same MAC-VRF, but they
   will not be part of any of the above split-horizon-groups.

   Traffic received in a given split-horizon-group will never be
   forwarded to a member of the same split-horizon-group.

   As far as BUM flooding is concerned, a flooding list will be created
   with the sub-list created by the inclusive multicast routes and the
   sub-list created for VPLS in the WAN. BUM frames received from a
   local attachment circuit will be flooded to both sub-lists. BUM
   frames received from the DC or the WAN will be forwarded to the
   flooding list observing the split-horizon-group rule described above.

   Note that the DC GWs are not allowed to have an EVPN binding and a
   pseudowire to the same far-end within the same MAC-VRF in order to
   avoid loops and packet duplication:

      o If an EVPN binding exists between two DC GWs and an attempt is
      made to setup a pseudowire between them, the pseudowire will be
      kept operationally down. The corresponding OAM signaling will be



Rabadan et al.          Expires August 18, 2014                [Page 10]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


      triggered.

      o If a pseudowire exists between two DC GWs and an attempt is made
      to setup an EVPN binding, the pseudowire will be brought
      operationally down before establishing the EVPN binding.

   The optimizations procedures described in section 2.5 can also be
   applied to this option.

3.2.2. Multi-homing procedures on the DC GWs

   Single-active multi-homing MUST be supported on the DC GWs. All-
   active multi-homing is not supported by VPLS.

   All the single-active multi-homing procedures as described by [EVPN-
   Overlays] will be followed for the DCI-ESI.

   The non-DF DC GW for the DCI-ESI will block the transmission and
   reception of all the bindings in the "WAN aplit-horizon-group" for
   BUM and unicast traffic.

3.3. PBB-VPLS DCI for EVPN-Overlay networks

3.3.1. Control/Data Plane setup procedures on the DC GWs

   In this case, there is no impact on the procedures described in
   [RFC7041] for the B-component. However the I-component instances
   become EVI instances with EVPN-Overlay bindings and potentially local
   attachment circuits. M EVI instances can be multiplexed into the same
   B-component instance. This option provides significant savings in
   terms of pseudowires to be maintained in the WAN.

   The DCI-ESI concept described in section 3.2.1 will also be used for
   the PBB-VPLS-based DCI.

   B-component pseudowires and I-component EVPN-overlay bindings
   established to the same far-end will be compared. The following rules
   will be observed:

      o Attempts to setup a pseudowire between the two DC GWs within the
      B-component context will never be blocked.

      o If a pseudowire exists between two DC GWs for the B-component
      and an attempt is made to setup an EVPN binding on an I-component
      linked to that B-component, the EVPN binding will be kept
      operationally down. Note that the BGP EVPN routes will still be
      valid but not used.




Rabadan et al.          Expires August 18, 2014                [Page 11]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


      o The EVPN binding will only be up and used as long as there is no
      pseudowire to the same far-end in the corresponding B-component.
      The EVPN bindings in the I-components will be brought down before
      the pseudowire in the B-component is brought up.

   The optimizations procedures described in section 2.5 can also be
   applied to this DCI option.

3.3.2. Multi-homing procedures on the DC GWs

   Single-active multi-homing MUST be supported on the DC GWs. All-
   active multi-homing MAY be supported. Procedures for the support of
   all-active multi-homing are for further study.

   All the single-active multi-homing procedures as described by [EVPN-
   Overlays] will be followed for the DCI-ESI for each EVI instance
   connected to B-component.

   The non-DF DC GW for the DCI-ESI will block the transmission and
   reception of all the EVPN bindings in the corresponding I-components
   for BUM and unicast traffic.


3.4. EVPN-MPLS DCI for EVPN-Overlay networks

   If EVPN for MPLS tunnels, EVPN-MPLS hereafter, is supported in the
   WAN, an end-to-end EVPN solution can be deployed. The following
   sections describe the proposed solution as well as the impact
   required on the [EVPN] procedures.

3.4.1. Control Plane setup procedures on the DC GWs

   The DC GWs MUST establish separate BGP sessions for sending/receiving
   EVPN routes to/from the DC and to/from the WAN. Normally each DC GW
   will setup one (two) BGP EVPN session(s) to the DC RR(s) and one(two)
   session(s) to the WAN RR(s). The route-distinguisher (RD) per EVI can
   be used for the EVPN routes sent to both, WAN and DC RRs. On the
   contrary, although reusing the same value is possible, different
   route-targets are expected to be handled for the same EVI in the WAN.

   As in the other discussed options, a DCI-ESI will be configured on
   the DC GWs for multi-homing.

   Received EVPN routes will never be reflected on the DC GWs but
   consumed and re-advertised (if needed):

      o Ethernet A-D routes, ES routes and inclusive multicast routes
        are consumed by the DC GWs and processed locally for the



Rabadan et al.          Expires August 18, 2014                [Page 12]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


        corresponding [EVPN] procedures.

      o MAC/IP advertisement routes will be received, imported and if
        they become active in the MAC FIB, the information will be re-
        advertised as a new route:

        + The RD will be the DC GW's RD for the service.

        + The ESI will be set to the DCI-ESI.

        + The Ethernet-tag will be 0 or a new value.

        + The MAC length, MAC address, IP Length and IP address values
        will be kept from the previously received NLRI.

        + The MPLS label will be 0 or a local label.

        + The appropriate RTs and [RFC5512] BGP Encapsulation extended
        community will be used according to [EVPN-Overlays].

   The DC GWs will also generate the following local EVPN routes that
   will be sent to the DC and WAN, with their corresponding RT and
   [RFC5512] BGP Encapsulation extended community values:

      o ES route for the DCI-ESI.

      o Ethernet A-D routes per ESI and EVI for the DCI-ESI.

      o Inclusive multicast routes with independent tunnel type value
        for the WAN and DC. E.g. a P2MP LSP may be used in the WAN
        whereas ingress replication is used in the DC.

      o MAC/IP advertisement routes for MAC addresses learnt in local
        attachment circuits. Note that these routes will not include the
        DCI-ESI, but ESI=0 or different from 0 for local Ethernet
        Segments (ES).

   Note that each DC GW will receive two copies of each of the above
   routes generated by the peer DC GW (one copy for the DC encapsulation
   and one copy for the WAN encapsulation). This is the expected
   behavior on the DC GW:

      o ES and A-D (per ESI) routes: regular BGP selection will be
        applied.

      o Inclusive multicast routes: if the Ethernet Tag ID matches on
        both routes, regular BGP selection applies and only one route
        will be active. It is recommended to influence the BGP selection



Rabadan et al.          Expires August 18, 2014                [Page 13]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


        so that the DC route is preferred. If the Ethernet Tag ID does
        not match, then BGP will consider them two separate routes. In
        that case, the EVI service will select the DC route.

      o MAC/IP advertisement routes for local attachment circuits: as
        above, the DC GW will select only one. The decision will be made
        at BGP or service level, depending on the Ethernet Tags.

   The optimizations procedures described in section 2.5 can also be
   applied to this option.

3.4.2. Data Plane setup procedures on the DC GWs

   The procedure explained at the end of the previous section will make
   sure there are no loops or packet duplication between the DC GWs of
   the same DC since only one EVPN binding will be setup in the data
   plane between the two nodes.

   As for the rest of the EVPN tunnel bindings, two flooding lists will
   be setup by each DC GW for the same MAC-VRF:

      o EVPN-overlay flooding list (composed of bindings to the remote
        NVEs or multicast tunnel to the NVEs).

      o EVPN-mpls flooding list (composed of MP2P and or LSM tunnel to
        the remote PEs)

   Each flooding list will be part of a separate split-horizon group.
   Traffic generated from a local AC can be flooded to both
   split-horizon-groups. Traffic from a binding of a split-horizon-group
   can be flooded to the other split-horizon-group and local ACs, but
   never to a member of its own split-horizon-group.

3.4.3. Multi-homing procedures on the DC GWs

   Single-active as well as all-active multi-homing MUST be supported.

   All the multi-homing procedures as described by [EVPN] will be
   followed for the DF election for DCI-ESI, as well as the backup-path
   (single-active) and aliasing (all-active) procedures on the remote
   PEs/NVEs. The following changes are required at the DC GW with
   respect to the DCI-ESI:

      o Single-active multi-homing; assuming a WAN split-horizon-group,
        a DC split-horizon-group and local ACs on the DC GWs:

        + Forwarding behavior on the non-DF: the non-DF MUST NOT forward
          BUM or unicast traffic received from a given split-horizon-



Rabadan et al.          Expires August 18, 2014                [Page 14]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


          group to a member of his own split-horizon group or to the
          other split-horizon-group. Only forwarding to local ACs is
          allowed (as long as they are not part of an ES for which the
          node is non-DF).

        + Forwarding behavior on the DF: the DF MUST NOT forward BUM or
          unicast traffic received from a given split-horizon-group to a
          member of his own split-horizon group or to the non-DF.
          Forwarding to the other split-horizon-group and local ACs is
          allowed (as long as they are not part of an ES for which the
          node is non-DF).

      o All-active multi-homing; assuming a WAN split-horizon-group, a
        DC split-horizon-group and local ACs on the DC GWs:

        + Forwarding behavior on the non-DF: the non-DF follows the same
          behavior as the non-DF in the single-active case but only for
          BUM traffic. Unicast traffic received from a split-horizon-
          group MUST NOT be forwarded to a member of its own split-
          horizon-group but can be forwarded normally to the other
          split-horizon-group and local ACs. If a known unicast packet
          is identified as a "flooded" packet, the procedures for BUM
          traffic MUST be followed.

        + Forwarding behavior on the DF: the DF follows the same
          behavior as the DF in the single-active case but only for BUM
          traffic. Unicast traffic received from a split-horizon-group
          MUST NOT be forwarded to a member of its own split-horizon-
          group but can be forwarded normally to the other split-
          horizon-group and local ACs. If a known unicast packet is
          identified as a "flooded" packet, the procedures for BUM
          traffic MUST be followed.

      o No ESI label is required to be signaled for DCI-ESI for its use
        by the non-DF in the data path. This is possible because the
        non-DF and the DF will never forward BUM traffic (coming from a
        split-horizon-group) to each other.

3.4.4. Impact on MAC Mobility procedures

   Since the MAC/IP Advertisement routes are not reflected in the DC GWs
   but rather consumed and re-advertised if active, the MAC Mobility
   procedures can be constrained to each domain (DC or WAN) and resolved
   within each domain. In other words, if a MAC moves within the DC, the
   DC GW MUST NOT re-advertise the route to the WAN with a change in the
   sequence number. Only when the MAC moves from the WAN domain to the
   DC domain, the DC GW will re-advertise the MAC with a higher sequence
   number in the MAC Mobility extended community. In respect to the MAC



Rabadan et al.          Expires August 18, 2014                [Page 15]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


   Mobility procedures described in [EVPN] the MAC addresses learnt from
   the NVEs in the local DC or on the local ACs will be considered as
   local.

   The sequence numbers MUST NOT be propagated between domains. The
   sticky bit indication in the MAC Mobility extended community MUST be
   propagated between domains.

3.4.5. Data Center Gateway optimizations

   All the Data Center Gateway optimizations described in section 2.5
   MAY be applied to the DC GWs when the DCI is based on EVPN-MPLS.

   In particular, the use of the Unknown MAC route, as described in
   section 2.5.1, reduces the unknown flooding in the DC but also solves
   some transient packet duplication issues in cases of all-active
   multi-homing. This is explained in the following paragraph.

   Consider the diagram in Figure 2 for EVPN-MPLS DCI and all-active
   multi-homing, and the following sequence:

      a) MAC Address M1 is advertised from NVE3 in EVI-1.

      b) DC GW3 and DC GW4 learn M1 for EVI-1 and re-advertise M1 to the
         WAN with DCI2-ESI in the ESI field.

      c) DC GW1 and DC GW2 learn M1 and install DC GW3/GW4 as next-hops
         following the EVPN aliasing procedures.

      d) Before NVE1 learns M1, a packet arrives to NVE1 with
         destination M1. The packet is subsequently flooded.

      e) Since both DC GW1 and DC GW2 know M1, they both forward the
         packet to the WAN (hence creating packet duplication), unless
         there is an indication in the data plane that the packet has
         been flooded by NVE1. If the DC GWs signal the same VNI/VSID
         for MAC/IP advertisement and inclusive multicast routes for
         EVI-1, such data plane indication does not exist.

   This undesired situation can be avoided by the use of the Unknown MAC
   route. If this route is used, the NVEs will prune their unknown
   unicast flooding list, and the non-DF DC GW will not received unknown
   packets, only the DF will. This solves the MAC duplication issue
   described above.

3.4.6. Benefits of the EVPN-MPLS DCI solution

   Besides retaining the EVPN attributes between Data Centers and



Rabadan et al.          Expires August 18, 2014                [Page 16]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


   throughout the WAN, the EVPN-MPLS DCI solution on the DC GWs has some
   benefits compared to pure BGP EVPN RR or Inter-AS model B solutions
   without a gateway:

      o The solution supports the connectivity of local attachment
      circuits on the DC GWs.

      o Different data plane encapsulations can be supported in the DC
      and the WAN.

      o Optimized multicast solution, with independent inclusive
      multicast trees in DC and WAN.

      o MPLS Label aggregation: for the case where MPLS labels are
      signaled from the NVEs for MAC/IP Advertisement routes, this
      solution provides label aggregation. A remote PE MAY receive a
      single label per DC GW MAC-VRF as opposed to a label per NVE.

      o The DC GW will not propagate MAC mobility for the MACs moving
      within a DC. Mobility intra-DC is solved by all the NVEs in the
      DC. The MAC Mobility procedures on the DC GWs are only required in
      case of mobility across DCs.

      o Proxy-ARP function on the DGWs can be leveraged to reduce ARP
      flooding in the DC or/and in the WAN.

3.5. PBB-EVPN DCI for EVPN-Overlay networks

   [PBB-EVPN] is yet another DCI option. It requires the use of DC GWs
   where I-components and associated B-components are EVI instances.

3.5.1. Control/Data Plane setup procedures on the DC GWs

   EVPN will independently run in both components, the I-component EVI
   and B-component EVI. Compared to [PBB-EVPN], the DC C-MACs are no
   longer learnt in the data plane on the DC GW but in the control plane
   through EVPN running on the I-component. Remote C-MACs coming from
   remote PEs are still learnt in the data plane. B-MACs in the B-
   component will be assigned and advertised following the procedures
   described in [PBB-EVPN].

   A DCI-ESI will be configured on the DC GWs for multi-homing, but it
   will only be used in the EVPN control plane for the I-component EVI.
   No ESI will be used in the control plane of the B-component EVI as
   per [PBB-EVPN].

   The rest of the control plane procedures will follow [EVPN] for the
   I-component EVI and [PBB-EVPN] for the B-component EVI.



Rabadan et al.          Expires August 18, 2014                [Page 17]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


   From the data plane perspective, the I-component and B-component EVPN
   bindings established to the same far-end will be compared and the I-
   component EVPN-overlay binding will be kept down following the rules
   described in section 3.3.1.

3.5.2. Multi-homing procedures on the DC GWs

   Single-active as well as all-active multi-homing MUST be supported.

   The forwarding behavior of the DF and non-DF will be changed based on
   the description outlined in section 3.4.3, only replacing the "WAN
   split-horizon-group" for the B-component.

3.5.3. Impact on MAC Mobility procedures

   C-MACs learnt from the B-component will be advertised in EVPN within
   the I-component EVI scope. If the C-MAC was previously known in the
   I-component database, EVPN would advertise the C-MAC with a higher
   sequence number, as per [EVPN]. From a Mobility perspective and the
   related procedures described in [EVPN], the C-MACs learnt from the B-
   component are considered local.

3.5.4. Data Center Gateway optimizations

   All the considerations explained in section 3.4.5 are applicable to
   the PBB-EVPN DCI option.


5. Conventions and Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC-2119 [RFC2119].

   BUM: it refers to the Broadcast, Unknown unicast and Multicast
   traffic

   DF: Designated Forwarder

   DC GW: Data Center Gateway

   DCI: Data Center Interconnect

   ES: Ethernet Segment

   ESI: Ethernet Segment Identifier

   DCI-ESI: ESI defined on the DC GWs for multi-homing to/from the WAN



Rabadan et al.          Expires August 18, 2014                [Page 18]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


   EVI: EVPN Instance

   MAC-VRF: it refers to an EVI instance in a particular node

   NVE: Network Virtualization Edge

   TOR: Top-Of-Rack switch

   VNI/VSID: refers to VXLAN/NVGRE virtual identifiers


6. Security Considerations

   This section will be completed in future versions.

7. IANA Considerations


8. References

8.1. Normative References


   [RFC4761]Kompella, K., Ed., and Y. Rekhter, Ed., "Virtual Private LAN
   Service (VPLS) Using BGP for Auto-Discovery and Signaling", RFC 4761,
   January 2007.

   [RFC4762]Lasserre, M., Ed., and V. Kompella, Ed., "Virtual Private
   LAN Service (VPLS) Using Label Distribution Protocol (LDP)
   Signaling", RFC 4762, January 2007.

   [RFC6074]Rosen, E., Davie, B., Radoaca, V., and W. Luo,
   "Provisioning, Auto-Discovery, and Signaling in Layer 2 Virtual
   Private Networks (L2VPNs)", RFC 6074, January 2011.


8.2. Informative References

   [E-VPN] Sajassi et al., "BGP MPLS Based Ethernet VPN", draft-ietf-
   l2vpn-evpn-05.txt, work in progress, February, 2014

   [PBB-EVPN] Sajassi et al., "PBB-EVPN", draft-ietf-l2vpn-pbb-evpn-06,
   work in progress, October, 2014

   [EVPN-Overlays] Sajassi-Drake et al., "A Network Virtualization
   Overlay Solution using EVPN", draft-sd-l2vpn-evpn-overlay-02.txt,
   work in progress, October, 2013




Rabadan et al.          Expires August 18, 2014                [Page 19]


Internet-Draft           DCI for E-VPN Overlays        February 14, 2014


9. Acknowledgments

   This document was prepared using 2-Word-v2.0.template.dot.


10. Authors' Addresses


   Jorge Rabadan
   Alcatel-Lucent
   777 E. Middlefield Road
   Mountain View, CA 94043 USA
   Email: jorge.rabadan@alcatel-lucent.com


   Senthil Sathappan
   Alcatel-Lucent
   Email: senthil.sathappan@alcatel-lucent.com


   Wim Henderickx
   Alcatel-Lucent
   Email: wim.henderickx@alcatel-lucent.com


   Florin Balus
   Nuage Networks
   Email: florin@nuagenetworks.net


   Senad Palislamovic
   Alcatel-Lucent
   Email: senad.palislamovic@alcatel-lucent.com


















Rabadan et al.          Expires August 18, 2014                [Page 20]