Network Working Group                                   Jeremy De Clercq
INTERNET DRAFT                                         Olivier Paridaens
<draft-ietf-l3vpn-ce-based-00.txt>                               Alcatel

                                                        Andrew Krywaniuk
                                                              Cliff Wang

                                                              March 2003
                                                 Expires September, 2003

                         An Architecture for
        Provider Provisioned CE-based Virtual Private Networks
                              using IPsec

                  <draft-ietf-l3vpn-ce-based-00.txt>

Status of this Memo


   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that other
   groups may also distribute working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months. Internet-Drafts may be updated, replaced, or obsoleted by
   other documents at any time.  It is not appropriate to use Internet-
   Drafts as reference material or to cite them other than as a
   ``working draft'' or ``work in progress.''

   To view the entire list of current Internet-Drafts, please check the
   "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow
   Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern
   Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au(Pacific
   Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast).

   Distribution of this memo is unlimited.

Abstract

   This document describes the procedures for a Service Provider to
   offer Virtual Private Network Services to its customers by
   provisioning the CE devices on behalf of the customer. The IPsec
   technology is used to protect the customer traffic.

0. SubIP-Area Section



De Clercq et al.        Expires September 2003                 [Page 1]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   SUMMARY

   The PPVPN framework document [FRAMEWORK] identifies three basic
   provider provisioned VPN types : Provider Provisioned Network Based
   (or PE-based) Layer 3 VPNs, Provider Provisioned Layer 2 VPNs and
   Provider Provisioned CE-based VPNs.

   This document describes a method enabling a Service Provider to offer
   VPN services to its customers by provisioning the CE devices on
   behalf of the customer (Provider Provisioned CE-based VPNs).

   For a CE-based VPN to be set up under the SP's control, the VPN
   customer informs the Service Provider of which sites (identified by a
   set of CE devices) should become part of the considered VPN and what
   the requested topology of the VPN should look like. The SP then
   configures and maintains a VPN database and manages the Customer's
   VPN.

   The model proposed in this document uses the IPsec protocol suite for
   the purpose of securing the customer VPN traffic.

   When the model proposed is used, the addition of one VPN site only
   requires a minimum amount of configuration. This is obtained via a
   provisioning scheme using a network management protocol.

   RELATED DOCUMENTS

   See References section (especially [CE-SOL], [TOUCH]).

   WHERE DOES IT FIT IN THE PICTURE OF THE SUB-IP WORK

   PPVPN box.

   WHY IS IT TARGETED AT THIS WG

   This document describes a mechanism for Provider Provisioned CE-based
   VPNs, which is clearly identified as an item within the scope of the
   PPVPN WG, as stated from the following WG charter extract: "This
   working group is responsible for defining and specifying a limited
   number of sets of solutions for supporting provider-provisioned
   virtual private networks (PPVPNs). The work effort will include the
   development of a framework document, a service requirements document
   and several individual technical approach documents that group
   technologies together to specify specific VPN service offerings. The
   framework will define the common components and pieces that are
   needed to build and deploy a PPVPN. Deployment scenarios will include
   provider-managed VPN components located on customer premises.".




De Clercq et al.        Expires September 2003                 [Page 2]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   JUSTIFICATION

   This document is justified by the fact that it defines a solution for
   PP CE-based VPNs, which are identified as a specific type of PPVPNs
   both in the WG charter and the general framework I-D. As described in
   the general framework I-D, PP CE-based VPNs have specific
   characteristics compared to Network-Based VPNs especially with
   regards to management and security.

Changes since last version

   Proposed changes to version -02.txt of this document, based on
   comments received on the ppvpn exploder, have mostly been dealt with
   in [CE-SOL].

   This version -03.txt differs not significantly from the previous
   version (-02.txt); this allows this (architecture) document to be
   used for reference's purpose.

1. Introduction

   The PPVPN framework document [FRAMEWORK] identifies three basic
   provider provisioned VPN types : Provider Provisioned Network Based
   (also termed PE-based) Layer 3 VPNs, Provider Provisioned Layer 2
   VPNs and Provider Provisioned CE-based VPNs.

   This document describes a method enabling a Service Provider to offer
   VPN services to its customers by provisioning the CE devices on
   behalf of the customer (Provider Provisioned CE-based VPNs).

   For a CE-based VPN to be set up under the SP's control, the VPN
   customer informs the Service Provider of which sites (identified by a
   set of CE devices) should become part of the considered VPN and what
   the requested topology of the VPN should look like. The SP then
   configures and updates its VPN database, and then provisions and
   manages the Customer's VPN.

   The model proposed in this document uses the IPsec protocol suite for
   the purpose of securely tunneling the customer VPN traffic.

   This specification proposes some mechanisms that can be used to
   enhance the scalability of the VPN provisioning (e.g. the addition of
   a VPN site to an existing VPN).

2. Reference Model

   The reference model upon which the mechanisms and procedures
   described in this document are based, is taken from the CE-based VPN



De Clercq et al.        Expires September 2003                 [Page 3]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   reference model described in [FRAMEWORK]. The most important aspects
   of that framework model and the restrictions that are relevant to
   this document are described in this section.



    +---------+  +------------------------------------+  +---------+
    |         |  |                                    |  |         |
    |         |  |                     +------+     +------+  : +------+
+------+ :    |  |                     |      |     |      |  : |  CE  |
|  CE  | :    |  |                     |   P  |     |  PE  |  : |device|
|device| :  +------+    VPN tunnel     |router|     |router|  : |  of  |
|  of  |=:====================================================:=|VPN  A|
|VPN  A| :  |      |                   +------+     +------+  : +------+
+------+ :  |  PE  |                                  |  |    :    |
+------+ :  |router|                                  |  |    :    |
|  CE  | :  |      |              VPN tunnel        +------+  : +------+
|device|=:====================================================:=|  CE  |
|  of  | :  +------+                                |  PE  |  : |device|
|VPN  B| :    |  |                                  |router|  : |  of  |
+------+ :    |  |  +------------+   +------------+ |      |  : |VPN  B|
    |    :    |  |  |  Customer  |   |   Network  | +------+  : +------+
    |Customer |  |  | management |   | management |   |  |    :    |
    |interface|  |  |  function  |   |  function  |   |  |Customer |
    |         |  |  +------------+   +------------+   |  |interface|
    |         |  |                                    |  |         |
    +---------+  +------------------------------------+  +---------+
    | Access  |  |<---------- SP network(s) --------->|  | Access  |
    | network |  |                                    |  | network |

   Figure 1: Reference model for provider provisioned CE-based VPNs

2.1 Entities in the reference model

   o Customer Edge (CE) device

      In the context of this solution, a CE device is a router located
      at the edge of a customer site, that has IP connectivity with a
      SP's PE device. A CE device maintains one or more VPN tunnel
      endpoints. The VPN-specific functions in the CE device are managed
      by the SP's management system. It is very common for the SP to
      manage the complete CE device, while it is also possible for the
      customer to self-manage part of the CE's functions. In such a
      situation, the separation of responsabilities is to be clearly
      defined in the SLA.

      Note that other functions that are normally applied by the PE
      router may need to be performed by the CE device in this context



De Clercq et al.        Expires September 2003                 [Page 4]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


      (e.g. NAT functionality, QoS classification, etc.).

      The CE device may also provide general (non VPN-oriented) Internet
      connectivity for the customer network. Such connectivity may be
      achieved via the SP's PE router that provides the VPN connectivity
      or some other router (of the same or another SP). In such a
      situation, the CE device must be able to distinguish between
      traffic to be sent through a VPN and traffic to be sent outside
      any VPN.

   o Provider Edge (PE) router

      In the context of Provider Provisioned CE-based VPNs, a PE router
      is a router, located at the edge of the Service Provider's
      network, that does not have any VPN-specific functionality. A PE
      router is attached via an access connection to one or more CE
      devices, and offers possibly limited or restricted IP connectivity
      over the access connections to these CE devices.

   o SP network

      A SP network is a network administrated by a single service
      provider. In the context of provider provisioned CE-based VPNs,
      the SP network consists of the Service Provider's IP (backbone)
      network and the Service Provider's management functions that
      manage both its own IP (backbone) network and the VPN customer's
      VPN functions on the CE devices.

   o Access connection

      An access connection represents an isolated layer 2 connectivity
      between a CE device and a PE router. This includes dedicated
      physical circuits, logical circuits (such as Frame Relay and ATM),
      IP tunnels (e.g., using IPsec, L2TP) and shared-medium access
      (such as Ethernet-based access). In the context of provider
      provisioned CE-based VPNs, the CE device and the PE router have
      layer 3 connectivity over the Access Connection.

   o VPN tunnel

      A VPN tunnel is a logical link between two entities which is
      created by encapsulating packets within an encapsulating header
      for purpose of transmission between those two entities for support
      of VPNs. In the context of provider provisioned CE-based VPNs, a
      VPN tunnel is an IP tunnel (e.g., using IPsec, L2TP, GRE, IP-in-
      IP) between two CE devices over the SP's network. In the context
      of this document, a VPN tunnel is achieved using IPsec in tunnel
      mode or via an IP-in-IP tunnel protected by IPsec in transport



De Clercq et al.        Expires September 2003                 [Page 5]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


      mode between two CE devices. [GRE-CE] describes how to use GRE
      encapsulation for CE to CE tunneling in a CE-based IPsec VPN
      scenario.

2.2 Assumed Reachability

   It is assumed in this specification that the CE devices have at least
   one IP address that is known to the SP network and that is routable
   within the SP's network. It is also assumed that every CE device
   knows how to reach the other CE devices attached to the common SP.
   This might be via a direct routing entry in the CE's routing table or
   alternatively via a default route towards the PE device it is
   attached to. These CE IP addresses will be known in the SP network,
   if not globally, then at least in the PE devices engaged in the VPN
   services.

   This means that either a routing protocol will be running between the
   CE and the PE device, exchanging routes belonging to the service
   provider's routing realm; or alternatively, the CE will have a
   configured default route or static route towards the PE, and the PE
   will have assigned an IP address to the CE device before or upon
   set-up of the IP service offered to the CE device.

   As some of the CE's interfaces are in the SP's routing space and some
   of the CE's interfaces are in the customer's routing space, CE
   devices will need to be able to operate in two distinct (separate and
   possibly overlapping) routing spaces.

   More details regarding the assumed CE's reachability are provided in
   [CE-SOL].

2.3 Assumed Service Provider's infrastructure

   It is assumed that the Service Provider has a secure provisioning
   channel to every attached CE device. This secure provisioning channel
   will be used to exchange VPN-specific configuration information
   between the SP's VPN database and the CE devices.

   The particular implementation of this provisioning channel is outside
   of the scope of this document. Some examples are: (i) manual
   configuration by a trusted party, directly on the considered network
   element; (ii) via a management protocol running over an IPsec-secured
   connection between the SP's management center and the considered
   network element; (iii) via a management protocol that has its own
   implicit security mechanisms. For scalability reasons, manual
   configuration is not recommended.

3. Configuring the CE-based VPN



De Clercq et al.        Expires September 2003                 [Page 6]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   Note that [CE-SOL] proposes a detailed instantiation of the
   procedures described in this section.

   As a Service Provider that is offering VPN services over its backbone
   network will be responsible for the configuration and the management
   of a potential large amount of VPNs, it is required that the
   provisioning actions for the initial establishment and the
   maintenance of a PP CE-based VPN would be minimal.

   The minimal configuration required is to first configure the Service
   Provider's VPN database with the CE device to be part of a well-
   defined VPN. Further device provisioning can be achieved through the
   management channel.

   For the purpose of CE device configuration, the Service Provider will
   maintain a VPN database per VPN customer. The exchange of information
   between the SP's VPN database and the targeted CE-devices is achieved
   using an information exchange/configuration protocol such as LDAP,
   SNMP, COPS, XML/HTTP etc. We will call this protocol 'the management
   protocol' throughout the rest of this document.

   It is assumed for the scope of this document that the management
   channel used for the configuration information exchange by the chosen
   protocol is sufficiently secured (e.g. by using a specific IPsec
   protection for that purpose, or by using a SSL or SSH protected
   channel, etc.).

3.1 (Minimal) configuration needed at the CE device

   - 'Management channel' information : the information needed to access
   (or to exchange information with) the SP's VPN database ('database
   reachability information', authentication information for the VPN
   database, encryption information for the configuration management
   channel, necessary credentials, etc.). Note that this is assumed to
   be present, and that setting up a secure management channel is
   outside of the scope of this document.

   - Peer CE devices : the IP addresses (in the SP's realm) of the peer
   CE devices that belong to the same VPN and to which a direct peering
   relationship is required. This information can be one IP address
   (e.g. in the case that the considered CE device is a 'spoke' in a
   'hub&spoke' topology); this information can be the complete set of
   the IP addresses of all the other CE devices belonging to the same
   VPN (in the case of a 'fully meshed' topology); or this information
   can be a subset of the IP addresses of the CE devices belonging to
   the same VPN (for more complex topologies).

   - Security Information : the information needed to set up and



De Clercq et al.        Expires September 2003                 [Page 7]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   maintain IPsec Security Associations with the Peer CE devices:

      - a set of transforms to use with IPsec (this information relates
      to the protection of the actual user data packets) for every
      Security Association.

      - tunnel property information for every SA that the considered CE
      participates in : traffic-driven tunnel or 'permanent' tunnel;
      tunnel mode or IPsec transport mode on an IP-in-IP tunnel; dynamic
      routing through the tunnel or not.

      - an IKE credential: (i) a pre-shared key or (ii) a private
      key/certificate pair and a certificate for a Certificate Authority
      (these are to be used to establish the IPsec security associations
      with the peer CE devices). This credential will be used for the
      generation of the keys for all the Security Associations the
      considered CE participates in.

   - VPN identifier: an identifier that uniquely identifies the VPN that
   the customer belongs to. The VPN identifier defined in [RFC2685] can
   be used for this purpose, or any unique identifier the PPVPN WG
   agrees upon. Note that the use of this VPN identifier will be
   necessary when one site is allowed to be part of more than one VPN,
   but that this specification does not currently specify that scenario.
   Another feature that may require the use of a unique VPN identifier
   is the IKE-based membership discovery mechanism described in section
   3.2.

   This means that the VPN database (identified by a VPN identifier) of
   the SP's management system contains a list of all the CE devices
   belonging to the specified VPN, a description of the topology (full
   mesh, hub&spoke, etc.) and some security-related information related
   to every CE-to-CE tunnel.

   Note that every CE also needs to be configured as to align the
   routing within the VPN with the tunneling between VPN sites. Section
   4 describes these issues in more detail.

3.2 Updating configuration information

   One of the most important requirements relating to scalability for
   PP-VPNs is that the addition/deletion of a certain site to/from an
   existing VPN should be possible with a minimal of configuration
   effort. Manual configuration of the information related to the new
   site into every existing CE in the particular VPN is not a scalable
   option. An automatic mechanism for membership discovery/distribution
   is therefor required.




De Clercq et al.        Expires September 2003                 [Page 8]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   The CE's IP address in the SP's realm, the VPN-ID of the VPN it
   belongs to, the 'role' of the new site in the existing topology (hub,
   spoke, etc.), and the necessary security information need to be
   configured in the concerned VPN database.

   Different methods to achieve the requested 'automatic' behavior are
   possible. This section describes some possible approaches but does
   not claim to be exhaustive.

   o using the SP's management system

      In this approach, the customer notifies the SP of the site to be
      added (out of band), and the SP's management system takes care of
      configuring its VPN database, takes care of provisioning the new
      CE device and takes care of updating the other CE devices that
      belong to the considered VPN. This provisioning also initiates the
      IPsec signaling between the new CE device and the existing CE
      devices.

   o pushing information from the SP's VPN database

      This approach can be seen as a more detailed description of an
      example of the previous approach.

      The customer first notifies the SP of the site to be added. The SP
      then configures its VPN database with the new membership
      information. This updating of the VPN database will trigger an
      information exchange between the SP's VPN database and the
      concerned CE devices (the new CE device and all the CE devices it
      has to peer with). This distribution of the updated information
      happens over a secured 'management channel', via a configuration
      protocol (such as COPS, DHCP, LDAP, SNMP, etc.).

      This method implies that the protocol used for the distribution of
      the configuration information to the CE devices should be usable
      in a "PUSH" mode from 'server' to 'client' (the management system
      - server - pushes the updated information to the concerned parties
      - clients -). It is to be noted that some information distribution
      technologies such as LDAP are natively not PUSH oriented. If the
      VPN configuration is stored using such a technology, it is
      necessary to define a mechanism that enables the CE to be
      triggered so as to fetch VPN configuration from its repository.
      Such a mechanism could be based on the use of SNMP messages sent
      to the CE device, with specific variables that trigger the fetch
      operation.

      Note that a solution using this method should integrate it in a
      reliable architure.



De Clercq et al.        Expires September 2003                 [Page 9]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   o using the SA signaling protocol (IKE)

      The customer first notifies the SP of the site to be added. The SP
      then configures the new CE device and its VPN database (this could
      eventually be done manually). When the new CE device is configured
      with the required configuration information, it will start the
      IPsec signaling (via IKE or its successor) with the peer CE
      devices it is configured with.

      When a peer CE device (that already belongs to the considered VPN)
      receives an initial IPsec signaling message, it MUST check whether
      the initiating CE device belongs to the set of CE devices the
      considered CE needs to peer with, by looking at its VPN
      configuration information. If the initiating CE device does not
      belong to that set according to its VPN configuration information,
      the CE device needs to update its VPN configuration information by
      fetching the 'latest' VPN configuration information from the SP's
      VPN database (e.g. using a protocol for database retrieval such as
      COPS, DHCP, FTP, HTTP, LDAP, etc.) over the secured management
      channel. If after the updating of the VPN configuration
      information, the initiating CE device is still not part of the CE
      devices the considered CE device has to peer with, then the
      considered CE device MUST reject the incoming IPsec SA
      establishment. If the initiating CE device belongs to the set of
      CE devices that the considered CE device must peer with, then the
      incoming IPsec SA establishment should be accepted and an IPsec
      'tunnel' should be established.

      The details of this procedure are outside of the scope of this
      document. See [CE-SOL] for a detailed example.

3.3 Setting up the VPN tunnels

   When one VPN site sends traffic to another VPN site belonging to the
   same VPN, these IP packets will be secured via IPsec. This means that
   an IPsec Security Association is needed between each pair of sites
   that directly exchange private VPN data with each other.

   The Internet Key Exchange protocol (IKE, [IKE]) or its successor will
   be used for the purpose of automatic setup of security associations
   between VPN sites within the same VPN.

   These dynamically established Security Associations can lead to
   'permanent' IPsec-secured tunnels. These tunnels are always available
   and not traffic-triggered. It is then required to frequently re-
   negotiate the SA (via IKE) before the IPsec timers of the connection
   time out. The set-up of a 'permanent' IPsec tunnel can be triggered
   by the configuration of a new peer CE device within the same VPN. An



De Clercq et al.        Expires September 2003                [Page 10]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   advantage of this method is that the IPsec tunnel is always
   available, and that eventual traffic does not encounter an extra
   delay due to the setup time of a new SA. The use of 'permanent' IPsec
   tunnels is recommended for CE-based site-to-site VPNs.

   Provider provisioned CE-based IPsec VPNs as described by this
   document SHOULD use 'permanent' IPsec Security Associations when
   dynamic routing through IPsec-secured tunnels is used.

   Alternatively, the IPsec SA setup can be triggered by the effective
   (data) traffic flow going from one site to another. In this case, the
   arrival of data packets at the CE device, coming from within the VPN
   site and going to another VPN site, will be noticed by the CE's IPsec
   or routing database, and an IKE exchange will be initiated to set up
   an IPsec secured connection between both parties. Once the secure
   tunnel is set up, the data packets can flow from one site to the
   other in a secure way. When no traffic flows for a certain duration
   of time, the secure tunnel will be torn down again. An advantage of
   this method is that an IPsec tunnel is only to be maintained when
   there is effectively traffic flowing. A disadvantage is the extra
   delay introduced for the traffic during IKE signaling and the
   interaction with the tunneled inter-site VPN routing information
   distribution.

   Provider provisioned CE-based IPsec VPNs as described by this
   document MAY use traffic-driven IPsec SA establishment when static
   intra-VPN inter-site routing is used (no dynamic routing through the
   IPsec tunnels), see section 4.3. Provider provisioned CE-based IPsec
   VPNs as described by this document SHOULD NOT use traffic-driven
   IPsec SA establishment when dynamic site-to-site routing through the
   IPsec-secured tunnels is used.

   The CE configuration determines whether traffic-driven SA
   establishment is used or not, and whether dynamic routing through
   IPsec tunnels is used or not.

   Note that IPsec tunnels are unidirectional in nature, but that the
   set up of one direction is accompanied by the set up of the reverse
   direction IPsec tunnel.

   This document describes two possible ways to use IPsec in CE-based
   VPN scenarios (see section 5): in 'transport mode' or in 'tunnel
   mode'. The CE configuration, IKE exchange and resulting SA's must
   specify which mode will be used.

   Note that the number of peer CE devices with which a specific CE
   device must have an IPsec connection to secure the data traffic, is
   dependent on the specific 'role' of the site in the considered VPN. A



De Clercq et al.        Expires September 2003                [Page 11]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   hub CE will for example have a larger number of tunnels to support
   than a spoke device.

3.4 VPN resilience

   When IPsec tunnels are used for VPN site-to-site connection, many
   times it is important for a SP to provide its customers with
   connection resilience. The site-to-site VPN could be broken due to
   either a VPN link failure or a VPN node failure. VPN resilience
   support would require both VPN node redundancy and VPN link
   redundancy.

   To support VPN node redundancy, extra CE routers can be placed. Two
   deployed CE routers can provide fail-over and potentially load
   balancing support. It is possible that only one end of a VPN link has
   redundancy. For example, in a hub-spoke topology, the hub node is
   more important than a spoke node. It is not unusual to provide
   redundancy just to the hub router.

   The VPN link resilience support comes from two levels: IPsec tunnel
   level and physical link level. To provide physical link redundancy, a
   customer may subscribe two links from the SP. At the IPsec tunnel
   level, a customer may set up more than one IPsec tunnel. If the
   remote site has only a single CE device, both tunnels will terminate
   at the same device. If the remote site has two CE devices, each
   tunnel may terminate at different CE devices.

   When the redundant VPN links connect to two separate CE devices at
   the remote site, the user (or the SP) must decide how to split the
   traffic between the two VPN links. One choice is to designate one
   link as the primary link which carries all the traffic and leave the
   other idle link as the backup. The other choice is to divide the
   traffic between the two links. When two links are used, the important
   issue is to detect any link failure and divert the traffic to the
   healthy link. One way to achieve this is to run routing keep-alive
   through both tunnels. When a link is down, the routing protocol is
   able to discover the link outage and reroute the traffic accordingly.

4. Exchanging and maintaining VPN routes

   One of the requirements for PP CE-based VPNs is that dynamic routing
   is not only supported within individual VPN sites, but also between
   the different VPN sites of a specific VPN. This means that when a
   change in the routing information in a specific site occurs, the
   other sites that belong to the same VPN must be notified of that
   change.

   This document assumes that the routing within a VPN site is



De Clercq et al.        Expires September 2003                [Page 12]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   controlled by the VPN customer, and thus is outside of the scope of
   this specification.

   On the other hand, the role of the CE device with regards to routing,
   the interaction between the intra-site and inter-site routing and the
   relationship between routing and VPN tunnels are addressed in this
   specification. For more detailed aspects concerning issues with
   regards to routing and CE-based IPsec VPNs, we refer to [TOUCH],
   [DUFFY] and [KNIGHT].

4.1 The CE device and VPN routing

   On the customer network side, a CE router connects to internal
   networks of an enterprise, where one or more subnets can reside. Many
   times, the CE router may interact with another internal router.

   The CE is involved in (i) the intra-site routing, (ii) the VPN tunnel
   termination, and (iii) the inter-site VPN routing. The CE device
   could be an integrated device providing both routing and IPsec tunnel
   termination. Sometimes, a dedicated VPN terminator may be used.
   Implementations in which the VPN tunnel terminator resides on a
   firewall are also very common. For the sake of simplicity, we assume
   that the CE router is an integrated device that participates in the
   intra-site routing (e.g. via an IGP) and at the same time terminates
   VPN tunnels.

   In the context of this document, the routing aspects within a VPN
   site (intra-site routing information distribution) are controlled by
   the VPN customer. The role of the SP is either to completely manage
   the inter-site reachability distribution or to configure the CE
   devices in such a way that the customer may transparently run routing
   protocols (or configure static routes) through the VPN tunnels.

4.2 IPsec and routing

   IPsec is a layer 3 tunneling protocol, which operates purely at the
   IP layer. The IETF IPsec Working Group specifies the IPsec standards
   ([IPSEC], [RFC2402], [RFC2406], [RFC2407], [RFC2408], and [IKE]). The
   interaction between IPsec and layer 3 routing is often troublesome
   and has been described in [DUFFY], [TOUCH] and [KNIGHT]. Depending on
   individual implementations, difficulty may arise when an IPsec user
   wants to support robust routing across IPsec VPNs sites.

   Details regarding the problems of the interaction between VPN routing
   and VPN tunneling, and some proposed solutions to counter these
   issues, can be found in [DUFFY], [TOUCH] and [KNIGHT].

4.3 Mechanisms to exchange VPN routes between VPN sites



De Clercq et al.        Expires September 2003                [Page 13]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   This document describes two alternative mechanisms for the purpose of
   exchanging VPN routes between VPN sites. The mechanism that is in use
   for a particular tunnel is identified by the CE configuration
   information.

4.3.1 Tunneling routing information between CE devices through the IPsec
   tunnels.

   In the first mechanism to exchange VPN reachability information,
   normal routing protocol messages are tunneled through the IPsec-
   secured tunnels between peering sites. The CE-to-CE IPsec-secured
   tunnels between VPN sites are then being seen as point-to-point links
   by the customer networks and are inserted as such in the routing
   protocol functions of the CE devices. This means that when a change
   in the reachability occurs in one particular site, a routing protocol
   (such as RIP, OSPF, etc.) will take care of the distribution of the
   new reachability information within the site, but also to all other
   sites, through the VPN tunnels that the considered CE is peering
   with.

   When routing protocol messages are tunneled through the IPsec-secured
   tunnels ('dynamic routing through IPsec-secured tunnels') as per this
   section, IPsec transport mode secured IP-in-IP tunnels as per [TOUCH]
   SHOULD be used (in contrast to IPsec tunnel mode tunnels).

   Note that other approaches may use a combination of dynamic routing
   and IPsec tunnel mode tunnels, though it is not clear how
   interoperability will then be assured.

   Another issue to consider is the fact that using a traffic-driven
   tunnel establishment mechanism and at the same time using an approach
   whereby a routing protocol (with a keep-alive mechanism) runs on top
   of the VPN tunnel, does not seem currently applicable: the delay
   introduced by the tunnel establishment phase could lead to a loss of
   routing updates and the routing protocol's keep-alive mechanism could
   interact with the tunnel establishment in an undesired way. Therefor,
   when dynamic routing is used through IPsec-secured CE-to-CE tunnels,
   traffic-driven SA establishment SHOULD NOT be used.

4.3.2 Exchanging VPN reachability information via SP's management

   In non-dynamic environments, SP's management can distribute static
   routes to the CE devices.

   When CE-to-CE IPsec tunnel mode tunnels are used (with the complete
   SPD-controlled filtering), a management-based inter-site reachability
   information distribution SHOULD be used.




De Clercq et al.        Expires September 2003                [Page 14]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   When traffic-driven SA establishment is used, a management-based
   inter-site reachability information distribution SHOULD be used.

   Within the sites themselves, the CE device may then inject the
   updated reachability information into the local routing protocol
   function.

   Note that this method does not require the use of the same
   'management protocol' for the configuration of the CE device and for
   the dynamic exchange of reachability information. A dedicated
   protocol may be used for that purpose.

4.3.3 Applicability of the mechanisms to exchange VPN routes between the
   VPN sites

   The 'tunneling approach' (as described in section 4.3.1) does not
   require a new interaction between the customer and the SP and offers
   transparent routing features to the VPN customer. Existing routing
   protocols can be used and the SP is not involved with the propagation
   of routing information updates. It prohibits some specific IPsec
   features though (such as traffic-initiated SA establishment, and the
   complete use of SPD-based filtering).

   The 'management approach' (as described in section 4.3.2) allows for
   the full use of specific IPsec features, but requires a new
   interaction between the CE device and the SP, and breaks the VPN
   customer's routing transparency. The SP participates in the inter-
   domain distribution of routing information.

4.4 Relationship between VPN membership, VPN tunnel status and routing

   When VPN membership changes, VPN tunnels may need to be intentionally
   deleted, and this change needs to be reflected in the routing
   information of all sites belonging to the VPN (see section 6).

   But VPN tunnels can also un-intentionally break for a variety of
   reasons. The routing within the VPN will be affected and updated
   information must be distributed over all VPN sites.

5. Tunneling IP traffic (user data) among VPN sites

   This section describes the processes that an IP packet that is sent
   from one VPN site to another will go through. This is dependent on
   the way that IPsec is used. This document describes two possible ways
   to use IPsec in CE-based VPN implementations: IPsec in tunnel mode,
   and IPsec in transport mode.

   An IP packet that is sent by an IP device in a certain site and



De Clercq et al.        Expires September 2003                [Page 15]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   destined for an IP device in another site belonging to the same VPN,
   will be forwarded as follows.

   The device in the sending site sends an IP packet (using a private
   address space) on its LAN network. The next hop for this destination
   IP address will (at some point in time) be the site's CE device
   (according to the routing/forwarding in the VPN site). The processing
   by the CE device now is dependent on the implemented mode for IPsec.

   The use of IPsec in tunnel mode has the advantage that the complete
   range of SPD-checks remain usuable, but has the disadvantage that
   dynamic routing through the tunnels is not supported. The use of
   IPsec in transport mode secured IP-in-IP tunnels has the advantage
   that dynamic routing through the tunnels is fully supported, but has
   the disadvantage that the complete range of SPD-checks is not
   supported.

   Note that the following description is not meant to specify an
   implementation strategy; any implementation procedure wich produces
   the same results is acceptable.

   o IPsec in tunnel mode

      When IPsec is used in tunnel mode in this context, the IPsec
      driver in the CE device must do the following:

      - analyze the private IP packets arriving from within the site and
      select/setup an appropriate SA with the appropriate destination CE
      device.

      - authenticate and/or encrypt the private IP packet according to
      the (tunnel mode-specific) rules described in the SA, AND
      encapsulate the packet in an IPsec header AND encapsulate the
      packet in a new 'outer' IP header. This 'outer' IP header has the
      CE's non-private (i.e. routable in the SP's realm) IP address in
      the source IP address field and the destination CE's non-private
      (i.e. routable in the SP's realm) IP address in the destination IP
      address field.

   o IPsec in transport mode (see also [TOUCH] for a detailed
      specification)

      When IPsec is used in transport mode in this context, the CE
      device must first analyze the private IP packets arriving from
      within the site and select the appropriate destination CE, based
      on the routing/forwarding information. The CE device then
      encapsulates the private IP packet into a new IP packet (IP-in-IP
      encapsulation/tunneling), with the own non-private (i.e. routable



De Clercq et al.        Expires September 2003                [Page 16]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


      in the SP's realm) IP address in the source address field of the
      new header and the destination CE's non-private (i.e. routable in
      the SP's realm) IP address in the destination address field of the
      new header. The CE device then processes this new IP packet to its
      IPsec driver.

      The IPsec driver in the CE device must then do the following:

      - analyze the IP packets that have been IP-in-IP encapsulated and
      select the appropriate SA (based on the packet's outer header
      destination address).

      - authenticate and/or encrypt the private IP packet according to
      the (transport mode-specific) rules described in the SA and insert
      an IPsec header (according to IPsec in transport mode).


   The CE device then sends the IPsec packet to the PE device, and the
   IPsec packet will then be forwarded using 'normal' IP forwarding
   across the SP's network, based on the outer header's IP destination
   address, that is the destination CE's 'global' (i.e. routable in the
   SP's realm) IP address. The egress PE device will also only examine
   the outer IP header and send the IP(sec) packet to the destined CE
   device. The egress CE device will recognize itself as the destination
   node (the IP packet has the CE's IP address in the outer IP
   destination address field). The destination CE's IPsec driver will
   then, based on the appropriate Security Association (identified by
   the packet's SPI field in the IPsec header), perform IPsec
   authentication and/or decryption. Dependent on whether tunnel mode or
   transport mode IPsec is used, the packet will be decapsulated by the
   IPsec driver itself or sent to the IP-in-IP decapsulation function.
   The resulting (private) IP packet will then be further processed in
   the CE's IP forwarding table and send on the LAN network to the
   appropriate destination IP device.

   Note that IPsec tunnels might unintentionally terminate or break.
   This may have serious consequences if VPN membership and VPN routing
   information is not changed accordingly within the VPN. Indeed, the
   unnoticed termination of a VPN tunnel can result in the creation of
   black holes.

   This means that a mechanism must exist to monitor the state of the
   VPN tunnels. The following possibilities are identified: (i) the use
   of an IPsec-specific keep-alive mechanism [IPSEC-DPD]; (ii) when a
   routing protocol runs on top of the IPsec VPN tunnel (see section
   4.3.1), the routing protocol's keep-alive mechanism can serve that
   purpose; and (iii) the SP's management system could directly control
   the state of the VPN tunnels.



De Clercq et al.        Expires September 2003                [Page 17]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


6. Deleting an existing VPN site

   When the VPN customer decides to delete an existing site from a
   certain VPN, the customer first needs to inform the SP. The SP will
   change the VPN's configuration information accordingly in its VPN
   database.

   In analogy with the configuration update mechanisms described in
   section 3.2, multiple approaches exist to delete a specific site from
   an existing VPN.

   The SP can gracefully tear down all the concerned IPsec tunnels at
   both ends via either manual configuration or via a re-provisioning of
   all impacted CE devices with the used management system. IKE will
   then be used to tear down the IPsec security associations.
   Alternatively, the IKE sessions could be just timed out.

   Alternatively, when IKE-based discovery is used, the SP will update
   its VPN database and will provision the to-be deleted CE to tear down
   its IPsec connections via IKE. The peering CE devices will then
   update their VPN configuration information as described in section
   3.2.

   In the meantime, if a routing protocol runs on top of the VPN tunnels
   (and keep-alives are used), it will discover the topology change and
   update the necessary routing information automatically. If on the
   other hand, the SP management takes care of the inter-site routing
   information distribution, it will have to update the routes in the
   affected CEs when VPN tunnels are deleted.

7. Provider provisioned CE-based VPNs and NAT

   NAT provides address translation between two realms, such as between
   the private address space and the public address space. For site-to-
   site intranet VPN, the tunneled data usually remain in the same
   customer network address space. NAT is usually not required, unless
   two sites have overlapping address spaces. In that case, NAT can be
   used to solve the address-overlapping problem.

   In normal IPsec tunnel mode operation, after the user data is
   encapsulated by IPsec, the resulted packets are protected by packet
   encryption and/or authentication. Any further NAT changes on the
   packets will be detected at the receiving tunnel end, resulting in
   the packets being dropped. Currently the IPsec Working Group is
   working on a new mechanism so that IPsec packet can be safely NATed.
   Before the IPsec WG finalizes the IPsec over NAT standards, this
   draft assumes that there is no NAT function performed between the two
   IPsec tunnel end points.



De Clercq et al.        Expires September 2003                [Page 18]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   The scenario in which NAT applies is when the CE router serves a
   split stack function and provides both site-to-site VPN connection
   and direct Internet access. In this case, customer address space
   traffic is separated into two streams, one part for site-to-site
   private traffic and one part for direct Internet traffic. In this
   case the CE device performs NAT processing for the direct Internet
   traffic. The NAT function performed by the CE router takes care of
   the customer address space to public Internet address space
   conversion. Note that in the case that a certain site has both VPN
   and Internet access, a firewall should be configured at the site's CE
   device.

   See [CE-SOL] for more details concerning solutions for simultaneous
   VPN and Internet Access connectivity.

8. Extranet functionality with provider provisioned CE-based VPNs

   For the time being, this document deals mostly with intranet
   functionality provided by CE-based VPNs. The provisioning of an
   extranet service will affect the routing, tunnel topology and
   security features of the concepts described so far. The PP CE-based
   extranet VPN functionality is TBD.

9. Quality of Service

   TBD

10. Security Considerations

   The security aspects of what is presented in this document are
   implicitly discussed in most of the sections. This draft is for a
   large part focussing on security aspects.

   Note that the security of the mechanisms presented here is highly
   dependent on the following factors:

      - the security of the 'management channel', used by the management
      protocol to configure the VPN CE devices.

      - the security of the CE-device itself

      - the security aspects of the credentials: the IPsec credential
      must be generated, provisioned, updated, and stored securely

      - for a VPN with a complex topology, every tunnel must use the
      same grade of security strength, otherwise, a single weak link
      degrades the whole VPN




De Clercq et al.        Expires September 2003                [Page 19]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


11. Acknowledgements

   The authors would like to thank the following persons for their
   valuable contributions to this document: Mahadevan Iyer, Cheng-Yin
   Lee, Lars Eggert, Brian Gleeson, Archana Khetan, Paul Knight, Sankar
   Ramamoorthi, Eric Rosen, Michael Choung Shieh, Joe Touch, Eric
   Vyncke, S. Felix Wu, Yu-Shun Wang, Alex Zinin.

12. References

   [CE-SOL] De Clercq, J., Lee, C., Knight, P., "Provider Provisioned
   CE-based Virtual Private Network solution using IPsec and HTTP",
   draft-declercq-ppvpn-ce-based-sol-00.txt, Work in progress

   [DUFFY] Duffy, M., "Framework for IPsec Protected Virtual Links for
   PPVPNs", draft-duffy-ppvpn-ipsec-vlink-00.txt, Work in progress

   [FRAMEWORK] Callon, R. et al., "A Framework for Layer 3 Provider
   Provisioned Virtual Private Networks", draft-ietf-ppvpn-framework-
   0x.txt, Work in progress

   [GRE-CE] Khetan, A., et al., "Use of GRE for routing support in IPsec
   VPNs", draft-khetan-sp-greipsec, Work in progress

   [IKE] Harkins, D. and Carrel, D., "The Internet Key Exchange (IKE)",
   November 1998, RFC 2409

   [IPSEC] Kent, S., Atkinson, R., "Security Architecture for the
   Internet Protocol", November 1998, RFC 2401

   [IPSEC-DPD] Huang, G., Beaulieu, S., Rochefort, D., "A Traffic-Based
   Method of Detecting Dead IKE Peers", draft-ietf-ipsec-dpd-0x.txt,
   Work in progress

   [KNIGHT] Knight, P., Gleeson, B., "A Method to Signal and Provide
   Dynamic Routing in IPsec VPNs", draft-knight-ppvpn-ipsec-dynroute,
   Work in progress

   [LEE-DHCP] Lee, C.Y., "Customer Equipment Auto-configuration", March
   2002, work in progress

   [RFC2402] Kent, S., Atkinson, R., "IP Authentication Header",
   November 1998, RFC 2402

   [RFC2406] Kent, S., Atkinson, R., "IP Encapsulating Security Payload
   (ESP)", November 1998, RFC 2406

   [RFC2407] Piper, D., "The Internet IP Security Domain of



De Clercq et al.        Expires September 2003                [Page 20]


Internet Draft     draft-ietf-ppvpn-ce-based-03.txt           March 2003


   Interpretation for ISAKMP" November 1998, RFC 2407

   [RFC2408] Maughan, D., et al., "Internet Security Association and Key
   Management Protocol (ISAKMP)", November 1998, RFC 2408

   [TOUCH] Touch, J. and Eggert, L., "Use of IPSEC transport mode for
   Virtual Networks", draft-touch-ipsec-vpn-0x.txt, Work in progress


13. Authors' Addresses

   Jeremy De Clercq
   Alcatel
   Fr. Wellesplein 1, 2018 Antwerpen, Belgium
   E-mail: jeremy.de_clercq@alcatel.be

   Olivier Paridaens
   Alcatel
   Fr. Wellesplein 1, 2018 Antwerpen, Belgium
   E-mail: olivier.paridaens@alcatel.be

   Cliff Wang





























De Clercq et al.        Expires September 2003                [Page 21]