Network Working Group                       Tissa Senevirathne(Nortel)
Internet Draft                              Waldemar Augustyn (Nortel)
                                            Pascal Menezes (TeraBeam)
Category: Informational

                                            July 2001

        A Framework for Virtual Metropolitan Internetworks (VMI)
                  draft-senevirathne-vmi-frame-01.txt

Status of this Memo

   This document is an Internet-Draft and is in full conformance with
      all provisions of Section 10 of RFC2026 [1].

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that
   other groups may also distribute working documents as Internet-
   Drafts. Internet-Drafts are draft documents valid for a maximum of
   six months and may be updated, replaced, or obsoleted by other
   documents at any time. It is inappropriate to use Internet- Drafts
   as reference material or to cite them other than as "work in
   progress."
   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt
   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.


   For potential updates to the above required-text see:
   http://www.ietf.org/ietf/1id-guidelines.txt


1. Abstract


   This document identifies requirements, issues, and solutions for
   Virtual Metropolitan Internetworks. Different VMI models are
   discussed. The strengths and weakness of each model against customer
   requirements are analyzed.

Table of Content


  1. Abstract........................................................1
  2. Introduction....................................................3
  2.1. VMI Taxonomy..................................................3
  2.2. Network Service Model for Virtual Metropolitan Internetwork...4
  2.3. VPN vs. VMI...................................................4
  2.4. VMIS Requirements.............................................5
  3. Service Characteristics.........................................6
  3.1. Network Models................................................6
  3.1.1. Many-to-many network model..................................6
  3.1.2. Point-to-multipoint or hub and spoke network model..........6
  3.1.3. Point-to-point network model................................7


Seneviratne, et.al,     Expires- January 2002                       1


                 draft-senevirathne-vmi-frame-01.txt        July 2001


  3.2. Transparent Deployment........................................7
  3.3. Routable deployment...........................................7
  3.4. Demarcation Point.............................................8
  3.5. Scaling.......................................................8
  3.6. Data Security and Authentication..............................9
  3.7. Transport mechanisms in the core.............................10
  3.7.1. QoS and Traffic Engineering................................10
  3.7.2. Reliability................................................11
  3.7.3. Reconfiguration............................................12
  3.7.4. Broadcast and Multicast forwarding.........................12
  3.7.5. Layer 2 Address Learning...................................13
  3.7.6. End point discovery........................................13
  3.7.7. Address Transparency.......................................13
  3.7.8. Customer Packet immunity...................................13
  3.7.9. Geographically Distributed Sites, Long Haul................14
  3.8. VMI Services.................................................14
  3.8.1. Transparent L2 Networks....................................15
  3.8.2. Virtual Service Provisioning...............................15
  3.8.3. Class of Service...........................................15
  3.8.4. Bandwidth Allocation and Policing..........................15
  3.8.5. Packet Loss Reports........................................16
  3.8.6. Service Gradation..........................................16
  3.8.7. Service Gateway Functions..................................16
  3.8.8. Zero Cost Service Access...................................17
  3.9. Service Management...........................................17
  3.9.1. Infrastructure Configuration...............................17
  3.9.2. Operations Management......................................17
  3.9.3. Service Instantiation......................................17
  3.9.4. Automated Provisioning.....................................18
  3.10. Hierarchical Interconnection model..........................18
  3.10.1. Transport.................................................19
  3.10.2. Layer 2 Address Learning..................................19
  3.10.3. Reliability...............................................19
  3.10.4. Scalability...............................................19
  3.10.5. Administration............................................20
  3.10.6. Accounting................................................20
  3.10.7. Service Provisioning and end-point discovery..............20
  3.10.8. Virtual Service Provisioning..............................20
  4. Service Exchange Points........................................20
  5. Security Considerations........................................21
  6.References......................................................21
  7. Acknowledgments................................................21
  8. Author's Addresses.............................................21






Senevirathne, et.al.     Expires-January 2002                        2


                 draft-senevirathne-vmi-frame-01.txt        July 2001

2. Introduction


   Over the last several years, the available bandwidth in the core
   networks has increased dramatically, at the same time offering price
   per Mbyte has gone down. Most businesses today have multi site
   office complexes. Some of these organizations are exploring the
   opportunity of outsourcing their inter site enterprise network to
   Service Providers. With the opportunities in this segment
   increasing, Service Providers are building a new class of network
   service based on Metropolitan Area Networks. This new class of
   services intends to provide a virtual Internetwork for customers who
   wish to connect their geographically dispersed sites using public
   networks at very high speeds. In this document we define these
   networks as Virtual Metropolitan Internetwork or VMI. The service
   that provides VMI is defined as Virtual Metropolitan Internetwork
   Services, VMIS.

   Issues, requirements, architecture and business model of VMI service
   are significantly different from that of the better known Virtual
   Private Networks (VPN) or traditional Internetworks. This document
   intends to provide a framework for VMIS and documents issues,
   requirements, architecture and business model when providing VMIS.

   In the past, both the service and the business models were mainly
   dictated by the network infrastructure installed by the Service
   Providers. Here, we envision the opposite relationship. A ProviderÆs
   business model and its customer service requirements would derive
   the network infrastructure and its capabilities. The architecture of
   a ProviderÆs infrastructure consists of a backbone transport
   mechanism, network management and accounting, traffic engineering
   and QoS capabilities etc. Hence, with a given infrastructure a
   Provider may only be able to satisfy some subset of requirements and
   business models. In this document we present, different
   infrastructure models and identify the types of requirements and
   business models that can be satisfied within the architecture.

2.1. VMI Taxonomy


   In this section we define Taxonomy and Analogy of Virtual
   Metropolitan Internetworkto facilitate considerations of problems at
   hand.

   o VMI Infrastructure. A collection of VMI Service ProviderÆs
     equipment and methods used to implement VMI service. All
     VMI networks share a common infrastructure.

   o VMI Network. A virtual network service offered by a VMI
     Service Provider, as seen by a Customer. Multiple,
     mutually exclusive networks exist within the same, shared
     infrastructure.

   o VMI Service Management. A VMI Service ProviderÆs means


Senevirathne, et.al.     Expires-January 2002                        3


                 draft-senevirathne-vmi-frame-01.txt        July 2001

     of managing operations of VMI networks.

   Given the above taxonomy, VMI can be viewed as a stack of disks on a
   base or "Tower of Disks". Each of the disks represents a separate
   virtual network. The base represents the Infrastructure. The pin in
   the middle represents the Service Management.


                                |
                        -----------------
                       |                 | VMI-2
                        -----------------
                                |
                                |
                                | <--------- Service Management
                                |
                                |
                        -----------------
                       |                 | VMI-1
                        -----------------
                                |
                      ---------------------
                     |                     | Infrastructure
                      ---------------------



2.2. Network Service Model for Virtual Metropolitan Internetwork


   Presently there are several models that are used to provide VMI
   Services. Each of these models may suit customers based on their
   requirement. Some service providers exclusively support only one
   model; such providers may not be able to accommodate some customer
   requirement. In this section we summarize possible service models
   that different customers may require.

   o Full mesh, many-to-many network model

   o Partial mesh model

   o point-to-multipoint (hub and spoke) model

   o point-to-point model

   o hierarchical, multi domain model


2.3. VPN vs. VMI


   Understanding the key differences between VPN and VMI is important.
   This understanding facilitates one to appreciate the problems,
   issues, and strength and business model of VMI clearly.


Senevirathne, et.al.     Expires-January 2002                        4


                 draft-senevirathne-vmi-frame-01.txt        July 2001


   o Service. In simple terms, VMI is a service, while VPN is a
     technology. A VMI specification is broader than that of a VPN,
     covering areas that VPNs typically consider beyond their scope.

   o User/Provider Context. VMI recognizes the demarcation between the
     User and the Provider and the consequences of asymmetric
     responsibilities and expectations. The Provider is the entity that
     offers its services to the receiving entity, the User.

   o Gateway Functions. VMI networks, similarly to VPNs, are private,
     intended for interconnecting sites belonging to the same
     administrative entity.  Unlike VPNs, however, VMIs define optional
     Provider Gateway Functions, which are methods for secure access to
     VMI networks for the purpose of supplying additional network
     services.


2.4. VMIS Requirements


   VMIS requirements derive the service model. VMIS requirements can be
   broadly classified into several areas.

   o number of sites that required to be connected.

   o Required network topology; hub and spoke with all sites connecting
     to the data center or fully meshed emulated Local Area Network or
     combination.

   o Type of connectivity required; Layer 2 or IP routing.

   o Traffic engineering and QoS requirements; end-to-end guaranteed
     bandwidth.

   o Security; traffic separation, payload encryption and
     authentication.

   o Billing requirements; wholesale models vs. per byte billing.

   o Service Level Agreement (SLA); Time of the day bandwidth vs. flat
     bandwidth

   o Traffic reporting and accounting; statistics, exceptions, faults,
     etc.

   o Gateway Functions; option for the Provider to offer extra services
     on the private VMI networks.







Senevirathne, et.al.     Expires-January 2002                        5


                 draft-senevirathne-vmi-frame-01.txt        July 2001

3. Service Characteristics


3.1. Network Models


3.1.1. Many-to-many network model


              ------                         -------
   Customer 1|      |                       |       | Customer 1
   --------- | PE A |                       | PE B  |------
    Site A   |      |       ---             |       | Site B
              ------       |   |             -------
                |    \     |   |            /   |
                |     -----|   | -----------    |
                |      ----|---|--------        |
                |     |                 |       |
                |     |                 |       |
                 -----|  Metro Core     |-------
                      |                 |
                       ----|---|--------
                           |   |    |
                           | Virtual|Internet
                            ---     |
                             |      |
                             |      |
                          -------   |
            Customer 1   |       |  |
          -------------- | PE C  |--
            Site C       |       |
                          -------

   Simple many-to-many Virtual Metropolitan Internetwork

   Many-to-many VMI network model is ideally suited for customers who
   want to have multiple sites connected, via a public network, to form
   a one single network. In this topology, either they extend their
   Local Network as a layer 2 connectivity or may choose to introduce
   to create a routed virtual core. Many-to-many Virtual Metropolitan
   Internetwork model is particularly useful for the environment where
   service offering is to extend the customer's enterprise network
   using the shared network. Many-to-many VMI model is by far the most
   general model. Using this model, one may construct any of the other
   models.


3.1.2. Point-to-multipoint or hub and spoke network model


   The point-to-multipoint model is a subset the of many-to-many-point
   model. Point-to-multi-point model may be used for services that
   requires connectivity to a centralized site from several remote
   sites. Such a service is very similar to more traditional frame
   relay deployment. Unlike frame relay networks, VMI point-to-



Senevirathne, et.al.     Expires-January 2002                        6


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   multipoint network may provide additional services that are beyond
   the capabilities of classical frame relay networks.


3.1.3. Point-to-point network model


   The point-to-point network model is the simplest of all VMI network
   models. Due to its simplicity, point-to-point model can be used for
   a variety of services that could not be provisioned using other
   service models. As an example, providing video or voice services
   require strict end-to-end timing. Providing such end-to-end timing
   using other service models may not be possible. VMI point-to-point
   services are  classified into two categories:

   o The VMI services that require data services, either in packet or
     cell form are called Data service VMI (DSVMI).

   o The VMI service that require emulation of some characteristics of
     a transport media, such as SONET are called, in this document,
     Circuit Emulation VMI (CEVMI).


3.2. Transparent Deployment


   In the transparent model, customers may consider the VMI as a
   virtual and transparent extension to their Local Network. In other
   words; all the sites appear as access points in the same Local
   Network.

   The Transparent point-to-multi-point and many-to-many models are
   mainly used by the customers who wish to extend the Local Network
   over the VMI without any network layer address translation. Due to
   the anatomy of these models, network edge devices that provide
   interface between the customer sites may require to implement
   specific forwarding policies. In more traditional frame relay
   networks; when deploying hub and spoke models; concepts such as
   split horizon may be implemented to prevent reflection of broadcast
   traffic. However when providing transparent deployment model
   customers may wish to have connectivity from remote site to a remote
   site.  Cost concern customers who do not wish to purchase fully
   meshed many-to-many network and yet achieve many-to-many
   reachability may also use point-to-multi-point deployment. VMI
   service providers who wish to offer transparent deployment model
   MUST have forwarding policies defined to properly forward inter-site
   traffic. A simple such policy is to treat each virtual connection as
   a separate circuit. A more complex forwarding policies may require
   to be defined if all the virtual links to be treated like a single
   interface and yet achieve inter-site forwarding.


3.3. Routable deployment



Senevirathne, et.al.     Expires-January 2002                        7


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   In the routable model the VMI access points may be considered as
   belonging to the same sub-network. Thus VMI edge devices providing
   routing functionality. However, customers may wish to maintain their
   private address space shielded from the VMI control plane. At the
   same time, VMI Service Providers may not wish to maintain customer
   address space. As such, VMI infrastructure may require having
   technologies that could transport customer's traffic using some
   local addressing and transport mechanisms.


3.4. Demarcation Point


   The VMI service recognizes the need for proper demarcation between
   the administrative entities involved in the service.  Each entity
   participating in the service must be able to extend its operations
   management and monitoring system to guarantee the delivery of its
   obligations.

   o Customer-Provider. This type of relationship is sometimes referred
     to as ôretailö.  In this case, the Provider must have the means
     for proving with reasonable certainty that the service, as
     described in SLA, is provided throughout the Providers network up
     to the demarcation points.  These demarcation points are typically
     devices owned by the provider that directly connect to customerÆs
     equipment.  In cases of transparent deployment of VMI services,
     these devices can be very simple supporting little more than Layer
     1 loopbacks.   In cases of routed deployment of VMI services, the
     devices are considerably more complex.  Typically, these are
     switches or routers that are Provider owned or managed but reside
     on customer premise.

   o Provider-Provider. This type of relationship is sometimes referred
     to as ôwholesaleö.  In this case, too, the essential objective is
     to prove with reasonable certainty that the service  provided to
     another Provider is operational throughout the network up  to the
     demarcation point.

   o Service domain-Service domain. Even within a single organization,
     it is useful to define demarcation points between major service
     domains. This type of demarcation is similar to Provider-Provider.


3.5. Scaling


   It is important that the VMIS architecture is able to support a
   large number of customers with each customer having multiple sites
   requiring  unconstraint connectivity.

   Typically, scalable solutions attempt to place per-VMI parameters in
   the edge devices while keeping core networks free from the need to
   maintain any state or VMI specific data. The basic premise behind
   this strategy is that the infrastructure can grow, potentially


Senevirathne, et.al.     Expires-January 2002                        8


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   indefinitely, by adding edge devices with all per-VMI data localized
   in these devices.

   This strategy works well if the solution can further localize the
   per-site data to the devices directly attaching to the customer
   site. As an example if an edge device needs to learn MAC addresses
   of the nodes participating in a VMI, it is more scalable if the
   learning is limited to the MAC addresses originating in the site
   directly attached to the device.  Otherwise, if that device has to
   learn all MAC addresses, including those originating at other sites,
   the device may be overwhelmed by some large sites, perhaps connected
   via large edge devices.


3.6. Data Security and Authentication


   In VMI Services, customer's data is exposed to the shared network at
   the Provider side. A VMI service provider might use services from
   other upstream providers for global backbone connectivity. Hence,
   privacy may be an issue. Traditionally, it has been the
   responsibility of the customer to use their own technology, such as
   IPSec, to provide privacy, but with VMI service offerings of speeds
   in the range of 1Gbps, the performance of CE based solutions becomes
   an issues. It is quite attractive if the VMI service provider can
   offer encryption and security. It is quite possible, a customer
   sharing the infrastructure with the competitors. Hence, it is
   required that customers have ability to protect and authenticate
   data.

   o Simple Traffic Separation. In many situations a simple, virtual
     separation of traffic may be sufficient. This style of security,
     frequently referred to as Frame-Relay-like, has gained wide
     acceptance in the market place. The provider is entrusted with
     making sure VMI traffic remains inaccessible to other customers.
     The level of security protects against attacks from other
     Customers, but not from the Provider itself.

   o Advanced Security. For more sensitive applications, more secure
     means, involving encryption and authentication techniques, are
     necessary. Essentially there are two approaches to solve this. In
     the first approach, customers may choose to install on premise
     data encryption devices. Here, there needs to be an out of band
     management connectivity between encryption devices. In the second
     approach Service provider edge devices may themselves provide data
     encryption and authentication.

   In the case of many-to-many connectivity, Security Association
   establishment and Encryption key distribution become more complex.
   Security Architecture for many-to-many networks is presented in [2].
   It discusses security architecture of common access networks based
   on a case study related to VMI.



Senevirathne, et.al.     Expires-January 2002                        9


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   The security solution for point-to-point VMI depends on the type of
   VMI service provided. Hence, the security solution provided for
   DSVMI may be different from the CEVMI. Within the DSVMI, based on
   the payload types different security solution may be needed. As an
   example, when transporting IP only payloads one may choose IP Sec
   solution whereas when transporting Layer 2 payloads, service
   providers may require to provide a security solution that is
   protocol agnostic. Regardless, some customers may wish to have an on
   premise security solution. Especially, CEVMI customers may use a
   front-end scramblers/encryptors.


3.7. Transport mechanisms in the core


   There are multiple transport mechanisms capable of providing the
   necessary connectivity. Choice of the appropriate transport depends
   on the set of requirements that a given Service Provider wants to
   address. As an example, a Provider may choose to have a complete
   Layer 2 topology with simpler layer 2 protocol to provide redundancy
   and reconfiguration. On the other hand, if a  Provider chooses to
   support the entire set of requirements, he may choose more
   sophisticated protocol such as MPLS. In theory, it is possible that
   multiple transport mechanisms are supported in the core.

   It is possible that a VMI Service Provider enrolls a customer where
   all the sites are NOT within the footprint of the provider. This
   would be the case if the VMI service provider uses one set of
   transport mechanisms (e.g. MPLS) within its footprint and another
   (e.g. IP-Sec, GRE, etc.) for service consistency through some public
   backbone (Internet). Obviously various usage policies (QoS) would
   have to be different based on the available resources.

   When providing a CEVMI service, the transport method should be able
   to tunnel the circuits over the VMI infrastructure. The transport in
   the VMI core should be capable of supporting emulation of Time
   Division Multiplexed (TDM) circuits. Generalized MPLS presented in
   [], provides a solution based on MPLS. The chosen transport method
   should be able to provide required reliability, end-to-end timing,
   service provisioning etc., as agreed upon in the SLA.

   When providing a DSVMI service, at least the following issues must
   be taken into consideration when selecting a Transport Mechanism:
   end-to-end QoS, type of payload (Layer 2/ IP), reliability,
   reconfiguration.


3.7.1. QoS and Traffic Engineering


   Customers may now wish to have the same QoS and Traffic Engineering
   characteristics as they do in a locally connected network. Hence,
   now the problem is not providing end-to-end QoS service but
   providing network wide QoS and traffic engineering. In general, a


Senevirathne, et.al.     Expires-January 2002                       10


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   given flow, whether Layer 2 or 3, is point-to-point in nature. The
   broadcast and multicast nature of local networks can be emulated
   using set of fully meshed point-to-point networks. This approach
   simplifies the problem and reduces the issue to providing required
   QoS level per point-to-point link. If all sites are symmetric in QoS
   and Traffic Engineering requirements, the management of QoS and
   parameters becomes simple. However, in practice, different sites may
   have varying Traffic engineering parameters. In such configurations,
   it is important that participating nodes have knowledge of the
   requirements of the other nodes and impose shaping and policing
   appropriately.

   Consider the deployment scenario below. When, node A is sourcing
   broadcast traffic it is required not to over-subscribe the link
   between A-C, which is only 50% incoming rate of ingress port A'.
   Hence, it is essential that participating nodes are capable of
   performing egress traffic shaping, as required.

                       B(node)
                      / |
        5 Mbits/sec  /  |
                    /   |
                   /    |
   5 Mbits/sec    /     |
   --------A'-- A(node) | 5 Mbits/sec
                  \     |
                   \    |
                    \   |
      2.5 Mbits/sec  \  |
                      \C(node)

   The discovery of these parameters can be either via an automatic
   process or via manual configuration. It is possible that some of
   these information can be piggy backed to routing protocols such as
   OSPF and BGP and advertised to other devices, thereby reducing the
   management burden. Alternatively, one may have a system wide
   management system that would provision the services.

   The QoS and Traffic Engineering service in Circuit emulation VMI
   (CEVMI) may require satisfying different Traffic Engineering
   parameters than Data Service VMI (DSVMI). Exact set of parameters
   depends on the Type of circuit emulation service offered and
   customer requirements. As an example, if the ATM circuits are
   emulated customer may wish to have only UBR service or CBR service.
   Another example is a customer who wishes to extend Frame Relay
   network over the VMI. When offering Circuit Emulation services,
   transport protocol in the VMI core should be capable of satisfying
   different requirements of different categories of circuits.


3.7.2. Reliability




Senevirathne, et.al.     Expires-January 2002                       11


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   Different customers may require different levels of reliability. The
   reliability is here defined as protection against  service
   infrastructure failures. The protection against such failures should
   be built into the infrastructure and backbone transport protocols.
   The protection can be defined in many flavors such as
   Primary/Secondary, Fast Reroute. Protection can also have additional
   attributes such as time to restore (50 msec) as well. As an example,
   MPLS fast reroute and backup path protection may be needed for some
   customers who mandate higher level of reliability. The methods for
   providing desired level of reliability for Circuit Emulation VMI
   (CEVMI) may be different from methods for providing desired level of
   reliability for Data Service VMI (DSVMI). When providing certain
   Circuit Emulation VMI, providers must maintain timing requirements.
   The committed levels of reliability may be included in Service Level
   Agreement. Thus, Service Providers are legally bound to satisfy
   these requirements. Inability to provide agreed upon reliability
   levels may have financial and/or legal ramifications. At the same
   time, these levels of protection may be essential for operation of
   their applications.

   Providing redundancy in point-to-multipoint deployments may be much
   simpler than many-to-many deployment model. It is anticipated that
   when purchasing a SLA, the redundancy is specified for the entire
   VMI service rather than per site basis. Hence provisioning such
   redundancy in a many-to-many network may cause serious complexities
   and expenses to the VMI service providers. [wa: I merged the
   reliability concepts mentioned in this paragraph with the text
   above.


3.7.3. Reconfiguration


   It is important to provide uninterrupted services to the customers
   during maintenance and upgrades. Thus, methods are essential to be
   built into the infrastructure for graceful reconfiguration during
   maintenance. Unlike, redundancy provided to address required
   reliability, the redundancy required during maintenance is temporary
   and may be configured manually.


3.7.4. Broadcast and Multicast forwarding


   In general, not all links in a network have the same transport layer
   characteristics. Broadcast, multicast packets are required to be
   replicated to multiple links with different characteristics. When
   providing Layer 2 connectivity the Provider Edge devices are
   required to replicate unknown unicast packets as well. Hence, the
   Provider Edge devices MUST have methods to replicates packets over
   different transport layer characteristics.

   In cases of asymmetric topologies, such as point-to-multipoint,
   broadcast and multicast forwarding issues are different at specific


Senevirathne, et.al.     Expires-January 2002                       12


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   points in the networks.  For example,at the remote sites of a hub-
   and-spoke network, the forwarding policies are similar to those
   applied to point-to-point networks, but at the hub site forwarding
   policies becomes more complex based on whether remote-site to-remote
   site connectivity is needed and whether transparent deployment model
   or routable model is in place. Such forwarding polices may be
   include as part of the Service Level Agreement (SLA).


3.7.5. Layer 2 Address Learning


   If the transport mechanism used in the core is based on layer 2, it
   is possible that the devices in the middle are required to learn a
   large number of MAC addresses. In addition, they may be required to
   perform forwarding based on different customer contexts. If this is
   chosen then the MAC Forwarding data MUST remain unique per VLAN to
   eliminate multiple sites having the same MAC address (Decnet, SNA
   etc..)

   If one chooses to tunnel layer 2 traffic across the core, the
   address learning and forwarding decisions  should be limited only to
   the Provider Edge devices. Thus helping to scale better. Most
   devices have a finite number of MAC addresses they can support. A
   Denial of Service attack can be easily performed by blasting large
   amount of MAC addresses into the network. Hence, PE devices should
   have methods to limit the number of MAC addresses learned from a
   given port.


3.7.6. End point discovery


   There may be the need for the end points of participating sites to
   establish connectivity. These end-points may be either manually
   configured or learned via some protocol. OSPF Opaque LSA are one
   popular protocol extension that is used to learn end points of the
   participating devices. [3] Presents how OSPF Opaque LSA may be used
   to discover different Layer 2 reachability.


3.7.7. Address Transparency


   A customer considers a VMI to be a totally transparent network.
   Packets, headers and payload, are transferred unaltered.  Broadcast,
   and multicast addresses are recognized and treated accordingly. VLAN
   tags are preserved, if present.


3.7.8. Customer Packet immunity


   VMI services do not rely on the global uniqueness of customersÆ
   addresses, such as MAC addresses, or any addresses. Of course, the
   addresses must remain consistent within each VMI for useful service.


Senevirathne, et.al.     Expires-January 2002                       13


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   Nevertheless, the ProviderÆs infrastructure must be immune to the
   Customer misaddressed, invalid, corrupted, or maliciously altered
   packets.


3.7.9. Geographically Distributed Sites, Long Haul


   In general, VMIs are intended for connectivity between sites located
   within a metropolitan reach. However the notion of metropolitan
   reach may not necessarily imply geographic proximity.  Geographic
   distances may vary significantly, in many cases exceeding the limits
   of the underlying technologies used to build the VMI infrastructure.
   In these cases, additional technologies may be employed for the sole
   purpose of spanning wide geographic distances.  These technologies
   are called long haul.


3.7.9.1.  Native Long Haul


   Here, we define Native Long Haul as the ability to extend the
   geographic reach  while keeping the same, common infrastructure and
   preserving the service management of the original local network.
   Referring to the analogy of the Tower of Disks, if the base of the
   tower and its pin remain the same, the long haul extension is
   native. As an example, if the local network is based on optical
   technology and if there exists an optical technology based means to
   connect the sites that are located outside the local domain, but
   comprise the same infrastructure and are managed by the same
   operations system, that is considered a native long haul. On the
   other hand if the sites are connected using some other technology
   (such as frame relay in the otherwise all Ethernet network) then it
   is considered a non-native long haul.


3.7.9.2.  Non-native long haul


   It is possible to extend the geographic reach using non-native long
   haul methods. However, these methods must meet the same requirements
   applicable to native methods, such as end-to-end traffic
   engineering, reliability etc.

   Native long haul may not always be best suited for all network
   models. For example, a many-to-many connectivity through a native
   long haul may not be financially viable. In these cases, a non-
   native solution may be a good alternative.


3.8. VMI Services


   For each customer a VMI represents a network that is tailored
   specifically for the needs of that customer.  For a customer, a VMI



Senevirathne, et.al.     Expires-January 2002                       14


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   services are defined by the set of characteristics a supplied VMI
   has.


3.8.1. Transparent L2 Networks


   A customer may wish to perform his own assignments and
   administration of the addressing, including layer VLAN tagging. The
   ability to perform such administration by the customers creates a
   truly Virtual Enterprise network extension. All the building blocks
   in the VMI architecture, including the transport, need to
   participate in order to build a VMI service with virtual service
   provisioning requirement. A customer's VLAN tag must be able to be
   encapsulated in the transport mechanism, to provide transparency.


3.8.2. Virtual Service Provisioning.


   A Provider may offer a method for the Customer to modify the
   characteristics of its VMI networks in an automated manner. A
   typical use would be requesting more bandwidth or reallocating
   priorities.


3.8.3. Class of Service


   A VMI recognizes a range of traffic categories.  The categories have
   significance only within a given VMI.  The availability and
   allocation of different traffic categories is defined by a Service
   Level Agreement and enforced by the Providers VMI Infrastructure.

   Typically, a VMI distinguishes guaranteed delivery traffic and best
   effort traffic. Within each of these two categories, one or more
   priorities may be further distinguished.  The guaranteed category of
   traffic may also allocate particular latency and jitter commitments
   to one of its stated priorities.


3.8.4. Bandwidth Allocation and Policing


   The amount of traffic bandwidth allowed to flow within a VMI, total
   as well as for each traffic category, is determined by a SLA
   agreement.  The provider implements traffic policing mechanisms to
   enforce the SLA.

   For the traffic coming into the ProviderÆs network, the maximum
   bandwidth limits are enforced.  The customer may implement traffic
   shaping mechanisms to avoid unnecessary packet drops.

   For the traffic leaving the ProviderÆs network, the maximum
   bandwidth limits are enforced.  The Provider may offer a back-off
   signaling mechanism to aide traffic shaping for the source.


Senevirathne, et.al.     Expires-January 2002                       15


                 draft-senevirathne-vmi-frame-01.txt        July 2001



3.8.5. Packet Loss Reports


   A VMI service may provide an integrated packet loss reporting system
   for signaling legitimate packet drops.  A legitimate packet drop is
   defined as a discard of a customerÆs packet for reasons other than
   network failure. The typical reasons are:

   o Source rate higher than VMI SLA
   o Destination VMI SLA below source rate
   o Destination flooded by traffic from many sources
   o Best effort congestion

   Packet drops due to network failures are not reported.  A packet
   drop due to congestion for a guaranteed traffic is considered a
   failure.


3.8.6. Service Gradation


   Not all VMI deployments will support all VMI features.  If certain
   features are not supported by the infrastructure, the related SLAs
   would denote these exceptions.  For example, it is conceivable that
   in certain situation, the infrastructure would not be redundant but
   still provide useful, perhaps less expensive, VMI service.
   Similarly, guaranteed service may not be offered by many
   infrastructures but the remaining best effort service could still be
   offered as a VMI service.

   It is useful to list major gradation of VMI service capabilities

   o High or standard reliability
   o Ability to offer guaranteed service
   o Limited distance scope or global reach
   o Internet gateway function availability
   o Service gateway function availability
   o Packet drop reporting
   o Traffic encryption or simple separation

   Conversely, it is useful to list the minimum set of capabilities

   o L2 private network service
   o Full support for L2 unicast, broadcast and multicast
   o Preservation of L2 headers, including VLAN tags if present
   o Ingress/Egress bandwidth policing
   o Full customer traffic separation
   o Full MAC address space independence


3.8.7. Service Gateway Functions



Senevirathne, et.al.     Expires-January 2002                       16


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   A VMI is quintessentially a private network service. Even if a
   Providers offers a services for certain customers, it remains a
   private network service. VMI allows an orderly access to customersÆ
   VMI network for the purpose of supplying value added services. One
   important service of that kind is a gateway to the public Internet.
   Other services include DHCP, local DNS, NAT, etc.

   o Public Internet gateway

   o Value added service gateways


3.8.8. Zero Cost Service Access


   A VMI service may be offered to the customer without a need for any
   devices located on the customer premises.  This is called zero cost
   service access, the Provider can provision and modify the service
   without the need to install or modify any equipment on the customer
   premises.  A physical connectivity means, e.g. a fiber cable, must
   exist.


3.9. Service Management


   The service management capabilities and access methods are common to
   all devices comprising VMI infrastructure. The operations of the
   entire VMI service system relies on the coherency of Service
   Management. It is represented by the pin in the Tower of Disks.


3.9.1. Infrastructure Configuration


   The infrastructure for VMI services, the base of the Tower of Disks,
   is a collection of a wide range of network equipment. Typically, a
   combination of SNMP, TL1, and CLI tools are used for setting this
   equipment up. Different methods of configuration can be offered by
   different equipment vendors.


3.9.2. Operations Management


   Once an infrastructure is set up, the VMI services are controlled by
   an Operations Management system. This system represents all methods
   and protocols that are used to provision, monitor, and maintain VMI
   services.  The methods and protocols must be common to all equipment
   participating in the service.


3.9.3. Service Instantiation





Senevirathne, et.al.     Expires-January 2002                       17


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   It is possible to create several instances of VMI service
   infrastructures sharing the same equipment within a network
   provider.  For example different wavelength can be dedicated to
   different VMI service instances in a WDM systems. Similarly, SONET
   transport system can be partitioned based on STS-1 frames for
   different VMI systems.  This capability extends wholesale
   opportunities for Providers.


3.9.4. Automated Provisioning


   Depending on the deployed transport mechanism, it may be necessary
   to perform complex provisioning along the paths taken between the
   sites. Manual provisioning may not be feasible. In such  situations,
   use of automatic discovery and signaling protocols facilitates
   network management. The automatic signaling of provisioning must
   have the ability to scale across global networks and be supported by
   all involved providers. A hierarchical system may simplify the
   solution. Additionally, the signaling layer must have visibility
   into path creation to enforce any required constraints.


3.10. Hierarchical Interconnection model


   In situations where connectivity spans different administrative
   domains, or the complexity of the infrastructure warrants
   subdivisions, a hierarchical interconnection model is applicable.
   Here we assume that there is a remote service provider that has
   agreed to participate in VMI services.

                                ^
                                | to pop(4L)
                                |
                            ----------
                           |          |
                   <-------| pop(3L)  |---------> to other pop(3L)
                            ----------
                         /              \
                        /                \
                       /                  \
                      /                    \
            ---------                        ----------
           |         |                      |          | to other
     <-----| pop(2L) | -------------------- |  pop (2L)|----->
            ---------                        ----------  pop (2L)
                |  UNION                         |  UNION
    ---------------------------             -------------------------
   |                           |           |                          |
   |  --------        -------- |           |                          |
   | |        |     |         ||           |                          |
   | | pop(1L)| ----| pop(1L) ||           |                          |
   |  --------        -------- |           |                          |


Senevirathne, et.al.     Expires-January 2002                       18


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   |     |              |      |           |                          |
   |  ---------     ---------  |           |                          |
   | | Local   |   | Local   | |           |                          |
   | | confederate | confederate           |                          |
   | |         |   |         | |           |                          |
   |  --------      ---------  |           |                          |
    ---------------------------             --------------------------

   Sample Hierarchical Interconnection model

   The above diagram depicts a hierarchical interconnection model. The
   local confederates are the more traditional VMI service providers.
   The first level POP connects multiple of local confederates.
   collection of local confederates that are below a single first level
   pop is considered as a "UNION". The Unions are interconnected using
   second level POP.


3.10.1. Transport


   The transport method outside the confederates should be able to
   provide required services. This may include the end to end traffic
   engineering, tunneling of non-native data, reliability and
   resilience. In order to have a non-fragmented consistent network
   service, all devices are required to support common and adequate
   transport mechanisms. MPLS appears to be a promising choice for this
   environment. However, this document does not limit the selection of
   transport methods.


3.10.2. Layer 2 Address Learning


   In the hierarchical interconnection model, the device outside the
   local confederate should not perform forwarding based on customer
   MAC addresses. Instead, when providing layer 2 services, forwarding
   should be performed using some other mechanism. In other words,
   devices outside the local confederate provide tunneling services.


3.10.3. Reliability


   The reliability of the local confederate depends on the chosen VMI
   model and SLA. The devices outside the local confederate should be
   able to provide required reliability. This may include the equipment
   level and transport methods.


3.10.4. Scalability


   The network topology outside the local confederate requires to scale
   to potentially large amount of inter site transit tunnels and
   traffic.


Senevirathne, et.al.     Expires-January 2002                       19


                 draft-senevirathne-vmi-frame-01.txt        July 2001



3.10.5. Administration


   With the type of diversity in the hierarchical interconnection
   model, it is possible that the local confederates belong to
   different autonomous systems. Hence, the routing, control plane and
   discovery should have knowledge of the inter AS nature.


3.10.6. Accounting


   There should be a comprehensive accounting system so customers can
   be eventually charged properly. Flat service model or bandwidth
   wholesale may not work well in the hierarchical network model.


3.10.7. Service Provisioning and end-point discovery


   Due to the diversity of end points, system wide network provisioning
   may be administratively prohibitive. It may be better if the
   provisioning is performed at the end points and control protocol or
   signaling method be used to automatically provision the service
   requirements. A good example for this is MPLS RSVP-TE and CR-LDP. On
   the other hand it is important to discover the endpoints. This may
   be achieved either using IGP or some EGP protocol. [3] presents how
   end points may be discovered using OPSF opaque LSA. [4] presents how
   these discoveries are done across Autonomous Systems using BGP-MP
   extensions.


3.10.8. Virtual Service Provisioning


   Some customers require a SLA that allows self-provisioning and
   administration of the inter site VMI. Virtual Service provisioning
   requires that all-participating providers and equipment has
   capabilities to support Virtual Provisioning. Supporting such a SLA
   in a multi-vendor, multi-owner, geographically dispersed
   infrastructure may be a difficult task.


3.11. Service Exchange Points


   Service exchange (SXP) or Internet Exchange Point (IXP) is a new
   Internet service-provisioning model.  IXP is a logical many-to-many
   service where participating customers can connect to each other as
   if they are connected a to Local Area Network. Customers of IXP are
   generally service providers who wish to connect to other service
   providers.




Senevirathne, et.al.     Expires-January 2002                       20


                 draft-senevirathne-vmi-frame-01.txt        July 2001

   IXP MUST have capabilities to restrict peering between participants
   to a chosen user group. Logically, each close user group (CUG) in
   IXP is identical to a Layer 2 NBVPN domain, where each end
   participant representing a separate end point.

   In theory IXP is Layer 2 NBVPN deployment. However, IXP may have
   some variation in the requirements. As an example, IXP may charge
   customer or each end-point of the CUG. Where as traditional Layer 2
   NBVPN may charge per entire Layer 2 NBVPN service.


5. Security Considerations


   This document does not discuss specific security solutions. This
   document, however identify and state the security requirements in
   Virtual Metropolitan Internetworks.


6.References


   1  Bradner, S., "The Internet Standards Process -- Revision 3", BCP
      9, RFC 2026, October 1996.

   2  Senevirathne, T., and et.al., "Security Architecture for Common
      Access Data Networks - A Case Study based on Virtual Metropolitan
      Internetwork", Work In Progress, December 2000.

   3  Senevirathne, T., and Billinghurst, P., "Distribution of 802.1Q
      VLAN information using OSPF Opaque LSA", Work In Progress,
      October 2000.

   4  Senevirathne, T., and et.al., "Distribution of 802.1Q VLAN
      information using BGP-MP extensions", Work in Progress, November
      2000.


7. Acknowledgments


   The on going work at the IETF and the work presented in the cited
   references has greatly influenced the work presented in this
   document. Authors also wish to extend appreciations to their
   respective employers and various other people who volunteered to
   review this work and provided comments and feedback.


8. Author's Addresses


   Tissa Senevirathne
   Force10 Networks
   1440 McCarthy Blvd,
   Milpitas, CA 95051
   Phone: 408 965 5103


Senevirathne, et.al.     Expires-January 2002                       21


                        draft-vmi-frame-01.txt               June 2001

   Email: tsenevir@hotmail.com

   Waldemar Augustyn
   Nortel Networks
   600 Technology Park
   Billerica, MA 01821
   Phone: 978 288 4993
   Email: waldemar@nortelnetworks.com

   Pascal Menezes
   TeraBeam Networks
   2300 Seventh Ave
   Seattle, WA 98121
   Email: Pascal.Menezes@Terabeam.com









































Senevirathne, et.al.    Expires-December 2001                      22


                        draft-vmi-frame-01.txt               June 2001


Full Copyright Statement

   "Copyright (C) The Internet Society (2001). All Rights Reserved.
   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph
   are included on all such copies and derivative works. However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into







































Senevirathne, et.al.    Expires-December 2001                      23