[Search] [pdf|bibtex] [Tracker] [Email] [Nits]

Versions: 00                                                            
Network Working Group                                              Y. Gu
Internet-Draft                                                    Huawei
Intended status: Standards Track                                  Y. Fan
Expires: December 10, 2011                                 China Telecom
                                                           June 12, 2011


            Policies and dynamic information migration in DC
                  draft-gu-opsa-policies-migration-00

Abstract

   Virtualization and Virtual Machine (VM) migration provide Data Center
   with feasibility and improves the utilization of limited physical
   resource, e.g. switches/routers, servers and links.  Meanwhile, a
   variety of policies (e.g.  ACL, firewalls, load balancers, IPS and
   QoS) are deployed in Data Center to improve system security and
   gurantee SLA.  Those polices are executed by rules configured or
   generated on network devices.  E.g. packet filtering policies are
   executed by Access Control List on switches or firewalls.  Another
   example is Load balancer (LB) who extablishes TCP/HTTP connections
   with external clients and balances connections among server farm.
   During this process, TCP connection tables are dynamically generated
   on LB.  When VM migrates, the network devices that processing and
   forward VM's packets may change.  In order to keep VM's running
   serives and guanrantee security on new place, VM-relevant policies,
   including static policies as well as the dynamically generated
   information, need to migrate with VM.

   This draft describes some examples of the policies that need to
   migrate with VM, the problems that need to consider when migrate
   polices in Data Center.  The goal is to justify that it is necessary
   for IETF to make new effort on management of virtualized Data Center.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."



Gu & Fan                Expires December 10, 2011               [Page 1]


Internet-Draft  policy and dynamic information migration       June 2011


   This Internet-Draft will expire on December 10, 2011.

Copyright Notice

   Copyright (c) 2011 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.



































Gu & Fan                Expires December 10, 2011               [Page 2]


Internet-Draft  policy and dynamic information migration       June 2011


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
   2.  Terminologies and concepts . . . . . . . . . . . . . . . . . .  5
   3.  Policies Classification  . . . . . . . . . . . . . . . . . . .  6
     3.1.  Static Policies  . . . . . . . . . . . . . . . . . . . . .  7
       3.1.1.  Use Cases  . . . . . . . . . . . . . . . . . . . . . .  7
     3.2.  Dynamic Information  . . . . . . . . . . . . . . . . . . .  9
       3.2.1.  Use Cases  . . . . . . . . . . . . . . . . . . . . . . 10
   4.  Scenarios  . . . . . . . . . . . . . . . . . . . . . . . . . . 22
     4.1.  Policy migration within a single site  . . . . . . . . . . 22
     4.2.  Polices migration in asymmetric architecture . . . . . . . 23
     4.3.  Policies migration between DC sites  . . . . . . . . . . . 24
   5.  General Considerations . . . . . . . . . . . . . . . . . . . . 25
     5.1.  Time-sensitive . . . . . . . . . . . . . . . . . . . . . . 26
     5.2.  Notification . . . . . . . . . . . . . . . . . . . . . . . 27
     5.3.  Functionality  . . . . . . . . . . . . . . . . . . . . . . 27
     5.4.  Resource Capability  . . . . . . . . . . . . . . . . . . . 29
     5.5.  Migration Feedback . . . . . . . . . . . . . . . . . . . . 29
   6.  Security Considerations  . . . . . . . . . . . . . . . . . . . 29
   7.  Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 29
   8.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 29
     8.1.  Normative Reference  . . . . . . . . . . . . . . . . . . . 29
     8.2.  Informative Reference  . . . . . . . . . . . . . . . . . . 30
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 30


























Gu & Fan                Expires December 10, 2011               [Page 3]


Internet-Draft  policy and dynamic information migration       June 2011


1.  Introduction

   Data centers can host tens or even thousands of different
   applications.  Some are simple applications such as web servers
   providing static content, while some may be very complex, e.g.
   e-commerce, that requiring all around privacy protection and data
   security.  Clients of Data Center, unlike server hosting clients,
   raise more strict QoS and Security requirements.

   To satisfy different level of security requirements and to manage and
   improve the performance of these applications, data centers typically
   deploy a large variety of policies on network devices, e.g. interface
   security functions on switches and routers, and middleboxes,
   including firewalls, load balancers, SSL offloaders, web caches and
   intrusion prevention boxes.

   To satisfy QoS requirements, Data Center also implement QoS mechanism
   as ISP network.  IEEE 802.1 DCB working group defines a series of
   standards to guarantee quality of service.

      802.1Qau - Congestion Notification

      802.1Qaz - Enhanced Transmission Selection

      802.1Qbb - Priority-based Flow Control

   Without regard to mobile network, the existing DC network management
   has a pre-assumption that end hosts will not move.  If an end host
   moves, because the physical link has to break down and the service
   also has to break down, the network can treat it as two separated
   parts: one host leave the network and another host join the network.

   Server Virtualization and Virtual Machine Migration changes the
   situation and disable the preassumption.  With server virtualization,
   providers can reduce networking cost.  To support the same volume of
   services, fewer network devices, servers and links are required than
   before.  Multiple Virtual Machines (VMs) are established within a
   single physical server and the VMs are allowed to relocate to a
   different servers within the same subnet of Data Center, or even
   among different sites of a Data Center.  This is so called VM
   Migration.  VM Migration brings flexibility to Data Center, meanwhile
   it makes network management more complex and challenging.

   Unlike servers power off and power on again, in which case, servers
   know and can bear services disruption, VM Migration requires that
   VM's IP Address shouldn't change and running service mustn't be
   disrupted.  Meawhile, security level should be guaranteed.  In order
   to satisfy the above requirements, the VM relevant polices need to



Gu & Fan                Expires December 10, 2011               [Page 4]


Internet-Draft  policy and dynamic information migration       June 2011


   migrate with VM.

   IEEE 802.1Qbg has developed VSI Discovery and Configuration Protocol
   (VDP) to enable ajacent bridge to discover VM and configure
   corresponding static polices.

   The author has presented the problem to IEEE 802.1 DCB task group.
   DCB has recognized this problem and make extension to VDP to notify
   VM status, which will be introduced in Section 5.2.  IEEE 802.1 DCB
   would like to know furture progress of this problem in IETF and ask
   the author to let know the furthur progress.

   In the following sections, detail examples and senarios are given to
   explain why and which polices need to migrate with VM.


2.  Terminologies and concepts

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

   Source Network Device, Source switch, or Source device: the network
   device/switch/device from where the VM migrates.  I.E. VM is
   originally located under the source network device/switch/device.

   Destination Network Device, Destination switch, or Destination
   device: the network device/switch/device to where the VM migrates.
   I.E. VM is relocated to the destination network device/switch/device.

   TCP connection table: A table containing TCP connection-specific
   information.

   VM: Virtual Machine; A completely isolated operation system which is
   installed by software on a normal operation system.  An normal
   operation system can be virtualized into several VM.

   ACL: Access Control List; A list of rules that specifies which
   packets can be permited, which should be denied.

   QoS: Quality of Service;

   FW: Firewall; a policy based security function, typically used for
   restricting access to/from specific devices and applications.

   LB: Load Balancer; A methodology to distribute workload across
   multiple computers or a computer cluster, network links, central
   processing units, disk drives, or other resources, to achieve optimal



Gu & Fan                Expires December 10, 2011               [Page 5]


Internet-Draft  policy and dynamic information migration       June 2011


   resource utilization, maximize throughput, minimize response time,
   and avoid overload.

   NAT: Network Address Translation.  Refer to RFC3303.


3.  Policies Classification

   In this section, we use the following figure <Fig1> as an example
   network architecture.  Some redundant links are omitted for
   simplicity. the new VM1 under Server4 represents the VM1 after
   migration.  VM1 and new VM1 don't exist at the same time.  In the
   real world, the networking of DC could be different.
              -------------------------------------------------
              |                     Gateway                   |
              -------------------------------------------------
                    /    \                 /       \
                   /      \               /         \
    -----   --------     --------     --------     --------    ------
    |FW-A|--| AGG1  |----| AGG2  |    | AGG3  |----| AGG4  |---|FW-B|
    -----   --------     --------     --------     --------    ------
            |     |     |     |        |      |    |     |
            |     \     /     |        |      \    /     |
            |      \   /      |        |       \  /      |
            |       \/        |        |        \/       |
            |       /\        |        |        /\       |
            |      /  \       |        |       /  \      |
            |     /    \      |        |      /    \     |
            |    |      |     |        |     |     |     |
           ----------   ----------     ----------   ----------
           |Switch1 |   |Switch2 |     |Switch3 |   |Switch4 |
           ----------   ----------     ----------   ----------
           |       \   /      |        |       \  /      |
           |        \/        |        |        \/       |
           |        /\        |        |        /\       |
           |       /  \       |        |       /  \      |
         ----------   ----------      ----------   ----------
         |Server1 |   |Server2 |      |Server3 |   |Server4 |
         ----------   ----------      ----------   ----------
          |      |                                  |     |
         VM1    VM2                               VM3    new VM1

         Figure 1: Basic networking for discussion in this draft.

   One key feature of VM migration is that VM's IP Address shouldn't
   change, otherwise the running service could be disrupted.  In order
   to achieve this, additional mechanism need to be developed.  However,
   these mechanism are out of the scope of this draft.  In policy



Gu & Fan                Expires December 10, 2011               [Page 6]


Internet-Draft  policy and dynamic information migration       June 2011


   migration problem statement, we assume that there already exist an
   effective mechanism to guarantee VM's IP Address unchanged.

   Two kinds of polices are introduced in this draft, i.e. static
   polices and dynamic polices (or dynamic information).

3.1.  Static Policies

   Administrators of DC establish VM profile, which may include static
   policies for the VM, e.g.  VLAN Name, bandwidth indication, and Qos
   parameters, etc.  And they are suppposed to be consistent during the
   lifetime of the VM.  No matter where VM migrates, the static policies
   need to move with VM for the sake of security, privacy and QoS.

   Static polices can be described by natural languages or mathematics
   fomula.  They are implementated by specific configurations on
   different network elements, e.g. a ACL item on physical ports of
   switches enables a packet filtering policies.  In this draft, we
   discuss the migration of policies, but the policies migration also
   implies the migration of configurations on network devices, because
   configurations are embodiment of policies.

   Policies that need to migrate with VM are those can influence VM's
   running services or are related to security and billing&accounting.
   Different network devices will contain different kind of policies.
   For example, it can be static Access Control Lists on Access
   switches, QoS on switches and routers, security rules on Firewalls,
   and load balancing mechanism on Load Balancer, etc.

3.1.1.  Use Cases

   It's hard to enumerate all the VM-related static polices.  In this
   section, we just give two examples of static policies to show the
   necessity of static polices migration.

3.1.1.1.  Access Control List

   <Fig2> shows the influence of lack of ACLs on destination switch.
   There is an Default ACL on source switch (Switch1) deny all packets
   from IP subnet 10.138.3.0 to Internet.  And another ACL 101 allows IP
   Address 10.138.3.1, VM1's IP Address, to send packets to Internet.
   VM1 has a running service on it.  During service provisioning, VM1 is
   migrated to Server4 under Switch4, Where there is default ACL to deny
   all packets.  Since ACL 101 on Switch1 is not migrated to Switch4,
   VM1's IP Address falls into a default ACL which deny all unmatching
   packets.  As a result, packets belonging to the running services are
   dropped, hence the running service is disrupted.




Gu & Fan                Expires December 10, 2011               [Page 7]


Internet-Draft  policy and dynamic information migration       June 2011


            -------------------------------------------------
            |                     Gateway                   |
            -------------------------------------------------
                 //\   \                     /       \
                / *     \                   /         \
 ------     --------     --------     --------     --------    ------
 |FW-A|----| AGG1  |----| AGG2  |     | AGG3  |----| AGG4  |---|FW-B|
 ------     --------     --------     --------     --------    ------
            |*     |     |     |      |      |    |     |
            |*     \     /     |      |      \    /     |
            |*      \   /      |      |       \  /      |
            |*       \/        |      |        \/       |
ACL 101:    |*       /\        |      |        /\       |
permit VM1  |*      /  \       |      |       /  \      |Default ACL:
Default ACL:|*     /    \      |      |      /    \     |deny all
deny all    |*    |     |      |      |     |      |    |Packets dropped
           ----------   ----------  ----------  ----------
           |Switch1 |   |Switch2 |  |Switch3 |  |Switch4 |
           ----------   ----------  ----------  ----------
            |*      \   /      |      |       \  /      |/\  \/
            |*       \/        |      |        \/       |*   /\
            |*       /\        |      |        /\       |*
            |*      /  \       |      |       /  \      |*
           ----------   ----------   ----------   ----------
           |Server1 |   |Server2 |   |Server3 |   |Server4 |
           ----------   ----------   ----------   ----------
            |*    |                                 |   |*
            |*    |                                 |   |*
           VM1    VM2                              VM3  new VM1

               Figure 2: VM migration without ACL migration

3.1.1.2.  VLAN

   VMs with similar properties could be grouped into a VLAN, e.g. end
   hosts providing Web service are grouped into Web-VLAN.  Each VLAN has
   a unique VLAN ID.  Only when a VM is configured into a specific VLAN,
   can it receives broadcast packets from other VMs in the same VLAN.

   In <Fig3>, VM1 belongs to VLAN1.  Assuming VLAN1 is not configured on
   the port on AGG4 that attaching Switch4, and VM1 migrates to new VM1,
   Broadcast pakcets from another VM belonging to VLAN1 are forwarded to
   AGG4, however they can not be forwarded to Switch4, since broadcast
   packets are sending only to VLAN1-enabled port.  The *** line
   represents VLAN1 broadcast, which can not access Server4.  New VM1
   can not receive VLAN1 broadcast.





Gu & Fan                Expires December 10, 2011               [Page 8]


Internet-Draft  policy and dynamic information migration       June 2011


                   -----------------------------------
                   |              Gateway            |
                   -----------------------------------
                                    |
                                    |
               ------------------------------------------
               |             Core Switch                |
               ------------------------------------------
                    /*     \            /       *\
                   /*       \          /        \/\
    ------    -------    -------    --------   -------   ------
    |FW-A|----|AGG1 |----| AGG2|    | AGG3  |--| AGG4|---|Fw-B|
    ------    -------    -------    --------   -------   ------
              |*   *|    |  |        |    |    |    | \/ No VLAN1
              |*   *\    /  |        |    \    /    | /\
              |*    *\  /   |        |     \  /     |
              |*     *\/    |        |      \/      |
              |*     /*\    |        |      /\      |
              |*    / * \   |        |     /  \     |
              |*   /   * \  |        |    /    \    |
              |*  |    * |  |        |   |      |   |
            ---------  ---------    ---------   ----------
            |Switch1|  |Switch2|    |Switch3|   |Switch4 |
            ---------  ---------    ---------   ----------
   VLAN1     |*    \  /   * |VLAN1   |     \  /     |
             |*     \/    * |        |      \/      |
             |*     /\    * |        |      /\      |
             |*    /  \  \/ |        |     /  \     |
            ---------  ---------    ---------  ----------
            |Server1|  |Server2|    |Server3|  |Server4 |
            ---------  ---------    ---------  ----------
             |*   |                             |   |
             |\/  |                             |   |
            VM1   VM2                         VM3  new VM1

               Figure 3: VM migration without VLAN migration

3.2.  Dynamic Information

   Except for the static policies, some dynamic information could also
   be generated by network devices to assist packet processing.  TCP
   connection table on proxy is an obvious example.  Proxy intervenes
   the direct connection between external client and internal server,
   which can protect internal server from attacking.  Proxy establishes
   TCP connection with external client for internal server, a TCP
   connection Table being generated on proxy.  Then proxy establishes
   TCP connection with internal server for external client, another TCP
   connection table being generated.  These TCP Connection table can not



Gu & Fan                Expires December 10, 2011               [Page 9]


Internet-Draft  policy and dynamic information migration       June 2011


   be configured by network manager, but is generated by network
   devices, e.g.  Firewalls and proxies, according to real connections.
   Another example is cumulative data, e.g. how many packets/TCP
   connecition requests an end host has sent.  This information can only
   be generated by network devices according to real traffic.
   Configurations could be generated dynamically by network devices
   themselves according to the dynamic informaiton, e.g.  Dynamic ACLs.

   In following section, we enumerate several kinds of dynamic
   information.  A dynamic information could be generated on different
   network devices, meanwhile a specific network devices could hold more
   than one kind of dynamic information.  In following section, we
   introduce possible dynamic informaiton on specific network devices.

3.2.1.  Use Cases

3.2.1.1.  Switches

   We use switch to represent Top of Racks, Bridges and switches.  These
   devices have slightly different definition and functions, and may be
   used or combinedly used in differenct scenarios.

3.2.1.1.1.  Dynamic ACL

   In <Fig4>, assuming a default ACL is configured on Switch1 that all
   traffic is denied unless the end host is authorized and
   authenticated.  VM1 has been authenticated on source device and a
   dynamic ACL has been generated which allows VM1's traffic to pass
   through.

   A 'Deny all' default ACL is configured on Switch4 too.  VM1 migrates
   to destination device which attaches to Swtich4.  Because Switch4
   regards new VM1 as an unathenticated end host, according to default
   ACL, all data packets from VM1 will be dropped.  In order to continue
   service, VM1 has to authenticate again.  In this case, running
   service is disrupted.  If authentication and dynamic ACL, generated
   on Switch1, can be migrated to Switch4, data packets from VM1 are
   permitted, hence, without regard to influence caused by features,
   running service can continue.












Gu & Fan                Expires December 10, 2011              [Page 10]


Internet-Draft  policy and dynamic information migration       June 2011


              -------------------------------------------------
              |                     Gateway                   |
              -------------------------------------------------
                     //\   \              /       \
                    / *     \            /         \
    ------      -------   -------     -------     ------   ------
    |FW-A|-----| AGG1 |---| AGG2 |    | AGG3|----| AGG4|---|FW-B|
    ------      -------   -------     -------     ------   ------
                |*   |     |   |       |     |    |  |
                |*   \     /   |       |     \    /  |
                |*    \   /    |       |      \  /   |
                |*     \/      |       |       \/    |
                |*     /\      |       |       /\    |
                |*    /  \     |       |      /  \   |
                |*   /    \    |       |     /    \  |
                |*  |     |    |       |    |     |  |
   Default ACL: ---------  ---------  ---------  --------- Default ACL:
   deny all     |Switch1|  |Switch2|  |Switch3|  |Switch4| deny all
                ---------  ---------  ---------  ---------
                 |*   \   /    |      |     \  /   /\| \/  VM1's pakcets
   Dynamic ACL:  |*    \/      |      |      \/     *| /\  are dropped.
   Permit VM!    |*    /\      |      |      /\     *|
                 |*   /  \     |      |     /  \    *|
               ---------  ---------  ---------  ---------
               |Server1|  |Server2|  |Server3|  |Server4|
               ---------  ---------  ---------  ---------
                |*   |                          |   *|
                |*   |                          |   *|
               VM1   VM2                       VM3    new VM1

           Figure 4: VM migration without dynamic ACL migration

3.2.1.1.2.  DHCP snooping

   Assuming Switch1 is DHCP Snooping Enabled and a DHCP Snooping mapping
   item is created for VM1: (IP-VM1: MAC-VM1).  This mapping is created
   dynamically by listening to DHCP Response message.  If VM1 migrate to
   destination device, since the IP Address of VM1 doesn't change, there
   is no DHCP Request sent by VM1 before lease time expires.  So on
   Switch4, no mapping information can be generated by listening to DHCP
   response, and all traffic from VM1 will be dropped.

   If DHCP snooping table on Switch1 can be migrated to Switch4, Switch4
   learns from the migrated DHCP snooping talbe that packet with IP-VM1
   and MAC-VM1 can be regularly forwarded.






Gu & Fan                Expires December 10, 2011              [Page 11]


Internet-Draft  policy and dynamic information migration       June 2011


           -------------------------------------------------
           |                     Gateway                   |
           -------------------------------------------------
                  //\   \               /     \
                 / *     \             /       \
    ------   -------     ------   --------   -------   ------
    |FW-A|---|AGG1 |----| AGG2|   | AGG3 |---|AGG4 |---|FW-B|
    ------   --------    ------   --------   -------   ------
              |*  |     |  |       |   |    |   |
   DHCP       |*  \     /  |       |   \    /   |
   Response   |*   \   /   |       |    \  /    |
       \      |*    \/     |       |     \/     |
        \     |*    /\     |       |     /\     |
         \    |*   /  \    |       |    /  \    |DHCP snooping enabled
          \   |*  /    \   |       |   /    \   |No DHCP request
   DHCP      >|* |     |   |       |   |    |   |from new VM1
   snooping ---------  ---------  ---------  ----------
   enabled  |Switch1|  |Switch2|  |Switch3|  |Switch4 |
            ---------  ---------  ---------  ----------
   ---------- |*   \   /  |       |     \  /   /\| \/ VM1's packets
   |Snooping| |*    \/    |       |      \/     *| /\ are dropped
   |Table   | |*    /\    |       |      /\     *|
   ---------- |*   /  \   |       |     /  \    *|
            --------  --------- ---------  ----------
           |Server1| |Server2|  |Server3|  |Server4 |
            --------  --------- ---------  ----------
              |*  |                         |   *|
              |*  |                         |   *|
             VM1  VM2                     VM3  new VM1

       Figure 5: VM migration without DHCP snooping table migration

3.2.1.1.3.  IGMP snooping

   IGMP snooping is similar to DHCP Snooping.  Multicast membership is
   created on ports by listening to IGMP JOIN or membership report
   messages.  When a switch receives a multicast packet, it forwards the
   packet to the port which has joined the multicast group.  In <Fig6>,
   assuming VM1 sends an IGMP JOIN Group1 message to Multicast server,
   port1 and port-uplink on Switch1 check the JOIN message and tag on
   port1 and port-uplink that they should forward Multicast packets from
   Group1.  If VM1 migrates to destination, since migration is
   transparent to VM, VM1 will not send IGMP membership report until
   next IGMP General Query is received.  If port2 on Swtich4 didn't join
   Group1, multicast packet from Group1 won't be forward to port2, then
   VM1 won't be able to receive Multicast packets.





Gu & Fan                Expires December 10, 2011              [Page 12]


Internet-Draft  policy and dynamic information migration       June 2011


            -------------------------------------------------
            |                     Gateway                   |
            -------------------------------------------------
                   /*    \                   /       *\
                  /*      \                 /         *\
  ------      --------     --------      --------    --------     ------
  |FW-A|-----| AGG1  |----| AGG2  |      | AGG3  |---| AGG4  |--- |FW-B|
  ------      --------     --------      --------    --------     ------
              |*     |     |     |       |      |    |    *|
   IGMP       |*     \     /     |       |      \    /    *|
 JOIN Snooping|*      \   /      |       |       \  /     *|
       \      |*       \/        |       |        \/      *|
        \     |*       /\        |       |        /\      *|
         \    |*      /  \       |       |       /  \     *|IGMP enabled
          \   |*     /    \      |       |      /    \    *|No IGMP JOIN
             >|*    |     |      |       |     |     |   \/|from new VM1
 IGMP         ----------   ----------   ----------   ----------   \/
 enabled      |Switch1 |   |Switch2 |   |Switch3 |   |Switch4 |   /\
              ----------   ----------   ----------   ----------
              |*      \   /      |       |       \  /      |
 ----------   |*       \/        |       |        \/       |
 |Group1: |   |*       /\        |       |        /\       |
 |Port1   |   |*      /  \       |       |       /  \      |
 ----------   ----------   ---------    ---------   ----------
              |Server1 |   |Server2|    |Server3|   |Server4 |
              ----------   ---------    ---------   ----------
              |*    |                                 |    |
              |\/   |                                 |    |
             VM1    VM2                             VM3    new VM1

       Figure 6: VM migration without IGMP snooping table migration

3.2.1.1.4.  Cumulative Data

   To ensure VMs consume no more than assigned bandwidth and to enable
   QoS control, billing and accounting, bridges need to cumulate packets
   number and calculate packet rate.  Lack of these cumulative data and
   statistic data on destination bridge decrease the accuracy.

3.2.1.2.  Firewall

   Fireall (FW) is a filtering device that separates LAN segments,
   giving each segment a different security level and establishing a
   security perimeter that controls the traffic flow between segments.
   There are different types of firewalls based on their packet-
   processing capabilities and their awareness of application-level
   information:




Gu & Fan                Expires December 10, 2011              [Page 13]


Internet-Draft  policy and dynamic information migration       June 2011


      Packet-filetering firewalls

      Proxy firewalls

      Stateful firewalls

      Hybrid firewalls

3.2.1.2.1.  Dynamic ACL

   Packet filtering Firewall filters packet according to ACL.  Dynamic
   ACL could be generated to protect internal network/servers from
   attacks, similar to dynamic ACL on Switches.

3.2.1.2.2.  TCP Connection Table

   Proxy Firewall proxies TCP connections between internal servers and
   external clients.  It establishes TCP connection with external client
   and then it establishes TCP connection with internal server.  TCP
   connection tables are cached on Proxy Firewall to indicate how to
   transform following packets belonging to this TCP connection.
       External Client        Firewall          Internal server
           |----- TCP SYN  ----->|
           |<-----SYN/ACK  ------|
           |-------ACK     ----->|
                                 |--------TCP SYN ------>|
                                 |<-------SYN/ACK -------|
                                 |--------ACK     ------>|
           |-------DATA   ------>|--------DATA    ------>|
           |<------DATA   -------|<-------DATA    -------|

                         Figure 7: Proxy Firewall



















Gu & Fan                Expires December 10, 2011              [Page 14]


Internet-Draft  policy and dynamic information migration       June 2011


                 -------------------------------------------------
                 |                     Gateway                   |
                 -------------------------------------------------
  ----------------     /*  \                  /    *\
  |TCP Conn Table|    /*    \                /      *\ No corresponding
  ----------------   /*      \              /        *\ TCP Conn Table
  ------ **** --------     --------     --------    -------- ****>------
  |FW-A|-----| AGG1  |----| AGG2  |     | AGG3 |----| AGG4 |----- |FW-B|
  ------ **** --------     --------     --------    --------  \/  ------
              |*     |     |     |       |      |    |     |  /\
              |*     \     /     |       |      \    /     |
              |*      \   /      |       |       \  /      |
              |*       \/        |       |        \/       |
              |*       /\        |       |        /\       |
              |*      /  \       |       |       /  \      |
              |*     /    \      |       |      /    \     |
              |*    |     |      |       |     |     |     |
             ----------   ----------    ----------   ----------
             |Switch1 |   |Switch2 |    |Switch3 |   |Switch4 |
             ----------   ----------    ----------   ----------
             |*      \   /      |       |       \  /       |
             |*       \/        |       |        \/        |
             |*       /\        |       |        /\        |
             |*      /  \       |       |       /  \       |
           ----------   ----------    ----------   ----------
           |Server1 |   |Server2 |    |Server3 |   |Server4 |
           ----------   ----------    ----------   ----------
             |*    |                                |      |
             |\/   |                                |      |
            VM1    VM2                             VM3    new VM1

       Figure 8: VM migration without TCP connection table migration

   A typical TCP Connection Table includes the following data:

      tcpConnState: The state of this TCP connection.

      tcpConnLocalAddress: The local IP address for this TCP connection.

      tcpConnLocalPort: The local port number for this TCP connection.

      tcpConnRemAddress: The remote IP address for this TCP connection.

      tcpConnRemPort: The remote port number for this TCP connection.

   A TCP Connectin Table could also includes the following information:





Gu & Fan                Expires December 10, 2011              [Page 15]


Internet-Draft  policy and dynamic information migration       June 2011


      Sequence Number: the sequence number in the packet header the
      sender is going to send.

      Acknowledgement Number: the sequence number in the packet header
      the receiver is hoping to receive.

      Idle time: the time that the tcp connection table hasn't been
      updated.

   Assuming TCP Connection Table item is generated for VM1 on Fw-A, the
   information is as follows:

      tcpConnState == Established

      tcpConnLocalAddress == 10.138.3.1

      tcpConnLocalPort == 1234

      tcpConnRemAddress: == 192.167.22.3

      tcpConnRemPort == 4321

   VM1 migrates to Server4 under Switch4, without TCP Connection Table
   items migration.  The packets belonging to this TCP Connection will
   continue coming, which will pass FW-B, instead of FW-A.  Because
   there is no TCP Connection Table for VM1 on FW-B, the following
   packets belonging to the TCP Connection will be dropped, hence the
   running service is broken down.

3.2.1.2.3.  Cumulative Data

   One example for cumulative data is unfinished TCP Connection
   established by a specific VM.  In order to avoid TCP SYN flood,
   Firewall may control the unfinished TCP Connection established by a
   single end host by setting a threshold.  For example, the NM set the
   threshold to 50, and VM1 has established 30 unfinished TCP
   connections.  If the cumulated TCP connection number isn't migrated
   to destination devices, the destination device will allow VM1 to
   establish up to 50 unfinished TCP connections.  For the single
   destination device, the unfinished TCP connections established by VM1
   is under control, but for the whole DC, VM1 has established 80
   unfinished TCP connections.  So VM1 has consume more resoureces than
   allowed.

3.2.1.2.4.  NAT Address Mapping

   Data center may implement network address translation.  Sometimes NAT
   function is enabled on Firewall.  Private IP Add. is assigned to



Gu & Fan                Expires December 10, 2011              [Page 16]


Internet-Draft  policy and dynamic information migration       June 2011


   internal VM.  When VM communicate with external client, NAT function
   assigns a public IP Add. to the communicating internal VM.  A mapping
   between private IP Add. and public IP Add. is recorded on NAT
   function.

   Assuming a mapping from Pri-IP-VM1 to Pub-IP-VM1 is generated on
   FW-A.  When VM1 migrates to Server4 without the address mapping,
   following packets from external client can not be translated to
   correct Pri-IP-VM1, and vice versa.
                  -------------------------------------------------
                  |                     Gateway                   |
                  -------------------------------------------------
                        //\   \              / \*  Dst: Pub-IP-VM1
     Src: Pub-IP-VM1   /*      \            /   \*  No Address mapping
  -----------------   /*        \          /     \* from Pub-IP-VM1
  |Address Mapping|  /*          \        /       \* to Private-IP-VM1
  ----------------- /*            \      /         \*Packets are dropped
   ------- ****>--------    --------   -------     ------- ****>------
   |NAT-A|------| AGG1  |---| AGG2  |  |AGG3 |----| AGG4 |-----|NAT-B|
   ------- <****--------    --------   -------     -------\/   -------
                |*     |     |    |     |    |    |     | /\
                |*     \     /    |     |    \    /     |
                |*      \   /     |     |     \  /      |
                |*       \/       |     |      \/       |
                |*       /\       |     |      /\       |
                |*      /  \      |     |     /  \      |
                |*     /    \     |     |    /    \     |
                |*    |     |     |     |   |     |     |
               ---------   ---------   ---------  ----------
               |Switch1|   |Switch2|   |Switch3|  |Switch4 |
               ---------   ---------   ---------  ----------
               |*      \   /      |     |      \  /     |
  Src:         |*       \/        |     |       \/      |
  Pri-IP-VM1   |*       /\        |     |       /\      |
               |*      /  \       |     |      /  \     |
              ---------   ---------    ---------  ----------
              |Server1|   |Server2|    |Server3|  |Server4 |
              ---------   ---------    ---------  ----------
               |*     |                            |    |
               |\/    |                            |    |
              VM1    VM2                          VM3  new VM1

       Figure 9: VM migration without NAT Address Mapping migration








Gu & Fan                Expires December 10, 2011              [Page 17]


Internet-Draft  policy and dynamic information migration       June 2011


3.2.1.3.  Load Balancer

   When a cluster of servers are organized to provide same services,
   Load Balancer (LB) is used to balance service load between servers.
   All external requests are sent to LB, before they are redirected to a
   specific server.  First, external clients extablish TCP connection
   with LB.  Then LB decides which specific server should take care of
   the requests.  Two ways to redirect the requests, i.e. transparent LB
   and proxy LB.  The following dynamic information applies for both
   ways.

3.2.1.3.1.  Connection Table

   Layer 4 LB identifies and makes load-balancing decesion by Layer 4
   Protocol, e.g.  TCP.  Layer 4 LB generates TCP connection table which
   is similar to TCP connection table on Firewall.  <Fig10>
       External Client        Load Balancer         Internal server
           | ------- TCP SYN  ------->|-------TCP SYN ----->|
           |<--------SYN/ACK  --------|<------SYN/ACK ------|
           |---------ACK      ------->|-------ACK     ----->|
           |---------HTTP REQ ------->|-------HTTP REQ----->|
           |<--------HTTP ACK --------|<------HTTP ACK------|
           |---------DATA     ------->|-------DATA    ----->|
           |<--------DATA     --------|<------DATA    ------|

                     Figure 10: Layer 4 Load Balancing

   Layer 5 LB identifies and makes load-balancing decesion by Layer 5
   protocol, e.g.  HTTP, SMTP and FTP, that is found in packet payload.
   The Layer 5 information could be HTTP URL, HTTP cookie, or HTTP
   header field user agent.  Layer 5 LB generates connection table for
   HTTP, SMTP, and FTP, etc.<Fig11>
       External Client        Load Balancer          Internal server
           | ------- TCP SYN  ----->|
           |<--------SYN/ACK  ------|
           |---------ACK      ----->|
           |---------HTTP Req ----->|
                                    |-------TCP SYN ----->|
                                    |<------SYN/ACK ------|
                                    |-------ACK     ----->|
                                    |-------HTTP Req----->|
           |---------HTTP ACK ----->|-------HTTP ACK----->|
           |---------DATA     ----->|-------DATA    ----->|
           |<--------DATA     ------|<------DATA    ------|

                     Figure 11: Layer 5 Load Balancing

   <Fig12> shows the result of lack of Connection table on destination



Gu & Fan                Expires December 10, 2011              [Page 18]


Internet-Draft  policy and dynamic information migration       June 2011


   LB.
         -------------------------------------------------
         |                     Gateway                   |
         -------------------------------------------------
-------------     /*  \                 /    *\ No VM1's Conn Table
|Conn Table |    /*    \               /      *\Packets could be dropped
-------------   /*      \             /        *\or mis-redirected
------ <**** -------    --------     -------   ------- ****>-------
|LB-A|------| AGG1 |---| AGG2  |     | AGG3|---|AGG4 |----- |LB-B|
------ ****> -------    --------     -------   ------- <**** ------
            |*     |     |     |     |     |    |   |*
            |*     \     /     |     |     \    /   |*
            |*      \   /      |     |      \  /    |*
            |*       \/        |     |       \/     |*
            |*       /\        |     |       /\     |*
            |*      /  \       |     |      /  \    |*
            |*     /    \      |     |     /    \   |*
            |*    |     |      |     |    |     |   |*
          ----------   ----------   ---------   ---------
          |Switch1 |   |Switch2 |   |Switch3|   |Switch4|
          ----------   ----------   ---------   ---------
           |*      \   /      |      |       \  /    |*
           |*       \/        |      |        \/     |*
           |*       /\        |      |        /\     |*
           |*      /  \       |      |       /  \    |*
          ----------   ----------   ----------   ----------
          |Server1 |   |Server2 |   |Server3 |   |Server4 |
          ----------   ----------   ----------   ----------
           |*   |                                |*  |
           |\/  |                                |\/ |
          VM1  VM2                              VM3  new VM1


        Figure 12: VM migration without Connection table migration

3.2.1.3.2.  Sticky Table

   Session persistence refers to the capability of the Load alancer to
   logically group multiple connections from the same client transaction
   to the service.  Session persistence is also known as stickiness or
   sticky connections because the goal is to stick two or more
   connections together as part of a single session.  This grouping is
   done to ensure the connections are handled and forwarded to the
   groups of servers that are aware of and expect to see the remaining
   connections.  This ensures that the client has a successful
   interaction with the application.

   LB usually keeps a sticky table for session persistence, which is



Gu & Fan                Expires December 10, 2011              [Page 19]


Internet-Draft  policy and dynamic information migration       June 2011


   used to match incoming requests to existing connections so that they
   can be grouped together.  A sticky table could include the following
   information.

      Sticky groups

         Group name (sticky group identifier)

         Type (sticky group type, according to stick methods)

         Sticky server farm

         Backup server farm

         Aggregate state (to indicate whether the state of the backup
         server is tied to the virtual server state)

         Sticky enabled on backup server farm (to indicate whether the
         backup server farm is sticky)

         Replicate (to indicate whether to replicate sticky entries on
         the standby device)

         Timeout (a mechanism to age out sticky table entries)

         Timeout active connections (to specify whether to time out
         sticky table entries if active connections exists after the
         sticky timer expires)

      Sticky methods

      Sticky connections

      Real servers

   In <Fig13>, two connections, represented by * line and $ line, are
   redirected to VM1, and a sticky table is generated on LB-A.  VM1
   migrates to new VM1, without Sticky table migrates with VM1.  A new
   connection, represented by & line, arriving at LB-B, which can not
   know that this connection should be redirected to new VM1, because of
   lack of sticky table.  As a result, LB-B redirect the new connection
   to other VM than new VM1.  This mis-redirection could cause key
   information lost.  For example, in e-business, refer to








Gu & Fan                Expires December 10, 2011              [Page 20]


Internet-Draft  policy and dynamic information migration       June 2011


                            External Client-X
                                 * $ &
                                 * $ &
                                 * $ &
              --------------------------------------------
              |                 Gateway                  |
              --------------------------------------------
                    /*$ \             /\*$&
                   /*$   \           /  \*$& No Sticky table for client-X
-------------     /*$     \         /    \*$& Connections from client-X
|Sticky Table|   /*$       \       /      \*$& may be redirected to
-------------   /*$         \     /        \*$& different servers
 ----- <*$*$*-------    ------   -------    -------- *$&*> ------
|LB-A|------| AGG1 |----| AGG2|  | AGG3|----| AGG4  |----- |LB-B|
 ----- *$*$> -------    ------   -------    --------<*$&*  ------
             |*$   |     |    |     |  |    |     |*$&
             |*$   \     /    |     |  \    /     |*$&
             |*$    \   /     |     |   \  /      |*$&
             |*$     \/       |     |    \/       |*$&
             |*$     /\       |     |    /\       |*$&
             |*$    /  \      |     |   /  \      |*$&
             |*$   /    \     |     |  /    \     |*$&
             |*$  |      |    |     | |      |    |*$&
           ---------   --------   ---------  ----------
           |Switch1|   |Switch2|  |Switch3|  |Switch4 |
           ---------   --------   ---------  ----------
            |*$    \  /       |     |    \  /     |*$&
            |*$     \/        |     |     \/      |*$&
            |*$     /\        |     |     / \     |*$&
            |*$    /  \       |     |    /   \    |*$&
         ----------   ----------  ---------  ----------
         |Server1 |   |Server2 |  |Server3|  |Server4 |
         ----------   ----------  ---------  ----------
           |$%   |                            |&  |*$
           |\/   |                            |\/ |\/
          VM1    VM2                         VM3  new VM1
******
$$$$$$   Three connectons from the same external clients.
&&&&&&

          Figure 13: VM migration without Sticky table migration

   While server or server farm move to a new location under a new LB, in
   order to keep the existing connections and sessions, as well as
   session persistence, both connection table and sticky table need to
   move with servers.





Gu & Fan                Expires December 10, 2011              [Page 21]


Internet-Draft  policy and dynamic information migration       June 2011


4.  Scenarios

   In previous section, we introduce two classes of policies, i.e.
   static policies and dynamic information.  The introduction is based
   on a simplized networking architecture and intentionally omit other
   factors.  The polices in the example network implies <1:1> partner,
   that is polices need to move from one device to another.  However, in
   real world, network architectures are much complicate than the
   simplized example networking.  We introduce several scenarios in this
   section.  Different architecture raise different requirements and
   migration partner style.

4.1.  Policy migration within a single site

   A DC site means a geographic location, which could include one
   subnet, or several subnets.  A VM migrates to a new location which is
   in the same subnet as original location.  In this case, there might
   not be policies on Middleboxes that need to be migrated, since new
   VM1 is under the same Middleboxes as VM1.  But dynamic information on
   switches need to migrate as well.































Gu & Fan                Expires December 10, 2011              [Page 22]


Internet-Draft  policy and dynamic information migration       June 2011


             -------------------------------------------------
             |                     Gateway                   |
             -------------------------------------------------
               /     \                     /       \
              /       \                   /         \
 ------   --------     --------       --------     --------       ------
 |FW-A|---| AGG1  |----| AGG2  |      | AGG3  |----| AGG4  |----- |FW-B|
 ------   --------     --------       --------     --------       ------
          |      |     |     |        |      |    |     |
          |      \     /     |        |      \    /     |
          |       \   /      |        |       \  /      |
          |        \/        |        |        \/       |
          |        /\        |        |        /\       |
          |       /  \       |        |       /  \      |
          |      /    \      |        |      /    \     |
          |     |     |      |        |     |     |     |
         ----------   ----------     ----------   ----------
         |Switch1 |   |Switch2 |     |Switch3 |   |Switch4 |
         ----------   ----------     ----------   ----------
          |       \   /      |        |       \  /      |
          |        \/        |        |        \/       |
          |        /\        |        |        /\       |
          |       /  \       |        |       /  \      |
        ----------   ----------     ----------   ----------
        |Server1 |   |Server2 |     |Server3 |   |Server4 |
        ----------   ----------     ----------   ----------
            |            |
          VM1           new VM1

        Figure 14: Policy migration within a single DataCenter site

4.2.  Polices migration in asymmetric architecture

   The network architecture could be asymmetric.  Refer to Fig16.  In
   this case, there is possible to encounter <1:n> or <n:1> partnership.
   The former partnership implies policies on one orginal device might
   need to migrate to n destination devices.  The latter partnership
   implies policies on multiple original devices might need to migrate
   to one destination devices.  For example, when VM1 migrate from
   Server 4 to Server1, the dynamic information on both Switch4 and ToR
   need to migrate to Switch1, which is 2:1 partnership&#12290;










Gu & Fan                Expires December 10, 2011              [Page 23]


Internet-Draft  policy and dynamic information migration       June 2011


           -------------------------------------------------
           |                     Gateway                   |
           -------------------------------------------------
                 /     \                  /       \
                /       \                /         \
   ------   -------     -------      --------     --------    ------
   |FW-A|---| AGG1|----| AGG2 |      | AGG3  |----| AGG4  |---|FW-B|
   ------   -------     -------      --------     --------    ------
             |    |     |     |       |      \   /      |
             |    \     /     |       |        \/       |
             |     \   /      |       |        /\       |
             |      \/        |      ----------   ----------
             |      /\        |      |Switch3 |   |Switch4 |
             |     /  \       |      ----------   ----------
             |    /    \      |       |       \  /      |
             |    |     |     |       |        \/       |
            ---------   ----------    |        /\       |
            |Switch1|   |Switch2 |   ----------   ----------
            ---------   ----------   |   ToR  |   |  ToR   |
             |       \  /     |      ----------   ----------
             |        \/      |       |        \/       |
             |        /\      |       |        /\       |
             |       /  \     |       |       /  \      |
            ----------   ---------   ----------   ----------
            |Server1 |   |Server2|   |Server3 |   |Server4 |
            ----------   ---------   ----------   ----------
                |                                       |
              new VM1                                  VM1

          Figure 15: Polices migration in asymmetric architecture

4.3.  Policies migration between DC sites

   Currently, most VM hot migration is limited in the same data center
   site.  But in the furture there might be requirements for VM
   migration between data center sites, as shown in <Fig17>.  A use case
   for this scenario is resource coordination between data center sites.
   A DC provier has two data center sites in different countries, and
   both of them serve external clients with similar applications.  Some
   of the applications are high priority applications, others are normal
   priority ones.  At first, requests from external clients are
   redirected to the data center site which are physically close to the
   external clients.  However, at some time, one data center is
   approaching its capability limits, which could be servers capability
   limits (i.e. few server capability are available for new requests) or
   network capability limits (i.e. network can not support so much
   packet processing requirement, and packet loss happens) and
   performance is degraded.  In order to guarantee high priority



Gu & Fan                Expires December 10, 2011              [Page 24]


Internet-Draft  policy and dynamic information migration       June 2011


   applications, and try best to make fewest damage to normal priority
   applications, some VMs, who provide normal priority applications, can
   be migrated to the other data center sites.

   In this case, policies on switches, middleboxes and gateways need to
   migrate to new devices on the other data center site.

   Policies migration between data centers also need to consider the
   asymmetric structure.
          ------------------          ---------------------
          |    Gateway     |          |     Gateway2      |
          ------------------          ---------------------
                /     \                     /       \
               /       \                   /         \
  ------   --------     --------       --------     --------      ------
  |FW-A|---| AGG1  |----| AGG2  |      | AGG3  |----| AGG4  |-----|FW-B|
  ------   --------     --------       --------     --------      ------
           |      |     |     |        |      |    |     |
           |      \     /     |        |      \    /     |
           |       \   /      |        |       \  /      |
           |        \/        |        |        \/       |
           |        /\        |        |        /\       |
           |       /  \       |        |       /  \      |
           |      /    \      |        |      /    \     |
           |     |     |      |        |     |     |     |
          ----------   ----------     ----------   ----------
          |Switch1 |   |Switch2 |     |Switch3 |   |Switch4 |
          ----------   ----------     ----------   ----------
           |       \   /      |        |       \  /      |
           |        \/        |        |        \/       |
           |        /\        |        |        /\       |
           |       /  \       |        |       /  \      |
         ----------   ----------     ----------   ----------
         |Server1 |   |Server2 |     |Server3 |   |Server4 |
         ----------   ----------     ----------   ----------
             |                                         |
           VM1                                       new VM1

               Figure 16: Polices migration between DC sites


5.  General Considerations

   In this section, we enumerate some general considerations on Policy
   migration.  Only with deep consideration on these factors can we
   achieve a successful VM migration and policy migration.





Gu & Fan                Expires December 10, 2011              [Page 25]


Internet-Draft  policy and dynamic information migration       June 2011


5.1.  Time-sensitive

   VM migration needs a period to finish memory and register copy.
   Fig.3 shows the VMware VMotion process.  There are three points we
   should pay attention to.

      Pre-copy period: VM begins to prepare for migration.  In this
      period, VM pre-copy memory state to the new VM on destination
      device.  The original VM is still power on and service is still
      running, which means the memory and register could keep changing.
      The new VM is power-on.

      VM not running period: The end phase of memory copy.  In this
      period, original VM stop running service, the memory will not
      change.  Original VM finish copying the rest changed memory and
      register to new VM.  New VM is still power-on.

      VM power-off point: After original VM receives the OK message from
      new VM, it turns off the power, and meanwhile the new memory
      starts to run.

   We can see that it's unrealistic to make a 'zero delay' VM migration,
   because there is at least about 1 second period (VM not running
   period) when neither VM is running.

   Assuming there is a 3rd-party device can GET and SET dynamic
   information.(This is only an possible way to migrate polices.
   Polices could also be migrated distributedly.  The author has no
   preferrence to any approaches in this draft.)  The 3rd-party device
   GET dynamic information at Time A, and finish SET at Time B. At Time
   A, the Sequence Number of VM1's TCP Connection is 99.  After 3rd-
   party device GET dynamic information, VM1's TCP connection keeps
   transferring packets and Sequence Number increase to 110, until VM
   not running period begins.  During VM not running period, no TCP
   packet is acknowledged by VM1, so the Sequence Number is 110.  At
   Time B, the destination Firewall is SET by Sequence Number 99.  When
   new VM1 starts, the packets belonging to VM1's TCP connection comes
   to Firewall-B with Sequence Number 111.  Since the receiving Sequence
   Number doesn't equal to the Acknowleadge Sequence Number of
   Firewall-B, this packet will be dropped and the running service is
   broken down.

   Another example: After Time A, VM1 may establish new TCP connections,
   which are not migrated to new Firewall, then following packets
   belonging to new TCP connections will be dropped and running services
   are disrupted.





Gu & Fan                Expires December 10, 2011              [Page 26]


Internet-Draft  policy and dynamic information migration       June 2011


 -------------------------------------------------------
    ^     ||    |   Power-on destination VM          ^
    |     ||    |                                    |
    |     ||--->| ---------------------------     VMotion
   VM     ||    |    Pre-copy memory state        abortable
  running ||--->|                                   |-------->Time A
 on source||    |                                   | /\
    |     ||--->|                                   |Dynamic Information
    |     ||--->|                                   |keep changing
    V     ||    |                                   | \/
 ---------------------------------------------------|-----------
    ^     |---->| Checkpoint-save src VM and        |
    |     |---->| transfer state                    |
    |     |     |-----------------------------------|
   VM not |---->| transfer remaining changed memory |-------->Time B
  running |     | and checkpoint-restore dst VM     V
    |     |     |--------------------------------------
  ~1sec   |<----| send restore OK
    V     |     |
 ------------------------------------------------------
          |    ||  Power-off source VM
          |    ||  VM running on destination

                     Figure 17: VMware VMotion process

5.2.  Notification

   As Section 5.1shows, policy migration is very sensitive to migration
   time.  Either later or earlier may cause service disruption.  The
   best migration time is after the original VM stops running and before
   the destination VM begins running.  Only Hypervisor knows the acurate
   VM status.  So we need a notification from server side, Hypervisor in
   specific, to tell network side when to migrate polices.

   In fact, the author has presented the problem to IEEE 802.1 DCB task
   group, which defines VSI Discovery and Configuration protocols (VDP).
   DCB has recognized this problem and make effort within its scope.
   DCB has extended VDP, in 802.1 Qbg, to enable notification from
   Hypervisor to adjacent bridge.  DCB expects corresponding IETF WG to
   make furthur effort to distribute the notification to devices from
   Layer 3 to Layer 7.  And IEEE 802.1 DCB working group would like to
   see further progress at IETF on this problem.

5.3.  Functionality

   Data center architecture could be heterogeneous, both in physical
   structure and functionality.  Section 4.2 and <Fig16> shows the
   asymmetric physical structure.  The following figure shows the case



Gu & Fan                Expires December 10, 2011              [Page 27]


Internet-Draft  policy and dynamic information migration       June 2011


   where functionality on devices is heterogenous.  While VM migrates
   from one location to a heterogeneous location, it's necessary to
   negotiate and make sure that essential functions are supported at
   destination location.  Otherwise, VM migration maybe fail.  Even VM
   successfully migrates, there might be security risk and/or service
   failure.  VM Managers who want to migrate VM between heterogeneous
   architecutre must be aware of the risk.

   <Fig19> shows two examples of functionality heterogeneous.  FW-A is
   the source Firewall with packet filtering function, and a dynamic ACL
   is generated on FW-A for VM1.  FW-B is the destination Firewall
   without packet filtering funtion.  In this case, dynamic ACL on FW-A
   can not be implemented on FW-B.  The similar case happens on Switch1
   and Switch4.  Switch1 has DHCP snooping function enabled, while
   Switch has not.  DHCP snooping table from Switch1 can not be
   implemented on Switch4.
             -------------------------------------------------
             |                     Gateway                   |
             -------------------------------------------------
                /     \                     /       \
               /       \                   /         \
 ------    --------     --------       ------    -------   ------
 |FW-A|----| AGG1 |----| AGG2  |      | AGG3|----| AGG4|---|FW-B|
 ------    --------     --------       ------    -------   ------
 with      |      |     |     |        |    |    |    | without
 Packet    |      \     /     |        |    \    /    | Packet filtering
 filtering |       \   /      |        |     \  /     |
           |        \/        |        |      \/      |
           |        /\        |        |      /\      |
           |       /  \       |        |     /  \     |
           |      /    \      |        |    /    \    |
           |     |     |      |        |   |      |   |
          ----------   ----------    ---------   ---------
 with     |Switch1 |   |Switch2 |    |Switch3|   |Switch4|  withou
 DHCP     ----------   ----------    ---------   ---------  DHCP
 Snooping  |       \   /      |        |      \  /     |  Snooping
           |        \/        |        |       \/      |
           |        /\        |        |       /\      |
           |       /  \       |        |      /  \     |
          ----------   ----------    ----------  ----------
          |Server1 |   |Server2 |    |Server3 |  |Server4 |
          ----------   ----------    ----------  ----------
              |                                      |
             VM1                                   new VM1

               Figure 18: Heterogeneous functions on devices





Gu & Fan                Expires December 10, 2011              [Page 28]


Internet-Draft  policy and dynamic information migration       June 2011


5.4.  Resource Capability

   Even if destination devices can support specific function that is
   required by dynamic information migration, there might be not enough
   resources to implement migrated polices.  For example, there are 5
   connection state items related to VM1 on source LB.  However, the
   destination LB has only 4 table items available.  That means the
   destination LB doesn't have enough resource to support this VM
   migration, and policies migration on network devices may fail.

5.5.  Migration Feedback

   Sometimes, VM manager has ensure, by some way, that proper
   functionality and plenty of resources are available on destionation
   devices, by which we mean all the devices that need to handle VM's
   packets, but dynamic information migration still fails.  In this
   case, VM manager or Hypervisor needs to know the results of policy
   migration.  The feedback from network can help VM Manager to decide
   whether to rollback the migration or continue migration with risk.


6.  Security Considerations

   The policies and dynamic information described above are all about
   security.  Besides, we need to be careful to avoid poisoned polices
   from untrusted source.  That means no mather how the polices are
   migrated, be it distributed or centralized way, authentication and
   verification are required.

   A reliable notifation from server side to network side is also
   necessary, so that the destination and original devices can be sure
   that a real VM migration happens.


7.  Acknowledgments

   I would like to thank the following people for contributing to this
   draft: Ning Zong, David harrington, Linda dunbar, Susan Hares, Serge
   manning, Barry Leiba, Jiang xingfeng, Song Wei, Robert Sultan, and
   many others.


8.  References

8.1.  Normative Reference

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", March 1997.



Gu & Fan                Expires December 10, 2011              [Page 29]


Internet-Draft  policy and dynamic information migration       June 2011


   [RFC3303]  Srisuresh, P., Kuthan, J., Rosenberg, J., Molitor, A., and
              A. Rayhan, "Middlebox communication architecture and
              framework", August 2002.

8.2.  Informative Reference

   [Data_Center_Fundamentals]
              "Data Center Fundamentals", 2003.


Authors' Addresses

   Gu Yingjie
   Huawei
   No. 101 Software Avenue
   Nanjing, Jiangsu Province  210001
   P.R.China

   Phone: +86-25-56624760
   Fax:   +86-25-56624702
   Email: guyingjie@huawei.com


   Fan Yongbing
   China Telecom
   No. 109 Zhongshan Road West
   Guangzhou, Guangdong Province
   P.R.China

   Phone: 86-20-38639121
   Fax:   86-20-38639487
   Email: fanyb@gsta.com



















Gu & Fan                Expires December 10, 2011              [Page 30]