Network Working Group Y. Gu Internet-Draft Huawei Intended status: Standards Track July 1, 2011 Expires: January 4, 2012 Policies and dynamic information migration in DCs: Solution Survey draft-gu-opsawg-policies-migration-solution-survey-00 Abstract This draft enumerates several kinds of potential solutions that may resolve the problem described in I-D.gu-opsawg-policies-migration. The goal is to, on one hand, make deeper analysis to this problem and, on the other hand, provide informaiton for gap analysis, and authors of gap analysis can make research on the potential solutions to find out whether there are existing protocols or mechanisms can be reused to compose a specific potential solution. The solutions introduced here are just high level thought and, if one solution is regarded as a preferred way, we need to analyze and define, if needed, detail mechanism and protocols in future. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on January 4, 2012. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents Gu Expires January 4, 2012 [Page 1]
Internet-Draft policy and dynamic information migration July 2011 carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminologies and concepts . . . . . . . . . . . . . . . . . . 3 3. Potential Solutions Overview . . . . . . . . . . . . . . . . . 3 4. Centralized Solution . . . . . . . . . . . . . . . . . . . . . 5 5. Middle Solution . . . . . . . . . . . . . . . . . . . . . . . 7 6. Distributed Solutions . . . . . . . . . . . . . . . . . . . . 9 6.1. D-PUSH Solution . . . . . . . . . . . . . . . . . . . . . 9 6.2. D-PULL Solution . . . . . . . . . . . . . . . . . . . . . 10 6.3. Pros and Cons of Distributed solutions . . . . . . . . . . 11 7. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 12 8. Security Considerations . . . . . . . . . . . . . . . . . . . 12 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 12 9.1. Normative Reference . . . . . . . . . . . . . . . . . . . 12 9.2. Informative Reference . . . . . . . . . . . . . . . . . . 13 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 13 Gu Expires January 4, 2012 [Page 2]
Internet-Draft policy and dynamic information migration July 2011 1. Introduction At the beginning of the draft, we briefly introduce the problem stated in I-D.gu-opsawg-policies-migration. Virtualization is widly used in Data Center, and VMs are allowed to do hot, or live, migration from one location to another location within or between data centers. VMs migration implies OS/software/memory copy on server side and policies, including static policies and dynamic information learned by network devices, migration on network side. I-D.gu-opsawg-polices-migration and this draft focus on policies migration on network side. In this draft, we first list some general requirements for solutions. Based on the requirements and the essence of this problem, the author of this draft envision three potential solutions for policies migration. The author would enphasize that it does not necessarily mean that final solution will be one of the potential solutions, which are just ways to help people understand the problem. People may find more suitable solution beyond these potential solutions in the future and the author would like to work on that. In the following sections, detail examples and senarios are given to explain why and which polices need to migrate with VM. 2. Terminologies and concepts The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. The document uses terms defined in [I-D.gu-opsawg-policies-migration]and [I-D.wang-opsawg-policy-migration-gap-analysis]. 3. Potential Solutions Overview There are three important components in this problem: VMs, source device and destination devices. The diagram in I-D.gu-opsawg- policies-migration can be abstrated as in Fig.1. COORDINATOR is a new entity introduced in this draft. It's noted as optional because it is not always required in all potential solutions introduced in this draft. When VM is running on original location, static policies have been configured on source devices and dynamic information has been learned by source devices. When we say POLICIES in the following text in this draft, it refers to both static policies and dynamic information. The potential solutions need at least three Gu Expires January 4, 2012 [Page 3]
Internet-Draft policy and dynamic information migration July 2011 mechnism to achieve policies migraiton: o Migration Notification: The Migration Notification message is a trigger to source and destination devices to do policies migration. The migration Notification message could be triggered by VM or a centralized manager, which could be network manager or a separated function entity who communicates with VM Manager and obtain VM's status in migration, call it COORDINATOR. Anyway, the Migration Notification message should be able to do either or both of the following actions: * Notify the source devices that a VM is going to emigrate and it's good time to move the VM's policies. The message must include a VM ID to uniquely identify the VM, The message may also include the ID of destination devices that VM is going to move to. The destination devices ID could be IP Address or other identifier that source devices can connect to. * Notify the destination devices that a VM is going to immigrate here and it's good time to obtain the VM's policies. The message must include a VM ID to uniquely identify the VM, The message may also include the ID of source devices that VM is going to move to. The source devices ID could be IP Address or other identifier that source devices can connect to. o Policies Transfering: Policies could be transferred directly between network devices, or be transferred through a third party, i.e. the COORDINATOR. o Migration Feedback: The Migration Feedback mesage can let VM manager or Hypervisor know whether policies are successfully migrated. If it's successfully migrated and enabled on destination devices, the VM can be resumed on new location. Otherwise, the VM migration must be stopped and VM need to be resumed on original location. Gu Expires January 4, 2012 [Page 4]
Internet-Draft policy and dynamic information migration July 2011 |-.-.-.-.-.-.-.-.-.-.-.-.-| . ------------------ . | | COORDINATOR | | Optional . ------------------ . Component |-.-.-./-.-.-.-\.-.-.-.-.-| / \ / \ / \ ----------------- ---------------------- |Source devices | |Destination devices | ----------------- ---------------------- | | | | | | | | ---------- ---------- | VM1 | | new VM1| ---------- ---------- Figure 1: Key componnets in policies migration In the following sections, different kind of potential solutions have been introduced, together with pros and cons of each solution. Finally, the author conclude the solutions and suggest one of the potential solutions. But the author would like to discuss with people and would like to explore better solutions beyond what have be introduced. 4. Centralized Solution A centralized entity take charge of policies migration between source and destination devices. To describe the centralized solution briefly, the centralized entity, or name it Coordinator, communicates with VM Manager to get VM's status. Before VM really migrates, the COORDINATOR should have already know the source and destination location of this VM. When COORDINATOR finds a specific VM is suspending, it requests for DI from source devices, according to the network topology. Then COORDINATOR sets the policies to corresponding destination devices. COORDINATOR sends feedback to VM Manager, and the latter decides whether to resume VM on destination server. Fig. 2 shows the process. Gu Expires January 4, 2012 [Page 5]
Internet-Draft policy and dynamic information migration July 2011 1. Migration -------------------- Notification ------------ | COORDINATOR |<-------------> |VM Manager| -------------------- 4.Feedback ------------ /\ /\ \ \ /\ 2. Get / / \ \ 3. Set | Policies / / \ \ Policies | / / V \ | / ----------- ----------- | | | | Source | | Dest. | | | | | Upper | | Upper | | | bi-directional | | Devices | | Devices | | | API | ----------- ----------- | | | V | ----------- ----------- | | Source | | Dest. | | | Bridge | | Bridge | | ----------- ----------- | | | | ---------- ---------- | | VM1 | |new VM1 | <---------- ---------- ---------- Figure 2: Centralized solution The advantages of centralized solution are: o Complete knowledge of network topology: The COORDINATOR can learn network topology from network manager, or it could be a network manager itself. It's not difficult to include Virtual switch in physical server into the network topology, hence the COORDINATOR can has a whole topology including both physical and virtual devices. So it's easier for COORDINATOR to learn which source adjacent bridge the VM is attaching to, and which are the upper layer devices. Also, the COORDINATOR can learn the destination adjacent bridge and destination devices. Even in asymmetric architecture, the COORDINATOR with some intelligence can analyze network structure and make decision on which devices are the partner. o Easier Feedback: the COORDINATOR can collect policies migration results for multiple devices, and, since it knows how many devices need to migrate policies, feedback the VM manager with SUCCESS or FAIL according to the statistic results of policies migration results from all related devices. o Full Control and easier logging: Policies migration is under control of COORDINATOR. Gu Expires January 4, 2012 [Page 6]
Internet-Draft policy and dynamic information migration July 2011 The disadvantages of centralized solution are: o Capacity bottleneck: Usually VM won't migrate frequently, but many VMs may migrate at one time. For example, VMs belonging to the same client may migrate from one location to another at one time, or VMs at one geographic location may migrate to another geographic location at one time for seamless disaster avoidance. If lots of VM migration happens at one time, the COORDINATOR could be a capacity bottleneck. o Longer service suspending time: Because COORDINATOR has to collect policies from source devices and then set them to destination devices, it will take longer time to finish policies migration process compared with Distributed solutions and Middle solutions. But the process could be shorten if upload and download of policies are allowed to happen synchronously. In order to mitigate the disadvantages of Centralized Solution, an idea of Middle solution is explored. 5. Middle Solution A middle solution is neither a fully distributed or fully centralized solution. A COORDINATOR device is placed to ease the migration notification and migration feedback process. But less workload is puted on COORDINATOR than Centralized solution which increases capacity and extensibility, and few or no updating requirements for software updating when devices functions are updated. A centralized entity is deployed to assist migration notification and asymetric policies migration. The basic process is: the COORDINATOR gets VM's status from VM Manager. Then the COORDINATOR sends migration notification to source devices according to network topology. Source devices upload corresponding DI to COORDINATOR. Meanwhile or shortly later, COORDINATOR notifies destination devices to download DI. After all polices are adopted by destination devices, COORDINATOR sends feedback to VM Manager. Gu Expires January 4, 2012 [Page 7]
Internet-Draft policy and dynamic information migration July 2011 1. Migration --------------------------- Notification --------------- | COORDINATOR |-----------------| VM Manager | --------------------------- 6. Feedback --------------- /\ /\ /\ /\ /\ 3. Upload DI/ / 3. 5.\ \ 5.Download DI | / / \ \ | / / \ \ | / V 2. 2. V \ | / ----------- ----------- \ | | | Source | | Dest. | | | | | Upper | | Upper | | | | | Devices | | Devices | | | | ----------- ----------- | |API | | | | | | / \ | | 2.V / \ V 4. Migration | ----------- ----------- Notification | | Source | | Dest. | | | Bridge | | Bridge | | ----------- ----------- | | | | | | | ---------- ---------- | | VM1 | |new VM1 |---------------------- ---------- ---------- Figure 3: Middle Solution The difference between Middle solution and Centralized solution is that policies in Middle solution are transparent to COORDINATOR, i.e. COORDINATOR can act as a DI transfer database without knowleadge of the content of DI. The software on COORDINATOR does not have to be updated every time a new element function is defined that creates or uses new DI. So the coordination functions provided by the COORDINATOR is as simple and general as possible. The advantages of middle solution are: Controllable: The COORDINATOR can be intelligent to assist Administrators to trace and control policies migration. Easier logging. General solution: The COORDINATOR needn't be aware of the content of DI, so that the COORDINATOR needn't be updated even new DI is supported. Gu Expires January 4, 2012 [Page 8]
Internet-Draft policy and dynamic information migration July 2011 Easier feedback: COORDINATOR can be aware whether all policies migration have completed. Then the COORDINATOR can feedback to VM Manager with SUCCESS or FAIL, and the latter will make decision on where to resume the VM. Good extensibility: Since COORDINATOR is only a coordinator to indicate migration notification and partnership, there won't be huge workload on it even when many VMs migrated at one time. The COORDINATOR in this case can support much larger scale of VM migration then centralized solution and middle solution with distributed migration notification. Quick policies migration: COORDINATOR is a DI transfer database, and it needn't GET and SET policies by itself. So the workload on COORDINATOR is much less than Centralized solution. The disadvantages of middle solution are: New mechanism is needed to support DI migration asymmetric structure. 6. Distributed Solutions 6.1. D-PUSH Solution A fully distributed solution doesn't need a COORDINATOR. To describe the Distributed solution briefly, the source Hypervisor, who knows about the VM's Status send VDP(VSI Discovery and Configuration Protocol, defined in IEEE802.1 Qbg) Associate message, with S-Bit set, to adjacent bridge. VM's ID is carried in VDP Associate mesasge, and S-Bit =1 means the specific VM is suspending now. The adjacent bridge distributes the notification to upper layer devices, including bridges and middleboxes. Then all related devices transfered the policies, that is relevant to the specific VM, to corresponding destination devices. When all policies have been migrated, successfully or failed, the source devices send feedback message to original Hypervisor. Let's call this process as D-PUSH, i.e. distributed solution with source devices push policies to destination devices. Fig. 4 shows the process. Gu Expires January 4, 2012 [Page 9]
Internet-Draft policy and dynamic information migration July 2011 ----------- ----------- | Upper |----------->| Upper | | Devices | 3.Transfer | Devices | ----------- ----------- 4. | ^ 2. Feedback | | Distribute | | Migration V | Notification ----------- ----------- | Source |----------->| Dest. | | Bridge | 3.Transfer | Bridge | ----------- ----------- | ^ 1. 4. | | Migration Feedback | | Notification | | VDP Associate V | (S-Bit=1) ---------- ---------- | VM1 | |new VM1 | ---------- ---------- Figure 4: Distributed solution with source devices PUSH policies to destination devices 6.2. D-PULL Solution An alternative distributed solution is D-PULL solution. The destination Hypervisor sends a VDP Associate message with M-Bit set, which means the specific VM, identified by the VMID in VDP message, is migrating. Upon receving the VDP Associate with M-Bit =1, the adjacent bridge distribute the VM's status to upper layer devices, including bridges and middleboxes. Then the destination devices will PULL the policies from corresponding devices. Fig. 5 shows the process. Gu Expires January 4, 2012 [Page 10]
Internet-Draft policy and dynamic information migration July 2011 ----------- ----------- | Upper |----------->| Upper | | Devices | 3.Transfer | Devices | ----------- ----------- 4. | ^ 2. Feedback | | Distribute | | Migration V | Notification ----------- ----------- | Source |----------->| Dest. | | Bridge | 3.Transfer | Bridge | ----------- ----------- | ^ 1. 4. | | Migration Feedback | | Notification | | VDP Associate V | (M-Bit=1) ---------- ---------- | VM1 | |new VM1 | ---------- ---------- Figure 5: Distributed solution with destination devices PULL policies from source devices 6.3. Pros and Cons of Distributed solutions D-PUSH and D-PULL solution have common pros and cons. Common advantages of D-PUSH and D-PULL solutions: o Good extensibility: Since the whole process are executed by network devices, without any centralized entity involving, there is no capacity bottleneck, which usually happens at centralized entity. Common disadvantages of D-PUSH and D-PULL solutions: o Out of control: Policies are migrated totally by network devices, administrator can not control them. Administrator may lose the trace of the policies, especially when network administrators have no idea of where the VM has gone. There might be security risk, e.g. policies conflict with existing policies on destination devices, or new policies can not be supported by destination devices. Additional solutions need to be defined to resolve these problems. o Lack of information: Currently, VDP doesn't carry destination or source devices addresses. That means, when source or destination Gu Expires January 4, 2012 [Page 11]
Internet-Draft policy and dynamic information migration July 2011 adjacent bridges receives the VDP messages, they only know about the VM ID and VM status, and have no idea of the IP addresses of destination or source devices. Without the information of source or destination devices, it's not feasible to transfer policies directly between source and destination devices. Either extend VDP or take advantage of a centralized entity to notify IP addresses of source or destination devices. o Hard to resolve asymmetric architecture: if the source and destination network structures are asymmetric, it's hard for network devices to decide where to PUSH or PULL the policies. In fact, it's because network devices have limited knowledge of network topology. o Hard to feedback: there might be more than one device that possess policies related to the specific VM. However the Hypervisor has no idea of how many devices have and need to migrate VM-related policies. Assuming each device feedback its policies migration results to Hypervisor, how could Hypervisor knows whether the received results are all the results it needs? 7. Conclusion This draft has introduces several kinds of potential solutions for problem stated in I-D.gu-opsawg-policies-migration. Pros and cons are listed for each solution. And according to the pros and cons analysis, the author feels that Middle solution with centralized migration notification is better. However all these solutions are just high level thought. The author welcome people who are interested in resolving this problem to work together to define a good solution for the problem. 8. Security Considerations Security mechanism should be considered and resolved no matter which solution is finally choosed. 9. References 9.1. Normative Reference [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", March 1997. [RFC3303] Srisuresh, P., Kuthan, J., Rosenberg, J., Molitor, A., and Gu Expires January 4, 2012 [Page 12]
Internet-Draft policy and dynamic information migration July 2011 A. Rayhan, "Middlebox communication architecture and framework", August 2002. 9.2. Informative Reference [I-D.gu-opsawg-policies-migration] Gu, Y. and Y. Fan, "I-D.gu-opsawg-policies-migration", 2011. [I-D.wang-opsawg-policy-migration-gap-analysis] Wang, D. and Y. Gu, "I-D.wang-opsawg-policy-migration-gap- analysis", 2011. Author's Address Gu Yingjie Huawei No. 101 Software Avenue Nanjing, Jiangsu Province 210001 P.R.China Phone: +86-25-56624760 Fax: +86-25-56624702 Email: guyingjie@huawei.com Gu Expires January 4, 2012 [Page 13]