ARMD Ning So
Internet Draft Verizon
Intended status: Information Track L. Dunbar
Expires: December 2011 Huawei
June 30, 2011
Address Resolution Requirements for VPN-oriented Data Center
Services
draft-so-armd-vdcs-ar-00.txt
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on December 30, 2011.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the BSD License.
So Expires December 30, 2011 [Page 1]
Internet-Draft Address Resolution for VDCS July 1, 2011
Abstract
VPN-oriented data center services seamlessly integrate the computing
and storage resources in data centers and the users together with the
traditional VPN services. This draft describes the address resolution
issues and requirements induced by those services.
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119 0.
Table of Contents
1. Introduction ................................................ 2
2. Terminology ................................................. 3
3. VDCS service description..................................... 3
3.1. Components of VDC ...................................... 4
3.2. Networking related components in support of VDCS ...... 5
4. Address resolution Scaling Issue for VDCS.................... 6
4.1. Address Resolution for VMs attached to L2VPN............ 6
4.2. Address Resolution for VMs attached to L3VPN............ 7
5. Conclusion and Recommendation................................ 9
6. Manageability Considerations................................. 9
7. Security Considerations...................................... 9
8. IANA Considerations ......................................... 9
9. Acknowledgments ............................................. 9
10. References ................................................. 9
Authors' Addresses ............................................ 10
Intellectual Property Statement................................ 10
Disclaimer of Validity ........................................ 11
1. Introduction
VPN-oriented Data Center Services (VDCS) integrate the virtual
resources in data centers and user together using VPN as the common
link. This kind of service is attractive to customers who often do
not want to use public Internet to access data center resources.
VDCS also have more restrictive requirements on what and how the
virtualized data center resources can be shared. In addition, it
provides a common service operational management framework using VPN
as the central control point(s).
So Expires December 30, 2011 [Page 2]
Internet-Draft Address Resolution for VDCS July 1, 2011
2. Terminology
Aggregation Switch: A Layer 2 switch interconnecting ToR switches
Bridge: IEEE802.1Q compliant device. In this draft, Bridge is used
interchangeably with Layer 2 switch.
DC: Data Center
DA: Destination Address
EOR: End of Row switches in data center.
FDB: Filtering Database for Bridge or Layer 2 switch
SA: Source Address
ToR: Top of Rack Switch. It is also known as access switch
VDCS: VPN oriented data center services
VM: Virtual Machines
VPN: Virtual Private Network
VPN-o-CS: VPN oriented Computing Service
3. VDCS service description
Many data centers offer virtualized services today, allowing clients
to lease virtual data center resources without actually owning any
physical servers or storage devices. However, majority of those
services do not include network infrastructure. Intra-data center,
inter-data center networks, and the networks connecting users to data
centers are designed and operated separately from the data center
server/storage systems. It is difficult for customers to integrate
the leased virtual data center resources with their own internal data
center resources, and make those leased resources appearing as if
they come from their internal infrastructure.
VDCS has the following characteristics:
So Expires December 30, 2011 [Page 3]
Internet-Draft Address Resolution for VDCS July 1, 2011
A secure collection of servers and/or virtual machines spanning
one or more data centers.
All the applications running on the Virtual resources in
network provider's data centers are connected with the
enterprise's VPN in the same way as applications running over
enterprise's internal data centers. Therefore, the enterprises
can treat those resources as if they are from their internal
data centers.
Provide the VPN equivalent level of traffic segregation and
privacy for those virtual resources attached to the VPN.
Make the virtual resources' location known to VPN customers.
Created by network provider with no end host configuration.
Allow VMs and user devices using VDCS associated with one VPN
to be partitioned into multiple subnets while still retain the
detailed knowledge of each other.
Allow VPN clients to use private IP addresses (IPv4 or IPv6)
for VDCS.
3.1. Components of VDCS
There are many components in VDCS system, including (but not limited
to):
Network back office support systems, such as provisioning,
billing, and etc,
VPN management systems such as monitoring, reporting, trouble
shooting, and etc.
Data center resource monitoring systems, which include
monitoring the utilization of servers and storage devices in
data centers
Data center resource management systems, which include VMs
placement to servers and racks based on the criteria associated
with VMs.
Others.
So Expires December 30, 2011 [Page 4]
Internet-Draft Address Resolution for VDCS July 1, 2011
This draft only focuses on networking (switching and routing) related
components within VDCS framework.
3.2. Networking related components in support of VDCS
In the figure below, Vx represents a VM or a server belonging to VPN-
x. The data center depicted in the figure has VMs belonging to 5
different VPNs, VPN-1, VPN-2, VPN-3, VPN-4, and VPN-5. Most data
centers have many rows of server racks. Each rack holds many servers
and has 1 or 2 Top of Rack (ToR) switches. Each server can have many
VMs. The ToRs can be connected to aggregation switches/routers, which
are then connected to Data Center gateway switches/routers. In some
data centers, ToRs may be directly connected to Data Center gateway
switches/routers.
It is essential to segregate traffic from VMs belonging to different
VPNs within one data center and across multiple data centers. VLAN is
usually used to segregate traffic from different VPNs within one data
center. However, when a data center needs to house virtual machines
belonging to more than 4095 VPNs, alternative segregation methods
have to be used.
The virtual machines in data center can be connected to VDCS via
L2VPN or L3VPN. For VMs belonging to L3VPN, the data center gateway
router and the VPN PE router have to maintain detailed VRF tables
that contain all the VM IP addresses associated with the each VPN.
For VMs belonging to L2VPN, the data center gateway switch and the
VPN edge switch have to maintain detailed Learned MAC Table that
contains all the VM MAC addresses associated with each VPN.
------------------------------------+-----------------+
Layer 2 based | |
+--------+ | |
|V1|V1|V3|----+ | |
+--------+ | +--------+ | +-----------+ |
+--------+ +---------| DC GW |--|--| | |
|V2|V1|VM|--------------|Switch/ | | | | |
+--------+ +---------|Router |--|--| VPN Edge/| |
+--------+ | | |--|--| Switched | |
|V2|V4|V5|----+ | |--|--| router | |
+--------+ +---------| |--|--| | |
| +-------+--------+ | | | |
+--------+ | | | | | |
|V2|V2|V2|----+ | | | | |
So Expires December 30, 2011 [Page 5]
Internet-Draft Address Resolution for VDCS July 1, 2011
+--------+ | | | | |
+--------+ | | | | |
|V4|V1|V1|------+ | | | |
+--------+ | + ----------+ |
| |
------------------------------------+-----------------+
Figure 1 VMs and Network in Data Center
When VMs belonging to one VPN are partitioned into multiple subnets,
it is necessary to have VLANs or other mechanisms to segregate
traffic from different subnets belonging to one VPN.
4. Address resolution Scaling Issue for VDCS
4.1. Address Resolution for VMs attached to L2VPN
Before severs in a data center are instantiated with VMs for a
particular VPLS L2VPN for the very first time (i.e. there is no VMs
in the data center belonging to the L2VPN yet), the data center
gateway router (CE router) should have the base VPLS configured
already, which means a full mesh of pseudo-wires between L2VPN PEs
already exist. The CE should have an attachment circuit (AC) built
for the VPLS service between CE and PE.
At the time of VDCS instantiation, the new VMs' MAC addresses are
learned and added to the CE and PE's MAC Table, so they can be
learned by other switches and end stations already on the L2VPN in
multiple sites as if they are on one LAN.
When a host or a VM in a data center needs to communicate with
another host/VM in the L2VPN, an ARP (IPv4) or a ND(IPv6) is flooded
to all PWs and all ACs (except the one from which the request is
coming from).
Under this scenario, all VMs' MAC addresses belonging to a particular
L2VPN are visible to each other. And the L2VPN's PEs and VSIs have to
learn and maintain the MAC and VLAN addresses for all the hosts/VMs
associated with this L2VPN. This may leads to address table
scalability problems for data center VSI and L2VPN PE.
For example, assuming there are 1000 L2VPNs with hosts/VMs residing
in this data center. That translates to 1000 VSIs on the CE, with
So Expires December 30, 2011 [Page 6]
Internet-Draft Address Resolution for VDCS July 1, 2011
each VSI containing the entire MAC and VLAN mapping for all the
switches and end-stations associated with all the L2VPNs. This
requires a very large amount of memory for the data center gateway
switch/router using current technology.
------------------------------------------------------+
Layer 2 based |
+--------+ | |
|V1|V1|V3|----+ | |
+--------+ | +--------+ | +-----------+ |
+--------+ +---------| DC GW |--|--| +----+ | |
|V2|V1|VM|--------------| | | | |VSI1| | |
+--------+ +---------| |--|--| +----+ | |
+--------+ | | |--|--| +----+ | |
|V2|V4|V5|----+ | |--|--| |VSI2| | |
+--------+ +---------| |--|--| +----+ | |
| +-------+--------+ | | |VSI3| | |
+--------+ | | | | +----+ | |
|V2|V2|V2|----+ | | | |VSI4| | |
+--------+ | | | +----+ | |
+--------+ | | | |VSI5| | |
|V4|V1|V1|------+ | | +----+ | |
+--------+ | + ----------+ |
| PE |
------------------------------------+-----------------+
Figure 2 L2VPN associated VMs in Data Center
4.2. Address Resolution for VMs attached to L3VPN
When severs in a data center are instantiated with VMs for a
particular L3VPN for the very first time (i.e. there were no VMs in
the data center belonging to the L3VPN yet), it assumes that all the
necessary L3VPN configuration has already been completed on the data
center gateway router (CE) and the L3VPN edge router (PE). There are
two scenarios for VMs attached to L3VPN:
Scenario 1: all the VMs belonging to the L3VPN client are added
as a separate site for the L3VPN. Under this scenario, the
provider data center becomes the additional site (or peers) to
the L3VPN.
So Expires December 30, 2011 [Page 7]
Internet-Draft Address Resolution for VDCS July 1, 2011
Scenario 2: Hosts or applications in client's own data centers
(or premises) see those VMs attached to L3VPN as if they are
from the same subnets. Under this scenario, the traditional
"subnet" concept is broken. VMs in the data center have to be
connected to their designated sites as if they are in one
subnet.
Under scenario 1, the APR/ND broadcast/multicast requests are
terminated at the CE. Similar to the condition described in the last
section on VMs attached to L2VPN, all IP addresses associated with
all L3VPNs in the data center have to be learned and maintained at
the CE and the L3VPN PE router.
This can require a very large amount of memory on the CE and PE
router using today's technology, especially when the CE and the PE
routers are hosting both L2VPN and L3VPN simultaneously. The amount
of memory requirement is even larger if those VMs addresses can't be
aggregated.
In addition, it is possible that IP addresses for VMs belonging to
different VPNs could be duplicated.
------------------------------------------------------+
Layer 2 based | |
+--------+ | |
|V1|V1|V3|----+ | |
+--------+ | +--------+ | +-----------+ |
+--------+ +---------| DC GW |--|--| +----+ | |
|V2|V1|VM|--------------|Switches| | | |CE1 | | |
+--------+ +---------| |--|--| +----+ | |
+--------+ | | |--|--| +----+ | |
|V2|V4|V5|----+ | |--|--| |CE2 | | |
+--------+ +---------| |--|--| +----+ | |
| +-------+--------+ | | |CE3 | | |
+--------+ | | | | +----+ | |
|V2|V2|V2|----+ | | | |CE4 | | |
+--------+ | | | +----+ | |
+--------+ | | | |CE5 | | |
|V4|V1|V1|------+ | | +----+ | |
+--------+ | + ----------+ |
| DC Gateway |
------------------------------------+-----------------+
Figure 3 L3VPN associated VMs in Data Center
So Expires December 30, 2011 [Page 8]
Internet-Draft Address Resolution for VDCS July 1, 2011
Under the Scenario 2, the ARP/ND messages from the VMs in the data
center have to be flooded to the corresponding sites to which those
VMs belonging. The data center gateway routers (CEs or PEs) have to
do both L2VPN and L3VPN.
5. Conclusion and Recommendation
Future data center can scale up to millions of virtual machines.
Theoretically, network service provider can make their data centers
hosting VMs for all of their VPN clients. Using current technology,
it is very difficult for routers in data center and at network edge
facing the data center to maintain all the VSIs or VRFs needed for
the huge number of VPNs and the VPN-associated VMs being deployed.
Therefore, we recommend ARMD WG to investigate alternative solutions
on address resolution and address scalability issues to make data
center gateway routers capable of supporting the VPN oriented data
center services.
6. Manageability Considerations
This document does not add additional manageability considerations.
7. Security Considerations
This document has no additional requirement for security.
8. IANA Considerations
9. Acknowledgments
We want to acknowledge the following people for their valuable inputs
to this draft: K.K.Ramakrishnan.
This document was prepared using 2-Word-v2.0.template.dot.
10. References
[VDCS] So, et al, "Requirement and Framework for VPN-Oriented Data
Center Services", draft-so-vdcs-00, June 2011.
[ARP] D.C. Plummer, "An Ethernet address resolution protocol."
RFC826, Nov 1982.
So Expires December 30, 2011 [Page 9]
Internet-Draft Address Resolution for VDCS July 1, 2011
[Microsoft Windows] "Microsoft Windows Server 2003 TCP/IP
implementation details."
http://www.microsoft.com/technet/prodtechnol/windowsserver2
003/technologies/networking/tcpip03.mspx, June 2003.
[Scaling Ethernet] Myers, et. al., " Rethinking the Service Model:
Scaling Ethernet to a Million Nodes", Carnegie Mellon
University and Rice University
[Cost of a Cloud] Greenberg, et. al., "The Cost of a Cloud: Research
Problems in Data Center Networks"
[Gratuitous ARP] S. Cheshire, "IPv4 Address Conflict Detection", RFC
5227, July 2008.
Authors' Addresses
Ning So
Verizon Inc.
2400 N. Glenville Ave.,
Richardson, TX75082
ning.so@verizonbusiness.com
Linda Dunbar
Huawei Technologies
5340 Legacy Drive, Suite 175
Plano, TX 75024, USA
Phone: (469) 277 5840
Email: ldunbar@huawei.com
Intellectual Property Statement
The IETF Trust takes no position regarding the validity or scope of
any Intellectual Property Rights or other rights that might be
claimed to pertain to the implementation or use of the technology
described in any IETF Document or the extent to which any license
So Expires December 30, 2011 [Page 10]
Internet-Draft Address Resolution for VDCS July 1, 2011
under such rights might or might not be available; nor does it
represent that it has made any independent effort to identify any
such rights.
Copies of Intellectual Property disclosures made to the IETF
Secretariat and any assurances of licenses to be made available, or
the result of an attempt made to obtain a general license or
permission for the use of such proprietary rights by implementers or
users of this specification can be obtained from the IETF on-line IPR
repository at http://www.ietf.org/ipr
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
any standard or specification contained in an IETF Document. Please
address the information to the IETF at ietf-ipr@ietf.org.
Disclaimer of Validity
All IETF Documents and the information contained therein are provided
on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE
IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL
WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY
WARRANTY THAT THE USE OF THE INFORMATION THEREIN WILL NOT INFRINGE
ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
So Expires December 30, 2011 [Page 11]