Y. Yu
IRCAT Working Group T. Tsou
Internet Draft R.Li
Intended status: Standard Track Huawei
Expires: December 2011 June 11, 2011
Inter-router Cooperative and Teaming for massive scalability
draft-yu-problem-statement-01.txt
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on December 11, 2011.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Yu&Tsou&Li Expires December 11, 2011 [Page 1]
Internet-Draft IRCAT Problem Statement June 2011
Abstract
To achieve the extremely high scalability, Cooperative router group
is a reasonable approach. Problems are stated in this document
regarding cooperative router group. New mechanisms are needed to be
developed and to be standardized.
Table of Contents
1. Introduction...................................................2
2. Terminology....................................................3
3. Inter Cooperative Router Group.................................3
4. Problem Statement..............................................4
4.1. Router Group Membership...................................4
4.2. Router Capability Exchange................................4
4.3. Cooperative Router Group Topology.........................4
4.3.1. An Example of Interior Router Group Network..........5
4.4. Cooperative Router Group Working Load Collaboration.......6
4.4.1. Local Resources Monitoring...........................6
4.4.2. Router Working Load Migration........................6
4.5. Unified Router Group Control Plane........................7
4.6. Router Control Plane and Forwarding Plane Separation......7
5. Conclusions and Recommendations................................7
6. Security Considerations........................................8
7. IANA Considerations............................................8
8. Acknowledgments................................................8
9. Reference:.....................................................8
1. Introduction
To achieve scalability, routers are replaced and are becoming larger
and larger. One way to achieve such a very high scalability is to
design and deploy tightly-coupled homogeneous hardware cluster
router.
Currently, the tightly-coupled homogeneous hardware cluster router
requests the same type of router from same vendor, which means they
are identical.
There could be another possibility of using different routers (maybe
from different vendors) to construct a loosely-coupled heterogeneous
cluster router by running one or more protocols to be standardized.
The key point of such a technology is about Cooperation and Teaming
among routers.
Yu&Tsou&Li Expires December 11, 2011 [Page 2]
Internet-Draft IRCAT Problem Statement June 2011
The cooperation includes, but not limited to, such scenario in which
one router may handle some control plane work on behalf of another
router. From the outside, the cooperative router group is required to
meet the following minimal requirements:
- They are viewed as a single router with its own router ID and is
operated as a single network node.
- They are always available and some redundancy and backups are built
between routers joining the same router group.
- They are extremely highly scalable as a new router can be inserted
to join such a group when needed.
2. Terminology
IRCAT: Inter Router Cooperative and Teaming
VRF: Virtual Routing and Forwarding
3. Inter Cooperative Router Group
To meet the massive scalability from the newer emerging application,
single chassis router or single entity router faces tough time. Even
though CPU and memory speed grow very fast, building a giant router
is still expensive. Single giant router loses the flexibility of
seamlessly scales from low to high. So unite a group of routers as a
single router is reasonable approach --- inter router cooperative
group. In the cooperative router group, routers have same router ID
and collaborate like single router to network. It is not required
that routers are the same type. They can be heterogeneous or
homogeneous routers in the cooperative router group.
Cooperative router group is cooperative. Routers in group are
collaborative to working on same tasks. For example, in cooperative
router group, BGP is not running on single router. It is running on
multiple routers in the router group. All BGP instances are load
balanced the whole BGP load. They have same router ID and same IP
addresses. The external BGP peers all connect to those common IP
addresses.
Cooperative router group can be seamlessly scale up and down without
disruption of services. New routers can be added into or remove from
Yu&Tsou&Li Expires December 11, 2011 [Page 3]
Internet-Draft IRCAT Problem Statement June 2011
cooperative router group corresponding to the load of router group
goes up and down.
To make routers to be cooperative, cooperative router group is facing
a series of problems. Some of problems can be solved by some existed
mechanisms. But for some problems, new mechanisms are necessary to be
developed and further study is needed.
4. Problem Statement
4.1. Router Group Membership
When a new router is added into the router group, it won't be member
of router group automatically. Automatic discovery, negotiation,
authentication etc. are the necessary procedures. After newly added
router becomes member of router group, its resources are added into
router group global resources. If a router leaves the router group,
router group must be able to detect this leave and remove this slice
of resources from global resources.
Router group membership mechanism may borrow some existing ideas, but
it still has some uniqueness under the circumstance of cooperative
router group.
4.2. Router Capability Exchange
With the join of new group member, how much new resources are added
in? Router capability is advertised out. Capability here refers the
router's used and available CPU and memory capacities etc.
Cooperative router group needs to know the whole resources, so
capability database of all members should be kept in router group.
Because capability of each member can be changed, capability database
is updated from time to time.
4.3. Cooperative Router Group Topology
Router group is cooperative and collaborative; this means members of
router group have interior interconnections and internal
communication. Interior network of router group can either be layer 2
or layer 3. Layer 2 interior network is simple and easy. Layer 3
interior network is better scaled and has some advances in multicast.
Each member of router group has private addresses for internal
communication. Those addresses are concealed to the outside of the
cooperative router group.
Yu&Tsou&Li Expires December 11, 2011 [Page 4]
Internet-Draft IRCAT Problem Statement June 2011
Meanwhile, cooperative router group keeps its router ID and a set of
public addresses. The outside of router group only knows the above
public information. In section 4.3.1, an example explains more about
private and public addresses of interior router group network.
Router group topology has to be obtained if router members need
internal unicast and multicast communication. Mac learning in layer 2
network or slightly changed IGP routing protocol in layer 3 networks
can do this job well.
But in cooperative router group, it needs more.
- Router group only has topology information for private addresses.
But it dose not have public address locations.
- In router group circumstance, same public IP addresses are
possible on multiple member routers because multiple instances of
application are running on different routers. For example, BGP in
router group may not be running on single member routers. It has
multiple instances which are running on multiple routers. Those BGP
instances have same router ID and common address to outside peers. If
BGP peer packet which has destination to the common address, are punt
into router group, but faces multiple places with same address, where
should this packet be forwarded?
4.3.1. An Example of Interior Router Group Network
10.1.1.1 10.1.1.2 10.1.1.3
+-----------+ +-----------+ +-----------+
| router A | | router B | | router C |
| (BGP 1) | ----- | (BGP 2) | ------- | (OSPF) |
| (1.1.1.1) | | (1.1.1.1) | | |
+-----------+ +-----------+ +-----------+
| LC | | LC | | LC |
+-----------+ +-----------+ +-----------+
| | | | | | | | |
Figure 1 Interior Router Group Network example
As Figure 1, 3 routers A, B and C are in cooperative router group.
Their private IP addresses are 10.1.1.1, 10.1.1.2, 10.1.1.3
respectly. Router group has public address 1.1.1.1. There are 2 BGP
instances which are running on router A and B. OSPF instance is
running on router C. BGP is load balanced, half of BGP peers are
handled by router A and second half peers are handled by router B.
BGP peers are all connect to IP address 1.1.1.1.
In this example, the following scenarios are needed to be addressed.
Yu&Tsou&Li Expires December 11, 2011 [Page 5]
Internet-Draft IRCAT Problem Statement June 2011
- If a BGP packet is punt up, its destination IP address is 1.1.1.1
and the sent peer is supposed to be handled by BGP instance on router
A, forward this packet to router A.
- If a BGP packet is punt up, its destination IP address is 1.1.1.1
and the sent peer is supposed to be handled by BGP instance on router
B, forward this packet to router B.
- If an OSPF packet is punt up, forward this packet to router C.
It shows a new mechanism needed to be developed in interior router
group network.
4.4. Cooperative Router Group Working Load Collaboration
Router group is cooperative. It is not right if some member routers
are extremely busy or incur to almost memory panic, in the meantime
some other member router have plenty resource but have nothing to do.
Under this circumstance, cooperative router group should have a
mechanism to move a part of working load from resource exhausted
routers to some others which have enough resources. Otherwise
cooperative router group is not "cooperative".
Ideally this working load collaboration is automatic and dynamic.
Working load move should be happened before router's resources are
really exhausted.
4.4.1. Local Resources Monitoring
It is too late if router completely run out of resource and then
remembers it can move some working load out. Router monitors its
local resource status from time to time. If it goes to alarm stage,
working load migration triggers.
4.4.2. Router Working Load Migration
If a router needs to off load some working load. It always it handles
this with 2 different manners. The first one is, this router does not
have other group member's capability information. It just asks for
help loudly (broadcast) and then waits for helper's responses. After
getting almost all helper's responses, now the router knows the
targets of load migration. Then it negotiates with target routers and
load migration starts if negotiation success. The second manner, this
router always collects group member's capability information, i.e.
capability database. So when the router needs to offload its working
load, it already has enough knowledge to know the helpers (migration
targets). This router can directly negotiates with migration target
Yu&Tsou&Li Expires December 11, 2011 [Page 6]
Internet-Draft IRCAT Problem Statement June 2011
routers and load migration starts if the negotiation success. The
optimal migration targets selection is a complex process, how to
simplify and compromise is another problem.
4.5. Unified Router Group Control Plane
Each router has its own control plane, so there are quite a few
control planes in a cooperative router group. On the other hand,
routers can be added into and remove from router group dynamically
and seamlessly. How to effectively utilize the control planes'
resource is very tough problem. Control plane virtualization becomes
the reasonable approach. A unified router group control plane becomes
necessary. By virtualization, the unified router group control plane
has global control plane resources pool and each router's control
plane becomes a slice of resources in global control plane resource
pool. When a new router joins into the router group, a new slice of
router control plane resources is added into global resource pool and
vice versa.
4.6. Router Control Plane and Forwarding Plane Separation
This problem is related to the router group control plane
virtualization. It is natural in single router that control plane
only manages the local chassis forwarding hardware. In cooperative
router group circumstance, especially with the unified router group
control plane, control plane on other remote router can manages local
forwarding hardware on behalf of local router control plane. So one
forwarding hardware and one control plane mapping is not true under
cooperative router group circumstance. Forwarding hardware can no
longer find out its specific control plane, but to face the unified
router group control plane.
5. Conclusions and Recommendations
To meet the extremely high scaling requirements from service
provider, we have concluded and recommend that:
- Cooperative router group is practical and effective approach to
meet the extremely high scaling requirement.
- Cooperative router group addressed new mechanisms needed to be
developed and further study and to be standardized.
- Inter Router Cooperative and Teaming (IRCAT) is to standardize
the protocols which are running on heterogeneous routers to let
them construct as cooperative router group.
Yu&Tsou&Li Expires December 11, 2011 [Page 7]
Internet-Draft IRCAT Problem Statement June 2011
6. Security Considerations
In cooperative router group, member routers are cooperative and
trusted. A malicious group member can easily down the entire router
group. So the sophisticated authentication is important when member
joins the router group.
Routers in router group still may be attacked from outside network.
7. IANA Considerations
This document does not add additional Security Considerations.
8. Acknowledgments
Cathy Wu helped to translate the format of this document. She is
gratefully acknowledged.
This document was prepared using 2-Word-v2.0.template.dot.
9. Reference:
[1] [CloudNET] T. Wood, "The Case for Enterprise-ready Virtual
Private Clouds"
[2] [RFC4271] Y. Rekhter, T. Li, S. Hares, "A Border Gateway
Protocol 4 (BGP-4)", RFC4271, 2006
Author's Addresses
Yu&Tsou&Li Expires December 11, 2011 [Page 8]
Internet-Draft IRCAT Problem Statement June 2011
Yang Yu
Huawei Technologies
2300 Central Expressway,
Santa Clara, CA 95050, USA
Email:yyu@huawei.com
Tina Tsou
Huawei Technologies
2300 Central Expressway,
Santa Clara, CA 95050, USA
Email:tena@huawei.com
Renwei Li
Huawei Technologies
2300 Central Expressway,
Santa Clara, CA 95050, USA
Email:renwei.li@huawei.com
Copyright (c) 2011IETF Trust and the persons identified as authors of
the code. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
o Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
Yu&Tsou&Li Expires December 11, 2011 [Page 9]
Internet-Draft IRCAT Problem Statement June 2011
o Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with the
distribution.
o Neither the name of Internet Society, IETF or IETF Trust, nor the
names of specific contributors, may be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Yu&Tsou&Li Expires December 11, 2011 [Page 10]