Using BGP for routing in large-scale data centers
draft-lapukhov-bgp-routing-large-dc-01

The information below is for an old version of the document
Document Type Active Internet-Draft (individual)
Last updated 2012-07-15
Replaced by RFC 7938, RFC 7938
Stream (None)
Intended RFC status (None)
Formats pdf htmlized (tools) htmlized bibtex
Stream Stream state (No stream defined)
Consensus Boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date
Responsible AD (None)
Send notices to (None)
IDR                                                          P. Lapukhov
Internet-Draft                                           Microsoft Corp.
Intended status: Informational                                 A. Premji
Expires: January 15, 2013                                Arista Networks
                                                           July 14, 2012

           Using BGP for routing in large-scale data centers
                 draft-lapukhov-bgp-routing-large-dc-01

Abstract

   Some service providers build and operate data centers that support
   over 100,000 servers.  In this document, such data-centers are
   referred to as "large-scale" data centers to differentiate them the
   from more common smaller infrastructures.  The data centers of this
   scale have a unique set of network requirements, with emphasis on
   operational simplicity and network stability.

   This document attempts to summarize the authors' experiences in
   designing and supporting large data centers, using BGP as the only
   control-plane protocol.  The intent here is to describe a proven and
   stable routing design that could be leveraged by others in the
   industry.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on January 15, 2013.

Copyright Notice

   Copyright (c) 2012 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal

Lapukhov & Premji       Expires January 15, 2013                [Page 1]
Internet-Draft     draft-lapukhov-bgp-routing-large-dc         July 2012

   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Traditional data center designs  . . . . . . . . . . . . . . .  3
     2.1.  Layer 2 Designs  . . . . . . . . . . . . . . . . . . . . .  3
     2.2.  Fully routed network designs . . . . . . . . . . . . . . .  4
   3.  Document structure . . . . . . . . . . . . . . . . . . . . . .  5
   4.  Network design requirements  . . . . . . . . . . . . . . . . .  5
     4.1.  Traffic patterns . . . . . . . . . . . . . . . . . . . . .  5
     4.2.  CAPEX minimization . . . . . . . . . . . . . . . . . . . .  6
     4.3.  OPEX minimization  . . . . . . . . . . . . . . . . . . . .  6
     4.4.  Traffic Engineering  . . . . . . . . . . . . . . . . . . .  7
   5.  Requirement List . . . . . . . . . . . . . . . . . . . . . . .  7
   6.  Network topology . . . . . . . . . . . . . . . . . . . . . . .  7
     6.1.  Clos topology overview . . . . . . . . . . . . . . . . . .  8
     6.2.  Clos topology properties . . . . . . . . . . . . . . . . .  8
     6.3.  Scaling Clos topology  . . . . . . . . . . . . . . . . . .  9
   7.  Routing design . . . . . . . . . . . . . . . . . . . . . . . . 10
     7.1.  Choosing the routing protocol  . . . . . . . . . . . . . . 10
     7.2.  BGP configuration for Clos topology  . . . . . . . . . . . 11
       7.2.1.  BGP Autonomous System numbering layout . . . . . . . . 11
       7.2.2.  Non-unique private BGP ASN's . . . . . . . . . . . . . 12
       7.2.3.  Prefix advertisement . . . . . . . . . . . . . . . . . 13
       7.2.4.  External connectivity  . . . . . . . . . . . . . . . . 13
     7.3.  ECMP Considerations  . . . . . . . . . . . . . . . . . . . 14
       7.3.1.  Basic ECMP . . . . . . . . . . . . . . . . . . . . . . 14
       7.3.2.  BGP ECMP over multiple ASN . . . . . . . . . . . . . . 15
     7.4.  BGP convergence properties . . . . . . . . . . . . . . . . 16
       7.4.1.  Convergence timing . . . . . . . . . . . . . . . . . . 16
       7.4.2.  Failure impact scope . . . . . . . . . . . . . . . . . 16
       7.4.3.  Third-party route injection  . . . . . . . . . . . . . 17
   8.  Security Considerations  . . . . . . . . . . . . . . . . . . . 17
   9.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 17
Show full document text