Internet Engineering Task Force                             A. Ford, Ed.
Internet-Draft                                       Roke Manor Research
Intended status: Experimental                                  C. Raiciu
Expires: April 22, 2010                        University College London
                                                                S. Barre
                                                Universite catholique de
                                                              J. Iyengar
                                           Franklin and Marshall College
                                                                 B. Ford
                                       Max Planck Institute for Software
                                                        October 19, 2009

         Architectural Guidelines for Multipath TCP Development

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at

   The list of Internet-Draft Shadow Directories can be accessed at

   This Internet-Draft will expire on April 22, 2010.

Copyright Notice

   Copyright (c) 2009 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents in effect on the date of

Ford, et al.             Expires April 22, 2010                 [Page 1]

Internet-Draft             MPTCP Architecture               October 2009

   publication of this document (
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.


   Often endpoints are connected by multiple paths, but the nature of
   TCP/IP restricts communications to a single path per socket.
   Resource usage within the network would be more efficient were these
   multiple paths able to be used concurrently.  This should enhance
   user experience through higher throughput and improved resilience to
   network failure.

   This document outlines architectural guidelines for the development
   of a Multipath Transport Protocol, with references to how these
   architectural components come together in the Multipath TCP (MPTCP)
   protocol.  This document also contains suggestions for functional
   separation within an implementation, maximising the flexibility that
   can be achieved with these architectural components.

Ford, et al.             Expires April 22, 2010                 [Page 2]

Internet-Draft             MPTCP Architecture               October 2009

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1.  Goals  . . . . . . . . . . . . . . . . . . . . . . . . . .  4
       1.1.1.  Functional Goals . . . . . . . . . . . . . . . . . . .  4
       1.1.2.  Performance/Efficiency Goals . . . . . . . . . . . . .  5
     1.2.  Motivation . . . . . . . . . . . . . . . . . . . . . . . .  5
     1.3.  Requirements Language  . . . . . . . . . . . . . . . . . .  6
   2.  Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . .  6
     2.1.  Terminology  . . . . . . . . . . . . . . . . . . . . . . .  7
     2.2.  Reference Scenario . . . . . . . . . . . . . . . . . . . .  7
     2.3.  Layered Representation . . . . . . . . . . . . . . . . . .  7
   3.  Multipath Architecture . . . . . . . . . . . . . . . . . . . .  8
     3.1.  Motivations  . . . . . . . . . . . . . . . . . . . . . . .  8
     3.2.  Decomposing Transport Functions  . . . . . . . . . . . . .  8
     3.3.  TCP Performance  . . . . . . . . . . . . . . . . . . . . . 10
   4.  Implementation Architecture  . . . . . . . . . . . . . . . . . 11
     4.1.  Functional Separation  . . . . . . . . . . . . . . . . . . 11
       4.1.1.  Application to default MPTCP protocol  . . . . . . . . 11
       4.1.2.  Generic architecture for MPCTP . . . . . . . . . . . . 14
     4.2.  PM/MPS interface . . . . . . . . . . . . . . . . . . . . . 15
   5.  Security Considerations  . . . . . . . . . . . . . . . . . . . 16
   6.  Interactions with Applications . . . . . . . . . . . . . . . . 16
   7.  Interactions with Middleboxes  . . . . . . . . . . . . . . . . 17
   8.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 17
   9.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 17
   10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 17
     10.1. Normative References . . . . . . . . . . . . . . . . . . . 17
     10.2. Informative References . . . . . . . . . . . . . . . . . . 17
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 18

Ford, et al.             Expires April 22, 2010                 [Page 3]

Internet-Draft             MPTCP Architecture               October 2009

1.  Introduction

   Multipath TCP (MPTCP) will provide a set of extensions for regular
   TCP [2] to allow one TCP connection to be spread across multiple
   paths.  This section describes the motivation behind the design of
   Multipath TCP.

   Companion documents to this architectural overview are those which
   provide details of the protocol extensions, congestion control
   algorithms, and application-level considerations.  Put together,
   these components build a complete Multipath TCP implementation.
   Other components, however, could be introduced in place of these, in
   accordance with the architecture specified in this document.

   Please note this document is a work-in-progress and covers several
   topics, some of which may be more appropriately moved to separate
   documents as this work evolves.

1.1.  Goals

   This section outlines what we perceive to be the most important goals
   for a multipath-capable transport protocol.  We divide these goals
   into two categories: functional goals and performance/efficiency

1.1.1.  Functional Goals

   o  Multihoming: The multipath transport protocol must allow for a
      logical transport endpoint as seen by the application to
      correspond to multiple physical network attachment points, such as
      multiple IP addresses on the same or different network interfaces.

   o  Application Compatibility: Multipath-capable equivalents of
      existing transports such as concurrent multipath versions of TCP,
      SCTP, or DCCP, must retain backward compatibility with existing
      APIs, so that existing applications can use the newer transports,
      merely by upgrading the operating systems of the end-hosts.

   o  Network Compatibility: Multipath transport protocols must remain
      backward compatible with the Internet as it exists today,
      including being able to traverse predominant existing middleboxes
      such as firewalls, NATs, and performance enhancing proxies [3].

   o  Automatic Negotiation: A host supporting a multipath-capable
      equivalent of an existing transport must be able to detect
      reliably whether a new communication partner supports the next-
      generation protocol, using it if so, and otherwise automatically
      falling back to the existing protocol (e.g., standard TCP, SCTP,

Ford, et al.             Expires April 22, 2010                 [Page 4]

Internet-Draft             MPTCP Architecture               October 2009

      or DCCP).

   o  End-to-End Reliability and Security: The multipath-capable
      equivalent of an existing transport must retain its end-to-end
      reliability properties and allow for end-to-end authentication
      and/or privacy protection in a network-compatible fashion, i.e.,
      maintain compatibility with legacy middleboxes.

1.1.2.  Performance/Efficiency Goals

   o  Multihoming and Multipath Capable: The protocol mustbe able to
      detect and utilize multiple available paths between two logical
      endpoints, either one path at a time (fail-over multipath) or
      several at once (concurrent multipath).

   o  Resource Pooling: Transports should be able to balance traffic
      among available paths, optimizing network utility in a global
      sense by shifting load away from congested bottlenecks and taking
      advantage of spare capacity wherever it may be located [4].

   o  TCP-Friendliness: The architecture must enable new multipath
      transport flows to coexist gracefully with predominant existing
      transport flows, competing for bandwidth neither unduly
      aggressively or unduly timidly (unless low-precedence operation is
      specifically requested by the application, such as with LEDBAT).

   o  Congestion State Sharing: Since popular applications such as HTTP
      often use multiple transport instances between the same pair of
      hosts, the protocol must avoid multiplicative explosions in
      multipath congestion control contexts - i.e., N transport
      instances times M multipath flows each - by enabling a multipath
      "bundle" of congestion control contexts to be shared cleanly among
      application-visible transport instances.

   o  Support Small Transactions: Recognizing that many applications
      today make heavy use of frequent small communications, such as
      HTTP conditional GET transactions or streaming media frames, next-
      generation transports should minimize the performance costs of
      supporting these common application behaviors, including by
      minimizing unnecessary protocol overhead on small packets and by
      unnecessary round-trip delays or state maintenance costs when
      applications use short transactions.

1.2.  Motivation

   As the Internet evolves, demands on Internet resources are ever-
   increasing, but often these resources (in particular, bandwidth)
   cannot be fully utilised due to protocol constraints on both the end-

Ford, et al.             Expires April 22, 2010                 [Page 5]

Internet-Draft             MPTCP Architecture               October 2009

   systems and within the network.  These unused resources, if they
   could be used, would lead to an enhanced user experience.  This would
   also reduce the necessary expenditure on network infrastructure which
   would otherwise be needed to create an equivalent experience

   By the application of resource pooling [4], these available resources
   can be 'pooled' such that they appear as a single logical resource to
   the user.  The purpose of Multipath TCP, therefore, is to provide a
   TCP to the user that is able to make use of multiple available paths.

   The achievement of resource pooling through Multipath TCP bring two
   key benefits:

   o  To increase the resilience of the connectivity by providing
      multiple paths, protecting end hosts from the failure of one.

   o  To increase the efficiency of the resource usage, and thus
      increase the network capacity available to end hosts.

   A Multipath TCP MUST follow the same service model as TCP [2]: byte
   oriented, in order reliable delivery.  To have a deployable protocol,
   MPTCP SHOULD adhere to the following "do no harm" philosophy:
   multipath TCP SHOULD behave no worse (throughput wise) than running a
   single TCP connection over any of its paths, and using multiple paths
   MUST not harm users using single path TCP at shared bottlenecks.  In
   addition, it should aim to be backwards-compatible where possible
   with existing, regular TCP.

   Multipath TCP as presented in [5] follows these aims, and achieves
   resource pooling by combining multiple TCP sessions running over
   multiple paths, and presenting them as a single TCP connection to the
   application.  This is not the only way of creating a Multipath TCP,
   however, and as such this architecture is designed so that other
   components can be used in place

1.3.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   document are to be interpreted as described in RFC 2119 [1].

2.  Fundamentals

Ford, et al.             Expires April 22, 2010                 [Page 6]

Internet-Draft             MPTCP Architecture               October 2009

2.1.  Terminology

   Path:  A sequence of links between a sender and a receiver, defined
      in this context by a source and destination address pair.

   Endpoint:  A host operating an MPTCP implementation, and either
      initiating or terminating a MPTCP connection.

   Subflow:  A flow of TCP packets operating over an individual path,
      which forms part of a larger MPTCP connection.

   Multipath TCP (MPTCP):  A modified version of the TCP [2] protocol
      that supports the simultaneous use of multiple paths between

2.2.  Reference Scenario


   Endpoints, routes.  Addresses/path selection mechanisms?

2.3.  Layered Representation

   A Multipath TCP operates at the transport layer, and its existence
   should be transparent to both higher and lower layers.  It is a set
   of additional features on top of standard TCP, and as such the impact
   on applications should be minimal, or entirely transparent, and these
   application considerations are discussed in detail in [6].
   Multipath-aware applications would be able to use an extended sockets
   API to have further influence on the behaviour of MPTCP, which is
   also specified in [6].  Figure 1 illustrates this architecture.

                                   |           Application         |
      +---------------+            +-------------------------------+
      |  Application  |            |             MPTCP             |
      +---------------+            + - - - - - - - + - - - - - - - +
      |      TCP      |            | Subflow (TCP) | Subflow (TCP) |
      +---------------+            +-------------------------------+
      |      IP       |            |       IP      |      IP       |
      +---------------+            +-------------------------------+

      Figure 1: Comparison of Standard TCP and MPTCP Protocol Stacks

   This layered model could apply to any multipath transport protocol,
   with any multipath, transport and network layers.  This is explored
   in more detail in Section 3.2.

Ford, et al.             Expires April 22, 2010                 [Page 7]

Internet-Draft             MPTCP Architecture               October 2009

   This document now proceeds with a detailed discussion of an
   architecture for developing a multipath TCP implementation,
   especially regarding the functional separation by which different
   components should be developed, in Section 3.

3.  Multipath Architecture

   This section describes the layered functional separation that drives
   the design of the MPTCP protocol.  Its main goal is to separate MPTCP
   in two parts that communicate through a well defined interface.  We
   first provide the motivations for this functional separation, then we
   describe in more details the two main components of the MPTCP

3.1.  Motivations

   The major goal behind MPTCP is to send data over different paths in
   the same time.  This assumes that an MPTCP implementation must be
   able to discover and use the multiple paths that connect two given
   hosts, when they exist.  However, different mechanisms can be
   envisioned for multipath discovery and use.  Examples are as follows:

   Use multiple addresses:  This is the method currently proposed in the
      MPTCP protocol design [5] - if hosts are multi-addressed,
      different address pairs may take different routes.

   Use a path selector value:  An end-host might be able to tag packets
      with a path selector value, or adjust existing packet metadata.
      If some network nodes are able to read this tag and use it as a
      path selector, the host can influence the outgoing path of the

   Next-hop selection:  In a network configuration where multiple next-
      hops can offer to forward packets, a host may decide to send some
      of its packets through one next-hop, and some through another.

   The above list is not exhaustive, and could grow as new network
   technologies are deployed.

3.2.  Decomposing Transport Functions

   As shown in Figure 2, we first loosely separate functions within
   transports into "application-oriented" and "network-oriented" parts.
   We use this separation of functions as an architectural framework
   that a multipath transport must recognize, primarily to maintain
   backward compatibility with applications and with the network.  The
   desire for network compatibility will impact design choices at the

Ford, et al.             Expires April 22, 2010                 [Page 8]

Internet-Draft             MPTCP Architecture               October 2009

   subflow level, while the need for application compatibility will
   primarily impact design choices at the higher, application-facing
   MPTCP layer.

   The top application-oriented "Semantic" functions are whatever
   communication abstractions are to be made available to applications,
   including providing the end-to-end reliability and ordering
   properties of abstractions like TCP's byte streams or SCTP's message-
   based multi-streams; these functions essentially deal with concerns
   of application-visible semantics.

   We consider the bottom part "network-oriented" because they represent
   functions that, while traditionally located in the ostensibly "end-
   to-end" Transport Layer, have proven in practice to be of great
   concern to network operators and the middleboxes they deploy in the
   network to enforce network usage policies [7][8] or optimize
   communication performance [9].  The network-oriented functions
   include congestion control and other performance-management functions
   ("Flow" performance functions), and endpoint/service identification
   functions (e.g., port numbers) that network operators and their
   middleboxes require to enforce network access and security policies
   ("Endpoint" functions).  These network-oriented transport functions
   are collectively labeled in figure Figure 2 as "Flow/Endpoint"

                               |   Application   |
    +---------------+     ---> +-----------------+
    |   Application |    /     |     Semantic    | (Application-Oriented
    +---------------+ <--      |     Functions   |      Functions)
    |   Transport   |          |-   -  -  -  -  -|
    +---------------+ <--      | Flow / Endpoint | (Network-Oriented
    |    Network    |    \     |   Functions     |     Functions)
    +---------------+     ---> +-----------------+
                               |   Network       |

              Figure 2: Decomposition of Transport Functions

   Following from the discussion above, a multipath transport would have
   to manage Flow/Endpoint functions for every path in an end-to-end
   connection, while providing a transparent single interface to the
   application.  In keeping with this architectural worldview, MPTCP
   divides the Transport Layer into two components: the MPTCP part,
   which is responsible for the Semantic functions of global ordering of
   application data and reliability; and the "legacy TCP" part, which

Ford, et al.             Expires April 22, 2010                 [Page 9]

Internet-Draft             MPTCP Architecture               October 2009

   implements the Flow/Endpoint functions.  The figure below shows how
   MPTCP implements this architecture:

        +--------------------------+    +-------------------------+
        |      Application         |    |      Application        |
        +--------------------------+    +-------------------------+
        |        Semantic          |    |         MPTCP           |
        |-  -  -  -  -  -  -  -  - |    + - - - - -  +  - - - - - +
        | Flow/Endpt |  Flow/Endpt |    |    TCP     |     TCP    |
        +--------------------------+    +-------------------------+
        |   Network  |   Network   |    |     IP     |     IP     |
        +--------------------------+    +-------------------------+

             Figure 3: Mapping Transport Architecture to MPTCP

3.3.  TCP Performance

   To provide multipath transport, Multipath TCP must send data over
   multiple paths.  A naive implementation could simply run a standard,
   unmodified TCP congestion control algorithm [10] on each subflow.  As
   listed in Section 1.1.1, however, it is a goal that MPTCP does not
   cause harm to other TCP flows, and such a naive approach would lead
   to a multipath session taking a disproportionate amount of bandwidth
   at shared bottlenecks.

   Therefore, in addition to purely sending data over multiple paths,
   MPTCP must do it in a way that will not affect TCP performance of
   non-multipath users.  This raises the need for an efficient multipath
   congestion control algorithm.  While this specification does not
   mandate the use of any particular algorithm for congestion control,
   it ensures that the protocol is designed in such a way that any
   congestion control algorithm can be designed, independently of the
   other components in use in the MPTCP implementation.  Consequently,
   our architecture for MPTCP decouples the policy from the mechanism.
   The policy is the decision of what path to use for each packet to
   send.  It is mainly driven by the implementation-dependent congestion
   control algorithm.  The mechanism is the technology used to ensure
   that a packet will be sent on the desired path.  This separation is
   intended to be relatively future-proof by allowing these components
   to evolve at different speeds.

   The decomposition of the transport functions, as described in
   Section 3.2, places the congestion control functionality in the flow/
   endpoint layer, which maps to an individual subflow in the MPTCP
   design.  In order to meet the "do-no-harm" philosophy, however, there
   must be an interface to the congestion control implementation that
   permits the appropriate coupling of congestion windows at the upper

Ford, et al.             Expires April 22, 2010                [Page 10]

Internet-Draft             MPTCP Architecture               October 2009

   (MPTCP) layer.

4.  Implementation Architecture

   This section provides suggestions for an architecture to implement an
   extensible, modular multipath transport protocol.

4.1.  Functional Separation

   This section describes a generic view of the internal implementation
   of a Multipath TCP, through which the technical components specified
   in the companion documents can fit together.  It shows how an
   implementation could be built that permits extensibility between
   components without changing the external representation.

   We first show the functional decomposition of an MPTCP solution that
   is completely contained in the transport layer.  That solution is
   described in more details in [5].  Then we generalize the approach to
   allow good extensibility of that solution.

4.1.1.  Application to default MPTCP protocol

   Although, in the default approach, MPTCP is fully contained in the
   transport layer, it can still be divided into two main modules.  One
   manages the scheduling of packets as well as congestion control.  The
   other one manages the control of paths.  The interface between the
   two is dealt with thanks to a Path Index.  As shown in Figure 4, the
   Path Manager announces to the MultiPath Scheduler what paths can be
   used trough path indices, and maintains the mapping between that
   value and the particular action that it must apply to use the path
   (an example of such a mapping is in Table 1).  In the case of the
   built-in Path Manager, the action is to replace an address/port pair
   with another one, in such a way that another path is used across the
   Internet to forward that packet.

Ford, et al.             Expires April 22, 2010                [Page 11]

Internet-Draft             MPTCP Architecture               October 2009

            Control plane    <--     |     -->    Data plane
   |                     Multipath Scheduler (MPS)                 |
                ^                    |          |
                |                    |   [A1,B1,|pA1,pB1]
                |For conn_id         |          |
                |<A1,B1,pA1,pB1>     |   +-------------+
                |Paths 1->4 can be   |   | Data packet |<--Path idx:3
                |used.               |   +-------------+   attached
                |                    |          |          by MPS
                |                    |          V
   |                         Path Manager (PM)   \[A1,B1]->[A1,B2] |
      /                           \  |                 \
     /-----------------------------\ |   /"\    /"\    /"\   /"\
     | rewriting table:             ||   | |    | |    | |   | |
     | Subflow id  <-->  network_id ||   | |    | |    | |   | |
     |                              ||   | |    | |    | |   | |
     |    [see table below]         ||   | |    | |    | |   | |
     |                              ||   \./    \./    \./   \./
     +------------------------------+|  path1  path2  path3 path4

      Figure 4: Functional separation of MPTCP in the transport layer

   The MultiPath Scheduler only deals with abstract paths, represented
   by numbers.  It only sees one address pair throughout the
   communication, that we call the connection identifier.  However, the
   MultiPath Scheduler must be able to perform per-subflow congestion
   control, and thus to distinguish between the subflows.  This leads to
   define a subflow identifier, that consists of the usual transport
   identifier extended with the path index:
   <addr_src,psrc,addr_dst,pdst,path_index>.  The following options,
   described in [5], are managed by the MultiPath Scheduler.

   o  MULTIPATH CAPABLE (MPC): Tell the peer that we support MPTCP.
      Note that the MPC option also holds a token, which is necessary
      only if the built-in Path Manager is used.  In the next section we
      describe the generalized case, where the token can be ignored by
      the receiver if another path manager is used.

   o  DATA SEQUENCE NUMBER (DSN): Identify the position of a set of
      bytes in the meta-flow.

   o  DATA FIN (DFIN): Terminate a meta-flow.

Ford, et al.             Expires April 22, 2010                [Page 12]

Internet-Draft             MPTCP Architecture               October 2009

   An implementation MUST use those options even if another Path Manager
   than the default one is implemented.

   The Path manager applies a particular technology to give the MPS the
   possibility to use several paths.  The built-in MPTCP Path Manager
   uses multiple IPv4 addresses as its mean to influence the forwarding
   of packets through the Internet.

   When the MPS starts a new connection, the PM chooses a token that
   will be used to identify the connection.  This is necessary to allow
   the PM applying the correct path index to incoming packets.  An
   example mapping table is given hereafter:

    |  connection id  |    subflow id    |  token  |    Network id   |
    | <A1,B1,pA1,pB1> |   <conn_id,pi1>  | token_1 | <A1,B1,pA1,pB1> |
    | <A1,B1,pA1,pB1> |   <conn_id,pi2>  | token_1 | <A2,B2,pA1,pB2> |
    | <A1,B1,pA1,pB1> |   <conn_id,pi3>  | token_1 | <A1,B2,pA1,pB2> |
    | <A1,B1,pA1,pB1> |   <conn_id,pi4>  | token_1 | <A2,B1,pA1,pB1> |
    | <A1,B1,pA1,pB3> | <connect_id,pi1> | token_2 | <A1,B1,pA1,pB3> |
    | <A1,B1,pA1,pB3> | <connect_id,pi2> | token_2 | <A2,B1,pA1,pB3> |

              Table 1: Example mapping table for built-in PM

   Table 1 shows an example where two connections are ongoing.  One is
   identified by token_1, the other one with token_2.  Since addresses
   are rewritten by the path manager, the attachment to the right
   connection is achieved thanks to the token, which is used at
   connection establishment and subflow establishment.  It is then
   remembered.  The first column holds the information that is exposed
   to the applications, while the last column shows the information that
   is actually written in packets that will fly through the network.  We
   note that additionnally to the addresses, ports can be rewritten,
   which contributes to supporting NATs.  The table also shows the role
   of the token, which is to attach various combinations of ports and
   addresses to a single connection.  The token is specific to the
   built-in path manager, and can be ignored if another path manager is
   used.  An implementation of the built-in path manager MUST implement
   the following options (defined in more details in [5]):

   o  Add Address (ADDR): Announce a new address we own

   o  Remove Addresse (REMADDR): Withdraw a previously announced address

   o  Join Connection (JOIN): Attach a new subflow to the current

Ford, et al.             Expires April 22, 2010                [Page 13]

Internet-Draft             MPTCP Architecture               October 2009

   Those options form the default MPTCP Path Manager, based on declaring
   IP addresses, and carries control information in TCP options.  An
   implementation of Multipath TCP can use any Path Manager, but it MUST
   be able to fallback to the default PM in case the other end does not
   support the custom PM.  Alternative Path Managers may be specified in
   separate documents in the future.

4.1.2.  Generic architecture for MPCTP

   Now that the functional decomposition has been shown for MPTCP with
   the built-in Path Manager, we show how that architecture can be
   generalized to allow the implementation of other Path Managers for
   MPTCP.  A general overview of the architecture is provided in
   Figure 5.  The Multipath Scheduler (MPS) learns about the number of
   available paths through notifications received from the Path Manager
   (PM).  From the point of view of the Multipath Scheduler, a path is
   just a number, called a Path Index.  Notifications from the PM to the
   MPS MAY contain supporting information about the paths, if relevant,
   so that the MPS can make more intelligent decisions about where to
   route traffic.  When the Multipath Scheduler initiates a
   communication to a new host, it can only send the packets to the
   default path.  But since the Path manager is layered below the MPS,
   it can detect that a new communication is happening, and tell the MPS
   about the other paths it knows about.

Ford, et al.             Expires April 22, 2010                [Page 14]

Internet-Draft             MPTCP Architecture               October 2009

            Control plane    <--     |     -->    Data plane
   |                     Multipath Scheduler (MPS)                 |
                ^                    |          |
                |                    |   [A1,B1,|pA1,pB1]
                |                    |          |
                |Announcing new      |   +-------------+
                |paths. (referred    |   | Data packet |<--Path idx:3
                |to as path indices) |   +-------------+   attached
                |                    |          |          by MPS
                |                    |          V
   |                         Path Manager (PM)   \__________zzzzz  |
      /                         \    |                       \
     /---------------------------\   |   /"\       /"\       /"\
     | subflow_id        Action  |   |   | |       | |       | |
     |<A1,B1,pA1,pB1,1>  xxxxx   |   |   | |       | |       | |
     |<A1,B1,pA1,pB1,2>  yyyyy   |   |   \./       \./       \./
     |<A1,B1,pA1,pB1,3>  zzzzz   |   |  path1     path2     path3

                 Figure 5: Overview of MPTCP architecture

   From then on, it is possible for the MPS to attach a Path Index to
   the control structure of its packets (internal to the MPTCP
   implementation), so that the Path Manager can map this Path Index to
   the corresponding action. (see table in the lower left part of
   Figure 5).  The particular action depends on the network mechanism
   used to select a path.  Examples are address rewriting, tunnelling or
   setting a path selector value inside the packet.

   The applicability of the architecture is not limited to the MPTCP
   protocol.  While we define in this document an MPTCP MPS (MPTCP
   Multipath Scheduler), other Multipath Schedulers can be defined.  For
   example, if an appropriate socket interface is designed, applications
   could behave as a Multipath Scheduler and decide where to send any
   particular data.  In this document we concentrate on the MPTCP case,

4.2.  PM/MPS interface

   The minimal set of requirement for a Path Manager is as follows:

   o  Outgoing untagged packets: Any outgoing packet flowing through the
      Path Manager is either tagged or untagged (by the MPS) with a path
      index.  If it is untagged, the packet is sent normally to the

Ford, et al.             Expires April 22, 2010                [Page 15]

Internet-Draft             MPTCP Architecture               October 2009

      Internet, as if no multi-path support were present.  Untagged
      packets can be used to trigger a path discovery procedure, that
      is, a Path Manager can listen to untagged packets and decide at
      some time to find if any other path than the default one is
      useable for the corresponding host pair.  Note that any other
      criteria could be used to decide when to start discovering
      available paths.  Note also that MPS scheduling will not be
      possible until the Path Manager has notified the available paths.
      The PM is thus the first entity coming into action.

   o  Outgoing tagged packets: The Path Manager maintains a table
      mapping path indices to actions.  The action is the operation that
      allows using a particular path.  Examples of possible actions are
      route selection, interface selection or packet transformation.
      When the PM sees a packet tagged with a path index, it looks up
      its table to find the appropriate action for that packet.  The tag
      is purely local.  It is removed before the packet is transmitted.

   o  Incoming packets: A Path Manager MUST ensure that each incoming
      path is mapped unambiguously to exactly one outgoing path.  Note
      that this requirement implies that the same number of incoming/
      outgoing paths must be established.  Moreover, a PM MUST tag any
      incoming path with the same Path Index as the one used for the
      corresponding outgoing path.  This is necessary for MPTCP to know
      what outgoing path is acknowledged by an incoming packet.

   o  Module interface: A PM MUST be able to notify the MPS about the
      number of available paths.  Such notifications MUST contain the
      path indices that are legal for use by the MPS.  In case the PM
      decides to stop providing service for one path, it MUST notify the
      MPS about path removal.  Additionnaly, a PM MAY provide
      complementary path information when available, such as link
      quality or preference level.

5.  Security Considerations

   Please see [11] for a threat analysis of Multipath TCP.

6.  Interactions with Applications

   Interactions with applications - incuding, but not limited to
   performances changes that may be expected, semantic changes, and new
   features that may be requested of an API, are presented in [6].

Ford, et al.             Expires April 22, 2010                [Page 16]

Internet-Draft             MPTCP Architecture               October 2009

7.  Interactions with Middleboxes


   List of issues that may arise with NATs, firewalls, proxies, etc?

   This will be an overview only, and protocol-specific solutions to
   this will be given in the companion docments.

8.  Acknowledgements

   Alan Ford, Costin Raiciu and Sebastien Barre are supported by Trilogy
   (, a research project (ICT-216372)
   partially funded by the European Community under its Seventh
   Framework Program.  The views expressed here are those of the
   author(s) only.  The European Commission is not liable for any use
   that may be made of the information in this document.

9.  IANA Considerations


10.  References

10.1.  Normative References

   [1]   Bradner, S., "Key words for use in RFCs to Indicate Requirement
         Levels", BCP 14, RFC 2119, March 1997.

10.2.  Informative References

   [2]   Postel, J., "Transmission Control Protocol", STD 7, RFC 793,
         September 1981.

   [3]   Carpenter, B. and S. Brim, "Middleboxes: Taxonomy and Issues",
         RFC 3234, February 2002.

   [4]   Wischik, D., Handley, M., and M. Bagnulo Braun, "The Resource
         Pooling Principle", ACM SIGCOMM CCR vol. 38 num. 5, pp. 47-52,
         October 2008,

   [5]   Ford, A., Raiciu, C., Handley, M., and S. Barre, "TCP
         Extensions for Multipath Operation with Multiple Addresses",
         draft-ford-mptcp-multiaddressed-01 (work in progress),

Ford, et al.             Expires April 22, 2010                [Page 17]

Internet-Draft             MPTCP Architecture               October 2009

         July 2009.

   [6]   Scharf, M. and A. Ford, "MPTCP Application Interface
         Considerations", draft-scharf-mptcp-api-00 (work in progress),
         October 2009.

   [7]   Srisuresh, P. and K. Egevang, "Traditional IP Network Address
         Translator (Traditional NAT)", RFC 3022, January 2001.

   [8]   Freed, N., "Behavior of and Requirements for Internet
         Firewalls", RFC 2979, October 2000.

   [9]   Border, J., Kojo, M., Griner, J., Montenegro, G., and Z.
         Shelby, "Performance Enhancing Proxies Intended to Mitigate
         Link-Related Degradations", RFC 3135, June 2001.

   [10]  Allman, M., Paxson, V., and W. Stevens, "TCP Congestion
         Control", RFC 2581, April 1999.

   [11]  Bagnulo, M., "Threat Analysis for Multi-addressed/Multi-path
         TCP", draft-bagnulo-mptcp-threat-00 (work in progress),
         October 2009.

Authors' Addresses

   Alan Ford (editor)
   Roke Manor Research
   Old Salisbury Lane
   Romsey, Hampshire  SO51 0ZN

   Phone: +44 1794 833 465

   Costin Raiciu
   University College London
   Gower Street
   London  WC1E 6BT


Ford, et al.             Expires April 22, 2010                [Page 18]

Internet-Draft             MPTCP Architecture               October 2009

   Sebastien Barre
   Universite catholique de Louvain
   Pl. Ste Barbe, 2
   Louvain-la-Neuve  1348

   Phone: +32 10 47 91 03

   Janardhan Iyengar
   Franklin and Marshall College
   Mathematics and Computer Science
   PO Box 3003
   Lancaster, PA  17604-3003

   Phone: 717-358-4774

   Bryan Ford
   Max Planck Institute for Software Systems


Ford, et al.             Expires April 22, 2010                [Page 19]