ALTO WG                                                     G. Bernstein
Internet-Draft                                         Grotto Networking
Intended status: Standards Track                                 S. Chen
Expires: September 6, 2018                             Tongji University
                                                                  K. Gao
                                                     Tsinghua University
                                                                  Y. Lee
                                                                  Huawei
                                                                W. Roome
                                                               M. Scharf
                                                                   Nokia
                                                                 Y. Yang
                                                         Yale University
                                                                J. Zhang
                                                       Tongji University
                                                           March 5, 2018


                 ALTO Extension: Path Vector Cost Type
                   draft-ietf-alto-path-vector-03.txt

Abstract

   The Application-Layer Traffic Optimization (ALTO) protocol [RFC7285]
   has defined several resources and services to provide clients with
   basic network information.  However, the base ALTO protocol and
   latest extensions only provide end-to-end metrics, which are
   insufficient to satisfy the demands of solving more complex network
   optimization problems.  This document introduces an extension to the
   base ALTO protocol, namely the path-vector extension, which allows
   ALTO clients to query information such as capacity regions for a
   given set of flows.  A non-normative example called multi-flow
   scheduling is presented to illustrate the limitations of existing
   ALTO (endpoint) cost maps.  After that, details of the extension are
   defined.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.





Bernstein, et al.       Expires September 6, 2018               [Page 1]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 6, 2018.

Copyright Notice

   Copyright (c) 2018 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   5
   3.  Use Case: Capacity Region for Multi-Flow Scheduling . . . . .   5
   4.  Overview of Path Vector Extensions  . . . . . . . . . . . . .   7
     4.1.  New Cost Type to Encode Path Vectors  . . . . . . . . . .   7
     4.2.  New Entity Domain to Provide ANE Properties . . . . . . .   8
     4.3.  New Service to Enable Multipart Resources . . . . . . . .   8
   5.  Path Vector Extension: Basic Data Types . . . . . . . . . . .   9
     5.1.  Cost Type . . . . . . . . . . . . . . . . . . . . . . . .   9
       5.1.1.  Cost Mode: array  . . . . . . . . . . . . . . . . . .   9
       5.1.2.  Cost Metric: ane-path . . . . . . . . . . . . . . . .   9
       5.1.3.  Path Vector Cost Type Semantics . . . . . . . . . . .   9
     5.2.  ANE Domain  . . . . . . . . . . . . . . . . . . . . . . .  10
       5.2.1.  Domain Name . . . . . . . . . . . . . . . . . . . . .  10
       5.2.2.  Domain-Specific Entity Addresses  . . . . . . . . . .  10
       5.2.3.  Hierarchy and Inheritance . . . . . . . . . . . . . .  11
     5.3.  Abstract Network Element Name . . . . . . . . . . . . . .  11
   6.  Path Vector Extension: Services . . . . . . . . . . . . . . .  11
     6.1.  Filtered Cost Map Extensions  . . . . . . . . . . . . . .  11



Bernstein, et al.       Expires September 6, 2018               [Page 2]


Internet-Draft         ALTO Extension: Path Vector            March 2018


       6.1.1.  Capabilities  . . . . . . . . . . . . . . . . . . . .  11
       6.1.2.  Accept Input Parameters . . . . . . . . . . . . . . .  12
       6.1.3.  Response  . . . . . . . . . . . . . . . . . . . . . .  12
     6.2.  Endpoint Cost Service Extensions  . . . . . . . . . . . .  12
       6.2.1.  Capabilities  . . . . . . . . . . . . . . . . . . . .  12
       6.2.2.  Accept Input Parameters . . . . . . . . . . . . . . .  12
       6.2.3.  Response  . . . . . . . . . . . . . . . . . . . . . .  13
     6.3.  Multipart Cost Property Service . . . . . . . . . . . . .  13
       6.3.1.  Media Type  . . . . . . . . . . . . . . . . . . . . .  13
       6.3.2.  HTTP Method . . . . . . . . . . . . . . . . . . . . .  13
       6.3.3.  Accept Input Parameters . . . . . . . . . . . . . . .  13
       6.3.4.  Capabilities  . . . . . . . . . . . . . . . . . . . .  14
       6.3.5.  Uses  . . . . . . . . . . . . . . . . . . . . . . . .  14
       6.3.6.  Response  . . . . . . . . . . . . . . . . . . . . . .  14
   7.  Examples  . . . . . . . . . . . . . . . . . . . . . . . . . .  14
     7.1.  Workflow  . . . . . . . . . . . . . . . . . . . . . . . .  15
     7.2.  Information Resource Directory Example  . . . . . . . . .  15
     7.3.  Example # 1 . . . . . . . . . . . . . . . . . . . . . . .  17
     7.4.  Example # 2 . . . . . . . . . . . . . . . . . . . . . . .  18
   8.  Compatibility . . . . . . . . . . . . . . . . . . . . . . . .  19
     8.1.  Compatibility with Legacy ALTO Clients/Servers  . . . . .  19
     8.2.  Compatibility with Multi-Cost Extensions  . . . . . . . .  20
     8.3.  Compatibility with Incremental Update . . . . . . . . . .  20
   9.  Design Decisions and Discussions  . . . . . . . . . . . . . .  20
     9.1.  Provide More General Calendar Extension . . . . . . . . .  20
   10. Security Considerations . . . . . . . . . . . . . . . . . . .  21
     10.1.  Privacy Concerns . . . . . . . . . . . . . . . . . . . .  21
     10.2.  Resource Consumption on ALTO Servers . . . . . . . . . .  21
   11. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  21
     11.1.  ALTO Cost Mode Registry  . . . . . . . . . . . . . . . .  21
     11.2.  ALTO Cost Metric Registry  . . . . . . . . . . . . . . .  22
     11.3.  ALTO Entity Domain Registry  . . . . . . . . . . . . . .  22
     11.4.  ALTO Network Element Property Type Registry  . . . . . .  22
   12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  22
   13. References  . . . . . . . . . . . . . . . . . . . . . . . . .  23
     13.1.  Normative References . . . . . . . . . . . . . . . . . .  23
     13.2.  Informative References . . . . . . . . . . . . . . . . .  23
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  24

1.  Introduction

   The base ALTO protocol [RFC7285] is designed to expose network
   information through services such as Cost Map and Endpoint Cost
   service.  These services use an extreme "single-node" network view
   abstraction, which represents the whole network with a single node
   and hosts with "endpoint groups" directly connected to the node.





Bernstein, et al.       Expires September 6, 2018               [Page 3]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   Although the "single-node" network view abstraction works well in
   many settings, it lacks the ability to support emerging use cases,
   such as inter-datacenter data transfers
   [I-D.lee-alto-app-net-info-exchange].  For these use cases,
   applications require a more powerful network view abstraction beyond
   the "single-node" abstraction to support application capabilities, in
   particular, the ability of multi-flow scheduling.

   To support capabilities like multi-flow scheduling,
   [I-D.yang-alto-topology] provides many candidate network view
   abstractions.  This document uses one of those abstractions called
   "path vector" abstraction.  The path vector abstraction use path
   vectors with abstract network elements to provide network graph view
   for applications.  Here, abstract network elements can be links,
   switches, middleboxes and their aggregations.  And a path vector
   presents a sequence of abstract network elements that end-to-end
   traffic goes through.  Each abstract network element can own several
   properties like "bandwidth" and "delay".  These information may help
   the application avoid network congestion, achieving better
   application performance.

   Providing path vector abstraction using ALTO introduces the following
   requirements:

   o  Encoding path vectors rather than scalar cost values in cost maps:
      Cost maps allow only scalar (numerical or ordinal) cost values,
      they cannot carry an array of abstract network elements as a cost.
      A new cost type is required to encode path vectors as costs in
      cost maps.

   o  Encoding the properties of abstract network elements: Unified
      property map can provide properties of endpoints and pids, but it
      cannot convey the properties of abstract network elements.  A new
      entity domain needs to be registered so that unified property map
      can encode the properties of abstract network elements.

   o  Encapsulating multiple map messages in a single session: Making
      multiple queries to get path vectors and the properties of
      abstract network elements introduce additional communication
      overhead.  A mechanism to provide multiple map messages in a
      single session is necessary.

   This document proposes the path vector extension which satisfies
   these additional requirements to the ALTO protocol.  Specifically, it
   encodes selected abstract network elements in an end-to-end path with
   a new cost type called "path-vector", and conveys the properties of
   abstract network elements using unified property map.




Bernstein, et al.       Expires September 6, 2018               [Page 4]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   The rest of this document is organized as follows.  Section 3 gives
   an example of multi-flow scheduling and illustrates the limitations
   of the base ALTO protocol in such a use case.  Section 4 gives an
   overview of the path vector extension.  Section 5 and Section 6
   define the formal extension.  Section 7 presents several examples.
   Section 8 and Section 9 discusses compatibility issues with other
   existing ALTO extensions and design decisions.  Section 10 and
   Section 11 review the security and IANA considerations.

2.  Terminology

   Besides the terms defined in [RFC7285], [RFC8189] and
   [I-D.ietf-alto-unified-props-new], this document also uses the
   following additional terms: Abstract Network Element, Abstract
   Network Element Name, Abstract Network Element Property, Abstract
   Network Element Property Map, Path Vector and Path-Vector.

   o  Abstract Network Element (ANE): An abstract network element is an
      abstraction of network components, it can be an aggregation of
      links, middle boxes, virtualized network function (VNF), or even a
      sub-network.  An abstract network element has two attributes: a
      name and a set of properties.

   o  Abstract Network Element Name (ANE Name): An abstract network
      element name is an identifier that uniquely identifies an abstract
      network element, as defined in Section 5.3.

   o  Abstract Network Element Property (ANE Property): An abstract
      network element property is a network-related property of an
      abstract network element.  It can be "bandwidth" for links and
      "delay" between two switches.

   o  Abstract Network Element Property Map (ANE Property Map): An
      abstract network element property map is a Filtered Property Map
      defined in [I-D.ietf-alto-unified-props-new] which supports the
      "ane" domain in its "domain-types" capability.

   o  Path Vector: A path vector is an array of ALTO Abstract Network
      Elements (ANEs).  It presents an abstract network path between
      entities such as PIDs or endpoints.  An ANE represents a selected
      part of an end-to-end path that the ALTO Server considers worth
      exposing.

3.  Use Case: Capacity Region for Multi-Flow Scheduling

   Assume that an application has control over a set of flows, some
   flows may go through shared links or switches and share a bottleneck.
   Existing cost maps can not reveal such information.



Bernstein, et al.       Expires September 6, 2018               [Page 5]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   Specifically, consider a network as shown in Figure 1.  The network
   has 7 switches (sw1 to sw7) forming a dumb-bell topology.  Switches
   sw1/sw3 provide access on one side, sw2/sw4 provide access on the
   other side, and sw5-sw7 form the backbone.  Endhosts eh1 to eh4 are
   connected to access switches sw1 to sw4 respectively.  Assume that
   the bandwidth of link eh1 -> sw1 and link sw1 -> sw5 are 150 Mbps,
   and the bandwidth of the rest links are 100 Mbps.

                                  +------+
                                  |      |
                                --+ sw6  +--
                              /   |      |  \
        PID1 +-----+         /    +------+   \          +-----+  PID2
        eh1__|     |_       /                 \     ____|     |__eh2
             | sw1 | \   +--|---+         +---|--+ /    | sw2 |
             +-----+  \  |      |         |      |/     +-----+
                       \_| sw5  +---------+ sw7  |
        PID3 +-----+   / |      |         |      |\     +-----+  PID4
        eh3__|     |__/  +------+         +------+ \____|     |__eh4
             | sw3 |                                    | sw4 |
             +-----+                                    +-----+

                      Figure 1: Raw Network Topology.

   The single-node ALTO topology abstraction of the network is shown in
   Figure 2.

                          +----------------------+
                 {eh1}    |                      |     {eh2}
                 PID1     |                      |     PID2
                   +------+                      +------+
                          |                      |
                          |                      |
                 {eh3}    |                      |     {eh4}
                 PID3     |                      |     PID4
                   +------+                      +------+
                          |                      |
                          +----------------------+

             Figure 2: Base Single-Node Topology Abstraction.

   Consider an application overlay (e.g., a large data analysis system)
   which wants to schedule the traffic among a set of end host source-
   destination pairs, say eh1 -> eh2 and eh1 -> eh4.  The application
   can request a cost map providing end-to-end available bandwidth,
   using 'availbw' as cost-metric and 'numerical' as cost-mode.





Bernstein, et al.       Expires September 6, 2018               [Page 6]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   The application will receive from ALTO server that the bandwidth of
   eh1 -> eh2 and eh1 -> eh4 are both 100 Mbps.  But this information is
   not enough.  Consider the following two cases:

   o  Case 1: If eh1 -> eh2 uses the path eh1 -> sw1 -> sw5 -> sw6 ->
      sw7 -> sw2 -> eh2 and eh1 -> eh4 uses path eh1 -> sw1 -> sw5 ->
      sw7 -> sw4 -> eh4, then the application will obtain 150 Mbps.

   o  Case 2: If eh1 -> eh2 uses the path eh1 -> sw1 -> sw5 -> sw7 ->
      sw2 -> eh2 and eh1 -> eh4 uses the path eh1 -> sw1 -> sw5 -> sw7
      -> sw4 -> eh4, then the application will obtain only 100 Mbps.

   To allow applications to distinguish the two aforementioned cases,
   the network needs to provide more details.  In particular:

   o  The network needs to expose more detailed routing information to
      show the shared bottlenecks.

   o  The network needs to provide the necessary abstraction to hide the
      real topology information while providing enough information to
      applications.

   The path-vector extension defined in this document meets all the
   requirements.

   See [I-D.bernstein-alto-topo] for a survey of use-cases where
   extended network topology information is needed.

4.  Overview of Path Vector Extensions

   This section presents an overview of approaches adopted by the path
   vector extension.  It assumes the readers are familiar with
   (Filtered) Cost Map and Endpoint Cost Service defined in [RFC7285]
   and their extensions defined in [RFC8189].  The path vector extension
   also requires the support of Filtered Property Map defined in
   [I-D.ietf-alto-unified-props-new].

   The path vector extension is composed of three building blocks: (1) a
   new cost type to encode path vectors; (2) a new entity domain for
   unified property extension [I-D.ietf-alto-unified-props-new] to
   encode properties of abstract network elements; and (3) a new service
   to provide path vector messages in a single session;

4.1.  New Cost Type to Encode Path Vectors

   Existing cost types defined in [RFC7285] only allow scalar cost
   values, they cannot be used to convey vector format information.
   This document defines a new cost mode to enable the cost value to



Bernstein, et al.       Expires September 6, 2018               [Page 7]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   carry an array of elements, and a new cost metric to pass ANE names
   as elements in the array.  Detailed information and specifications
   are given in Section 5.1.1 and Section 5.1.2.

4.2.  New Entity Domain to Provide ANE Properties

   Given the new cost type introduced by Section 4.1, Cost Map and
   Endpoint Cost Service can provide the ANE names along a flow path.
   However, only providing the ANE names without properties is not
   enough.  To detect shared bottlenecks, ALTO clients may expect
   information on specific ANE properties such as link capacity or
   delay.

   This document adopts the property map defined in
   [I-D.ietf-alto-unified-props-new] to encode the properties of
   abstract network elements.  A new domain "ane" is registered in the
   property map.  Each entity in the "ane" domain has an identifier of
   an ANE.  An ANE identifier is the ANE name used in the values of the
   "ane-path" metric defined in the present draft.  ANE properties are
   provided in information resources called "Property Map Resource" and
   "Filtered Property Map Resource".  The "Filtered Property Map"
   resource which supports the "ane" domain is used to encode the
   properties of ane entities, and it is called an ANE Property Map in
   this document.

4.3.  New Service to Enable Multipart Resources

   In the base ALTO protocol, ALTO servers use media types in the HTTP
   header to indicate the type of the response.  Typically one response
   only contains a single media type, such as "application/alto-
   costmap+json" or "application/alto-propmap+json".  This has limited
   the capability of ALTO servers to return multiple map messages in a
   single response.

   Thus, an ALTO client needs to make separate queries to get the
   information of related services.  This may cause a data
   synchronization problem between dependent ALTO services because when
   making the second query, the result for the first query may have
   already changed.  The same problem can happen to Network Map and Cost
   Map resources.  However, unlike Network Map and Cost Map which are
   considered more stable, Path Vectors and the dependent ANE Property
   Maps might change more frequently.

   Instead of introducing a new media type to encapsulate multiple types
   in a single response, this document adopts the "multipart/related"
   media type defined in [RFC2387].  In this way, a response can contain
   both the path vectors in a filtered cost map (or endpoint cost map)
   and the associated ANE Property Map. The media types of the cost map



Bernstein, et al.       Expires September 6, 2018               [Page 8]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   and the property map can still be retrieved from the response.  The
   interpretation of each media type in the "multipart/related" response
   is consistent with the base ALTO protocol.

5.  Path Vector Extension: Basic Data Types

   This section formally specifies a new cost type and a new entity
   domain.

5.1.  Cost Type

   This document extends the cost types defined in Section 6.1 of
   [RFC7285] by introducing a new cost mode "array" and a new cost
   metric "ane-path".  In the rest content, this document use "path-
   vector" to indicate the combination cost type of the cost mode
   "array" and the cost metric "ane-path".

5.1.1.  Cost Mode: array

   This document extends the CostMode defined in Section 10.5 of
   [RFC7285] with a new cost mode: "array".  This cost mode indicates
   that every cost value in a cost map represents an array rather than a
   simple value.  The values are arrays of JSONValue.  The specific type
   of each element in the array depends on the cost metric.

5.1.2.  Cost Metric: ane-path

   This document specifies a new cost metric: "ane-path".  This cost
   metric indicates that the cost value is a list of abstract network
   elements which the path from a source to a destination goes across.
   The values are arrays of ANE names which are defined in Section 5.3.

   The cost metric "ane-path" SHOULD NOT be used when the cost mode is
   not "array" unless it is explicitly specified by a future extension.
   If an ALTO client send queries with the cost metric "ane-path" and a
   non "array" cost mode, the ALTO server SHOULD return an error with
   the error code "E_INVALID_FIELD_VALUE"; If an ALTO server declares
   the support of a cost type with the cost metric "ane-path" and a non
   "array" cost mode, the ALTO client SHOULD assume such a cost type is
   invalid and ignore it.

5.1.3.  Path Vector Cost Type Semantics

   The new cost type follows the convention of the cost types in the
   base ALTO protocol.  Table 1 lists some of the current defined cost
   types and their semantics.





Bernstein, et al.       Expires September 6, 2018               [Page 9]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   +------------+--------------+---------------------------------------+
   | Cost Mode  | Cost Metric  | Semantics                             |
   +------------+--------------+---------------------------------------+
   | numerical  | routingcost  | a number representing the routing     |
   |            |              | cost                                  |
   | numerical  | hopcount     | a number representing the hop count   |
   | ordinal    | routingcost  | a ranking representing the routing    |
   |            |              | cost                                  |
   | ordinal    | hopcount     | a ranking representing the hop count  |
   | array      | ane-path     | a list representing the ane path      |
   +------------+--------------+---------------------------------------+

                  Table 1: Cost Types and Their Semantics

   The "routingcost" and "hopcount" can encoded in "numerical" or
   "ordinal", however, the cost metric "ane-path" can only be applied to
   the cost mode "array" defined in this document to convey path vector
   information.  The cost metric "ane-path" can not be used in
   "numerical" or "ordinal" unless it is defined in future extensions.
   If the ALTO server declares that it support cost type with cost
   metric being "ane-path" and cost mode not being "array", the ALTO
   client SHOULD ignore them.

5.2.  ANE Domain

   This document specifies a new entity domain in addition to the ones
   in [I-D.ietf-alto-unified-props-new].

5.2.1.  Domain Name

   ane

5.2.2.  Domain-Specific Entity Addresses

   The entity address of ane domain is encoded as a JSON string.  The
   string MUST be no more than 64 characters, and it MUST NOT contain
   characters other than US-ASCII alphanumeric characters
   (U+0030-U+0039, U+0041-U+005A, and U+0061-U+007A), the hyphen ("-",
   U+002D), the colon (":", U+003A), the at sign ("@", code point
   U+0040), the low line ("_", U+005F), or the "." separator (U+002E).
   The "." separator is reserved for future use and MUST NOT be used
   unless specifically indicated in this document, or an extension
   document.








Bernstein, et al.       Expires September 6, 2018              [Page 10]


Internet-Draft         ALTO Extension: Path Vector            March 2018


5.2.3.  Hierarchy and Inheritance

   There is no hierarchy or inheritance for properties associated with
   ANEs.

5.3.  Abstract Network Element Name

   An Abstract Network Element Name is encoded as an EntityAddr of the
   "ane" domain as defined in Section 3.4.2 of
   [I-D.ietf-alto-unified-props-new].

6.  Path Vector Extension: Services

   This section extends Filtered Cost Map Service and Endpoint Cost
   Service.  It also introduce a new service called "Multipart Cost
   Property Service".

6.1.  Filtered Cost Map Extensions

   This document extends the Filtered Cost Map defined in Section 4.1 of
   [RFC8189].

   The specifications for the "media type", "HTTP method" and "uses" are
   the same as defined in Section 4.1 of [RFC8189].

6.1.1.  Capabilities

   The FilteredCostMapCapabilities object is extended with a new member
   "property-map":

   object {
     [ResourceID property-map;]
   } PathVectorFilteredCostMapCapabilities : FilteredCostMapCapabilities

   property-map:  A resource ID defined in the same IRD pointing to an
      ANE Property Map as defined in Section 2.  This field MUST be
      present if the path vector cost type is present in the "cost-type-
      names" field.

   Other fields of the FilteredCostMapCapabilities object has the same
   format as defined in Section 4.1.1 of [RFC8189] with the following
   restriction:

   testable-cost-type-names:  The path vector cost type with "ane-path"
      as the cost metric and "array" as the cost mode MUST NOT be
      included in "testable-cost-type-names".





Bernstein, et al.       Expires September 6, 2018              [Page 11]


Internet-Draft         ALTO Extension: Path Vector            March 2018


6.1.2.  Accept Input Parameters

   The ReqFilteredCostMap uses the same format as defined in
   Section 4.1.2 of [RFC8189], with the following restrictions:

   constraints, or-constraints:  If the path vector cost type is
      included in either "cost-type" or "multi-cost-types", ALTO clients
      MUST NOT use it in "constraints" or "or-constraints".  Otherwise,
      the ALTO server MUST return an error with error code
      "E_INVALID_FIELD_VALUE".

   testable-cost-types:  The path vector cost type MUST NOT be included
      in the "testable-cost-types" field.  Otherwise, the ALTO server
      MUST return an error with error code "E_INVALID_FIELD_VALUE".

6.1.3.  Response

   If the ALTO client includes the cost type "path-vector" in the "cost-
   type" or "multi-cost-types" field of the input parameter, the
   response use the same format as defined in Section 4.1.3 of
   [RFC8189], but the corresponding cost value MUST be encoded as a
   JSONArray of AbstractNetworkElementName.

6.2.  Endpoint Cost Service Extensions

   This document extends the Endpoint Cost Service defined in
   Section 4.2 in [RFC8189].

   The specifications for "HTTP method", "media-type" and "uses" are the
   same as defined in Section 4.2 in [RFC8189].

6.2.1.  Capabilities

   The same as defined in Section 6.1.1.

6.2.2.  Accept Input Parameters

   The ReqEndpointCostMap uses the same format as defined in
   Section 4.2.2 of [RFC8189], with the following restrictions:

   cost-type, multi-cost-types:  ALTO clients MUST include the path
      vector cost type, e.g.  the one with "ane-path" as cost metric and
      "array" as cost mode, in either "cost-type" or "multi-cost-types"
      to activate the path vector extension.

   constraints, or-constraints:  If the path vector cost type is
      included in either "cost-type" or "multi-cost-types", ALTO clients
      MUST NOT use it in "constraints" or "or-constraints".  Otherwise,



Bernstein, et al.       Expires September 6, 2018              [Page 12]


Internet-Draft         ALTO Extension: Path Vector            March 2018


      the ALTO server MUST return an error with error code
      "E_INVALID_FIELD_VALUE".

   testable-cost-types:  The path vector cost type MUST NOT be included
      in the "testable-cost-types" field.  Otherwise, the ALTO server
      MUST return an error with error code "E_INVALID_FIELD_VALUE".

6.2.3.  Response

   If the ALTO client specifies the path vector cost type in the "cost-
   type" or "multi-cost-types" field of the input parameter, the
   response use the same format as defined in Section 4.2.3 of
   [RFC8189], but the corresponding cost value MUST be encoded as a
   JSONArray of AbstractNetworkElementName.

6.3.  Multipart Cost Property Service

   This document introduces a new ALTO service called "Multipart Cost
   Property Service", which provides the path vector information and the
   associated ANE property information in the same response.

6.3.1.  Media Type

   The media type of the Multipart Cost Property service is "multipart/
   related".

6.3.2.  HTTP Method

   The Multipart Cost Property service is requested using the HTTP POST
   method.

6.3.3.  Accept Input Parameters

   The input parameters of the Multipart Cost Property service MUST be
   encoded as a JSON object in the body of an HTTP POST request.  The
   media type of the request SHOULD be one of "application/alto-
   costmapfilter+json" and "application/alto-endpointcostparams+json".
   The format of the request body depends on the media type:

   o  If the media type of the request is "application/alto-
      costmapfilter+json", the request body MUST be the same type as
      defined by Section 6.1.2.

   o  If the media type of the request is "application/alto-
      endpointcostparams+json", the request body MUST be the same type
      as defined by Section 6.2.2.





Bernstein, et al.       Expires September 6, 2018              [Page 13]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   The path vector cost type MUST be the only cost type in the input
   parameter.

6.3.4.  Capabilities

   TBD

6.3.5.  Uses

   The "uses" attribute MUST be an array with at least one resource id.
   The first resource id MUST point to a Filtered Cost Map or an
   Endpoint Cost Service resource.  And the path vector cost type MUST
   be in its "cost-type" capability.  If there are more than one
   resource id in the "uses" attribute, the ALTO client SHOULD ignore
   any additional resource ids.

   According to Section 6.1.1, the "property-map" field MUST be present
   in the first resource.  So the ALTO client MUST infer that the
   Property Map pointed by the "property-map" field of the first
   resource is also a dependent resource.

6.3.6.  Response

   If an ALTO client sends a request of the media type "application/
   alto-costmapfilter+json" and accepts "multipart/related", the HTTP
   body of the response MUST consist of two parts with the media types
   "application/alto-costmap+json" and "application/alto-propmap+json"
   accordingly.  The part with media type "application/alto-
   costmap+json" MUST be the first part.  The content of the
   "application/alto-endpointcost+json" part has the same format as
   defined in Section 6.1.3.

   If an ALTO client sends a request of the media type "application/
   alto-endpointcostparams+json" and accepts "multipart/related", the
   HTTP body of the response MUST consist of two parts with the media
   types "application/alto-endpointcost+json" and "application/alto-
   propmap+json" accordingly.  The part with media type "application/
   alto-endpointcost+json" MUST be the first part.  The content of the
   "application/alto-endpointcost+json" part has the same format as
   defined in Section 6.2.3.

7.  Examples

   This section lists some examples of path vector queries and the
   corresponding responses.






Bernstein, et al.       Expires September 6, 2018              [Page 14]


Internet-Draft         ALTO Extension: Path Vector            March 2018


7.1.  Workflow

   This section gives a typical workflow of an ALTO client using the
   path-vector extension.

   1.  Send a GET request for the whole Information Resource Directory.

   2.  Look for the resource of the (Filtered) Cost Map/Endpoint Cost
       Service which contains the path vector cost type and get the
       resource ID of the dependent abstract network element property
       map.

   3.  Check whether the capabilities of the property map includes the
       desired "prop-types".

   4.  Send a path-vector request which accepts "multipart/related"
       media type following "application/alto-costmap+json" or
       "application/endpointcost+json".

7.2.  Information Resource Directory Example

   Here is an example of an Information Resource Directory.  In this
   example, filtered cost map "cost-map-pv" doesn't support the multi-
   cost extension but support the path-vector extension, "endpoint-
   multicost-map" supports both multi-cost extension and path-vector
   extension.  Filtered Property Map "propmap-delay-availbw" supports
   properties "availbw" and "delay", and "propmap-location" supports
   property "location".

     {
       "meta": {
         "cost-types": {
           "pv": {
             "cost-mode": "array",
             "cost-metric": "ane-path"
           },
           "num-routingcost": {
             "cost-mode": "numerical",
             "cost-metric": "routingcost"
           },
           "num-hopcount": {
             "cost-mode": "numerical",
             "cost-metric": "hopcount"
           }
         }
       },
       "resources": {
         "my-default-networkmap": {



Bernstein, et al.       Expires September 6, 2018              [Page 15]


Internet-Draft         ALTO Extension: Path Vector            March 2018


           "uri" : "http://alto.example.com/networkmap",
           "media-type" : "application/alto-networkmap+json"
         }
         "cost-map-pv" : {
           "uri": "http://alto.example.com/costmap/pv",
           "media-type": "application/alto-costmap+json",
           "accepts": "application/alto-costmapfilter+json",
           "capabilities": {
             "cost-type-names": [ "pv", "num-hopcount" ]
           },
           "property-map": "propmap-delay",
           "uses": [ "my-default-networkmap" ]
         },
         "endpoint-multicost-map" : {
           "uri": "http://alto.exmaple.com/endpointcostmap/multicost",
           "media-type": "application/alto-endpointcost+json",
           "accepts": "application/alto-endpointcostparams+json",
           "capabilities": {
             "cost-constraints": true,
             "cost-type-names": [ "pv", "num-routingcost" ],
             "max-cost-types": 2
           },
           "property-map": "propmap-availbw"
         },
         "propmap-availbw-delay" : {
           "uri": "http://alto.exmaple.com/propmap/availbw",
           "media-type": "application/alto-propmap+json",
           "accepts": "application/alto-propmapparams+json",
           "capabilities": {
             "domain-types": [ "ane" ],
             "prop-types": [ "availbw" ]
           }
         },
         "propmap-location" : {
           "uri": "http://alto.exmaple.com/propmap/delay",
           "media-type": "application/alto-propmap+json",
           "accepts": "application/alto-propmapparams+json",
           "capabilities": {
             "domain-types": [ "pid" ],
             "prop-types": [ "location" ]
           }
         }
       }
     }







Bernstein, et al.       Expires September 6, 2018              [Page 16]


Internet-Draft         ALTO Extension: Path Vector            March 2018


7.3.  Example # 1

   POST /costmap/pv HTTP/1.1
   Host: alto.example.com
   Accept: multipart/related, application/alto-costmap+json,
           application/alto-propmap+json, application/alto-error+json
   Content-Length: [TBD]
   Content-Type: application/alto-costmapfilter+json

   {
     "cost-type": {
       "cost-mode": "array",
       "cost-metric": "ane-path"
     },
     "pids": {
       "srcs": [ "PID1" ],
       "dsts": [ "PID2", "PID3" ]
     }
   }

   HTTP/1.1 200 OK
   Content-Length: [TBD]
   Content-Type: multipart/related; boundary=42

   --42
   Content-Type: application/alto-costmap+json

   {
     "meta": {
       "dependent-vtags": [
         {
           "resource-id": "default-network-map",
           "tag": "75ed013b3cb58f896e839582504f622838ce670f"
         }
       ],
       "cost-type": {
         "cost-mode": "array",
         "cost-metric": "ane-path"
       },
     },

     "cost-map": {
       "PID1": {
         "PID2": [ "ane:L001", "ane:L003" ],
         "PID3": [ "ane:L001", "ane:L004" ]
       }
     }
   }



Bernstein, et al.       Expires September 6, 2018              [Page 17]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   --42
   Content-Type: application/alto-propmap+json

   {
     "property-map": {
       "ane:L001": { "delay": 46},
       "ane:L003": { "delay": 50},
       "ane:L004": { "delay": 70}
     }
   }

   --42--

7.4.  Example # 2

POST /endpointcostmap/multicost HTTP/1.1
Host: alto.example.com
Accept: multipart/related, application/alto-endpointcost+json,
        application/alto-propmap+json, application/alto-error+json
Content-Length: [TBD]
Content-Type: application/alto-endpointcostparams+json

{
  "multi-cost-types": [
    {
      "cost-mode": "array",
      "cost-metric": "ane-path"
    },
    {
      "cost-mode": "numerical",
      "cost-metric": "routingcost"
    }
  ],
  "endpoints": {
    "srcs": [ "ipv4:192.0.2.2" ],
    "dsts": [ "ipv4:192.0.2.89",
              "ipv4:203.0.113.45",
              "ipv6:2001:db8::10" ]
  }
}

HTTP/1.1 200 OK
Content-Length: [TBD]
Content-Type: multipart/related; boundary=example-2

--example-2
Content-Type: application/alto-endpointcost+json




Bernstein, et al.       Expires September 6, 2018              [Page 18]


Internet-Draft         ALTO Extension: Path Vector            March 2018


{
  "meta": {
    "multi-cost-types": [
      {"cost-mode": "array", "cost-metric": "ane-path"},
      {"cost-mode": "numerical", "cost-metric": "routingcost"}
    ]
  },

  "endpoint-cost-map": {
    "ipv4:192.0.2.2": {
      "ipv4:192.0.2.89":   [[ "ane:L001", "ane:L003", "ane:L004" ], 77],
      "ipv4:203.0.113.45": [[ "ane:L001", "ane:L004", "ane:L005" ], 68],
      "ipv6:2001:db8::10": [[ "ane:L001", "ane:L005", "ane:L007" ], 98]
    }
  }
}

--example-2
Content-Type: application/alto-propmap+json

{
  "property-map": {
    "ane:L001": { "availbw": 50 },
    "ane:L003": { "availbw": 48 },
    "ane:L004": { "availbw": 55 },
    "ane:L005": { "availbw": 60 },
    "ane:L007": { "availbw": 35 }
  }
}

--example-2--

8.  Compatibility

8.1.  Compatibility with Legacy ALTO Clients/Servers

   The path vector extension on Filtered Cost Map and Endpoint Cost
   Service is backward compatible with the base ALTO protocol.  If the
   ALTO server provides path vector extended Filtered Cost Map or
   Endpoint Cost Service, but the client is a base ALTO client, then the
   client will ignore the path vector cost type without conducting any
   incompatibility.  If the client sents a request with path vector cost
   type, but the server is a base ALTO server, the server will return an
   "E_INVALID_FIELD_VALUE" error.







Bernstein, et al.       Expires September 6, 2018              [Page 19]


Internet-Draft         ALTO Extension: Path Vector            March 2018


8.2.  Compatibility with Multi-Cost Extensions

   Cost type path-vector is not a testable cost type.  Any format of
   constraints SHOULD NOT be applied to cost type path-vector in order
   for multi-cost to support the path-vector extension.  Specifically,

   o  Cost type path-vector MUST NOT be included in "testable-cost-
      types-names" or "testable-cost-types".

   o  When "testable-cost-types-names" is omitted in the "capabilities"
      and "testable-cost-types" is omitted in the input parameters,
      "constraints" or "or-constraints" SHOULD NOT add any format of
      constraints on cost type path-vector.

8.3.  Compatibility with Incremental Update

   [I-D.ietf-alto-incr-update-sse] defines incremental updates to ALTO
   resources and hence it can be applied to the path-vector resource
   defined in this document.

9.  Design Decisions and Discussions

9.1.  Provide More General Calendar Extension

   Cost Calendar is proposed as a useful ALTO extension to provide the
   historical cost values for Filtered Cost Map Service and Endpoint
   Cost Service.  Since path vector is an extension to these services,
   it SHOULD be compatible with Cost Calendar extension.

   However, the calendar of a path-vector (Endpoint) Cost Map is
   insufficient for the application which requires the historical data
   of routing state information.  The (Endpoint) Cost Map can only
   provide the changes of the paths.  But more useful information is the
   history of network element properties which are recorded in the
   dependent Network Element Property Map.

   Before the Unified Property Map is introduced as an ALTO extension,
   Filtered Cost Map Service and Endpoint Cost Service are the only
   resources which require the calendar supported.  Because other
   resources don't have to be updated frequently.  But Network Element
   Property Map as a use case of Unified Property Map will collect the
   real-time information of the network.  It SHOULD be updated as soon
   as possible once the metrics of network elements change.

   So the requirement is to provide a general calendar extension which
   not only meets the Filtered Cost Map and Endpoint Cost Service but
   also applies to the Property Map Service.




Bernstein, et al.       Expires September 6, 2018              [Page 20]


Internet-Draft         ALTO Extension: Path Vector            March 2018


10.  Security Considerations

10.1.  Privacy Concerns

   We can identify multiple potential security issues.  A main security
   issue is network privacy, as the path-vector information may reveal
   more network internal structures than the more abstract single-node
   abstraction.  The network should consider protection mechanisms to
   reduce information exposure, in particular, in settings where the
   network and the application do not belong to the same trust domain.
   On the other hand, in a setting of the same trust domain, a key
   benefit of the path-vector abstraction is reduced information
   transfer from the network to the application.

   The path-vector query may also reveal more information about the
   application.  In particular, the application may reveal all potential
   transfers sites (e.g., where the data source is replicated, and where
   the potential replication sites are).  The application should
   evaluate the potential privacy concerns.

   Beyond the privacy issues, the computation of the path-vector is
   unlikely to be cachable, in that the results will depend on the
   particular requests (e.g., where the flows are distributed).  Hence,
   this service may become an entry point for denial of service attacks
   on the availability of an ALTO server.  Hence, authenticity and
   authorization of this ALTO service may need to be better protected.

10.2.  Resource Consumption on ALTO Servers

   TODO: The Abstract Network Element Property Map is dynamically
   enriched when the (Filtered) Cost Map/Endpoint Cost Service is
   queried of the path-vector information.  The properties of the
   abstract network elements can consume a large amount of resources
   when cached.  So, a time-to-live is needed to remove outdated entries
   in the Abstract Network Element Property Map.

11.  IANA Considerations

11.1.  ALTO Cost Mode Registry

   This document specifies a new cost mode "array".  However, the base
   ALTO protocol does not have a Cost Mode Registry where new cost mode
   can be registered.  This new cost mode will be registered once the
   registry is defined either in a revised version of [RFC7285] or in
   another future extension.






Bernstein, et al.       Expires September 6, 2018              [Page 21]


Internet-Draft         ALTO Extension: Path Vector            March 2018


11.2.  ALTO Cost Metric Registry

   A new cost metric needs to be registered in the "ALTO Cost Metric
   Registry", listed in Table 2.

                   +-------------+---------------------+
                   | Identifier  | Intended Semantics  |
                   +-------------+---------------------+
                   | ane-path    | See Section 5.1.2   |
                   +-------------+---------------------+

                        Table 2: ALTO Cost Metrics

11.3.  ALTO Entity Domain Registry

   As proposed in Section 9.2 of [I-D.ietf-alto-unified-props-new],
   "ALTO Entity Domain Registry" is requested.  Besides, a new domain is
   to be registered, listed in Table 3.

   +-------------+--------------------------+--------------------------+
   | Identifier  | Entity Address Encoding  | Hierarchy & Inheritance  |
   +-------------+--------------------------+--------------------------+
   | ane         | See Section 5.2.2        | None                     |
   +-------------+--------------------------+--------------------------+

                        Table 3: ALTO Entity Domain

11.4.  ALTO Network Element Property Type Registry

   The "ALTO Abstract Network Element Property Type Registry" is
   required by the ALTO Entity Domain "ane", listed in Table 4.

                +-------------+--------------------------+
                | Identifier  | Intended Semantics       |
                +-------------+--------------------------+
                | availbw     | The available bandwidth  |
                | delay       | The transmission delay   |
                +-------------+--------------------------+

           Table 4: ALTO Abstract Network Element Property Types

12.  Acknowledgments

   The authors would like to thank discussions with Randriamasy Sabine,
   Andreas Voellmy, Erran Li, Haibin Son, Haizhou Du, Jiayuan Hu, Qiao
   Xiang, Tianyuan Liu, Xiao Shi, Xin Wang, and Yan Luo.





Bernstein, et al.       Expires September 6, 2018              [Page 22]


Internet-Draft         ALTO Extension: Path Vector            March 2018


13.  References

13.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997, <https://www.rfc-
              editor.org/info/rfc2119>.

13.2.  Informative References

   [I-D.amante-i2rs-topology-use-cases]
              Medved, J., Previdi, S., Lopez, V., and S. Amante,
              "Topology API Use Cases", draft-amante-i2rs-topology-use-
              cases-01 (work in progress), October 2013.

   [I-D.bernstein-alto-topo]
              Bernstein, G., Yang, Y., and Y. Lee, "ALTO Topology
              Service: Uses Cases, Requirements, and Framework", draft-
              bernstein-alto-topo-00 (work in progress), October 2013.

   [I-D.ietf-alto-cost-calendar]
              Randriamasy, S., Yang, Y., Wu, Q., Lingli, D., and N.
              Schwan, "ALTO Cost Calendar", draft-ietf-alto-cost-
              calendar-01 (work in progress), February 2017.

   [I-D.ietf-alto-incr-update-sse]
              Roome, W. and Y. Yang, "ALTO Incremental Updates Using
              Server-Sent Events (SSE)", draft-ietf-alto-incr-update-
              sse-08 (work in progress), January 2018.

   [I-D.ietf-alto-unified-props-new]
              Roome, W., Chen, S., xinwang2014@hotmail.com, x., Yang,
              Y., and J. Zhang, "Extensible Property Maps for the ALTO
              Protocol", draft-ietf-alto-unified-props-new-01 (work in
              progress), December 2017.

   [I-D.lee-alto-app-net-info-exchange]
              Lee, Y., Dhody, D., Wu, Q., Bernstein, G., and T. Choi,
              "ALTO Extensions to Support Application and Network
              Resource Information Exchange for High Bandwidth
              Applications in TE networks", draft-lee-alto-app-net-info-
              exchange-04 (work in progress), October 2013.

   [I-D.yang-alto-topology]
              Bernstein, G., Lee, Y., Roome, W., Scharf, M., and Y.
              Yang, "ALTO Topology Extensions: Node-Link Graphs", draft-
              yang-alto-topology-06 (work in progress), March 2015.



Bernstein, et al.       Expires September 6, 2018              [Page 23]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   [RFC2387]  Levinson, E., "The MIME Multipart/Related Content-type",
              RFC 2387, DOI 10.17487/RFC2387, August 1998,
              <https://www.rfc-editor.org/info/rfc2387>.

   [RFC7285]  Alimi, R., Ed., Penno, R., Ed., Yang, Y., Ed., Kiesel, S.,
              Previdi, S., Roome, W., Shalunov, S., and R. Woundy,
              "Application-Layer Traffic Optimization (ALTO) Protocol",
              RFC 7285, DOI 10.17487/RFC7285, September 2014,
              <https://www.rfc-editor.org/info/rfc7285>.

   [RFC8189]  Randriamasy, S., Roome, W., and N. Schwan, "Multi-Cost
              Application-Layer Traffic Optimization (ALTO)", RFC 8189,
              DOI 10.17487/RFC8189, October 2017, <https://www.rfc-
              editor.org/info/rfc8189>.

Authors' Addresses

   Greg Bernstein
   Grotto Networking
   Fremont, CA
   USA

   Email: gregb@grotto-networking.com


   Shiwei Dawn Chen
   Tongji University
   4800 Caoan Road
   Shanghai  201804
   China

   Email: dawn_chen_f@hotmail.com


   Kai Gao
   Tsinghua University
   Beijing  Beijing
   China

   Email: gaok12@mails.tsinghua.edu.cn


   Young Lee
   Huawei
   TX
   USA

   Email: leeyoung@huawei.com



Bernstein, et al.       Expires September 6, 2018              [Page 24]


Internet-Draft         ALTO Extension: Path Vector            March 2018


   Wendy Roome
   Nokia/Bell Labs (Retired)
   124 Burlington Rd
   Murray Hill, NJ  07974
   USA

   Phone: +1-908-464-6975
   Email: wendy@wdroome.com


   Michael Scharf
   Nokia
   Germany

   Email: michael.scharf@nokia.com


   Y. Richard Yang
   Yale University
   51 Prospect St
   New Haven  CT
   USA

   Email: yry@cs.yale.edu


   Jingxuan Jensen Zhang
   Tongji University
   4800 Caoan Road
   Shanghai  201804
   China

   Email: jingxuan.n.zhang@gmail.com


















Bernstein, et al.       Expires September 6, 2018              [Page 25]