SFC WG                                                         D. Lachos
Internet-Draft                                                   Unicamp
Intended status: Informational                                  Q. Xiang
Expires: January 14, 2021                         Tongji/Yale University
                                                           C. Rothenberg
                                                                 Y. Yang
                                                  Tongji/Yale University
                                                           July 13, 2020

            Multi-domain Service Function Chaining with ALTO


   The delivery of network services often require service functions and
   their specific order, called a service function chain (SFC).  A SFC
   request is usually composed by distributed resources which are
   expected to available across multiple domains with different
   technology and/or administration.  This document describes different
   standardization activities and research projects addressing the
   challenges posed by SFC across multiple domains (specifically,
   multiple administrative domains).  In addition, this document
   presents an initial approach to realize inter-domain service chains
   leveraging the Application Layer Traffic Optimization (ALTO)
   protocol.  Finally, another important concern of this document is to
   initiate a discussion (ALTO, SFC as well as other WGs) regarding if,
   how, and under what conditions ALTO can be useful to improve the
   multi-domain SFC process.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on January 14, 2021.

Lachos, et al.          Expires January 14, 2021                [Page 1]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   4
   3.  Context and Motivation  . . . . . . . . . . . . . . . . . . .   4
     3.1.  Standardization Activities  . . . . . . . . . . . . . . .   4
       3.1.1.  IETF  . . . . . . . . . . . . . . . . . . . . . . . .   4
       3.1.2.  ETSI  . . . . . . . . . . . . . . . . . . . . . . . .   5
       3.1.3.  MEF . . . . . . . . . . . . . . . . . . . . . . . . .   5
     3.2.  Research projects . . . . . . . . . . . . . . . . . . . .   5
   4.  ALTO for Multi-domain SFC . . . . . . . . . . . . . . . . . .   6
     4.1.  Advantages of using ALTO  . . . . . . . . . . . . . . . .   7
       4.1.1.  Inter-domain info discovery with ALTO Property Map  .   7
       4.1.2.  Inter-domain path computation with ALTO Cost Map  . .   7
     4.2.  Motivating Use Cases  . . . . . . . . . . . . . . . . . .   8
       4.2.1.  ALTO as part of the SFC eXchange Platform . . . . . .   8
       4.2.2.  Resource Orchestration for Multi-Domain, Geo-
               Distributed Data Analytics  . . . . . . . . . . . . .  10
   5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  12
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .  12
   7.  Summary and Outlook . . . . . . . . . . . . . . . . . . . . .  13
   8.  Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  13
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  13
     9.1.  Normative References  . . . . . . . . . . . . . . . . . .  13
     9.2.  Informative References  . . . . . . . . . . . . . . . . .  14
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  16

1.  Introduction

   The delivery of end-to-end services often requires various Service
   Functions (SFs).  Service Function Chaining (SFC) is an abstracted
   view of a service that defines a set of required SFs as well as the
   order in which they must be executed [RFC7665].  Multi-domain SFC is

Lachos, et al.          Expires January 14, 2021                [Page 2]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

   the ability to deploy SFC across multiple domains with different
   technology and/or administration.  To do so, an inter-domain
   communication process between different organizations is necessary in
   order to (i) exchange abstract topology, resource and service
   information, and then (ii) compute inter-domain service function

   Nowadays, different standardization efforts (e.g., IETF, MEF, ETSI)
   and research projects activities (e.g., 5GEx [H2020.5GEX], 5G-
   Transformer [H2020-5G-TRANSFORMER], T-NOVA [T-NOVA]) have been
   focused on multi-domain network service chaining.  Standarization is
   essential to provide recommendations to create interoperable
   architectures with standardized protocols, and solutions (being
   developed by different projects) are addressing a diverse range of
   requirements to provide network services provided using multiple
   administrative domains.

   More recently, the ALTO WG started to discuss the uses of ALTO as an
   information model for representing network resource and services in
   multi-domain scenarios:

   o  [DRAFT-ALTO-BROKER-MDO] proposes an ALTO-based Broker-assisted
      architecture where a broker plane works as a coordinator between a
      set of top-level control planes, i.e., Multi-domain Orchestrator
      (MdOs).  The ALTO services (with the proposed extensions) provides
      abstract maps with a simplified, yet enough information view about
      MdOs involved in the federation.  This information includes the
      abstract network topology, resource availability (e.g., CPUs,
      Memory, and Storage) and capabilities (e.g., supported NFs).

   o  The document [DRAFT-ALTO-UNICORN] presents Unicorn, a resource
      orchestration framework for multi-domain, geo-distributed data
      analytics.  This work resorts in ALTO as the information model to
      support the accurate, yet privacy-preserving resource discovery
      across different domains.  The key information to be provided by
      the use of ALTO including different types of resources, e.g., the
      computing, storage, and networking resources.

   In summary, this document offers (i) an overview reference of several
   initiatives (standardization efforts and projects) behind building a
   complete multi-domain SFC, and (ii) concrete use case examples of how
   ALTO can be incorporated in the multi-domain SFC architecture.

   The overall rationale of this document is to begin a discussion
   between the SFC and the ALTO WG (other WGs are welcome) concerning
   if, how, and under which conditions ALTO will be helpful in the SFC
   traversing different administrative domains.

Lachos, et al.          Expires January 14, 2021                [Page 3]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

2.  Terminology

   This document makes use of the terminology defined
   and [RFC7665].

3.  Context and Motivation

   In order to offer a complete end-to-end network service, the multi-
   domain approach involves two different aspects: multiple
   administrations or multi-domain single
   administrations [DRAFT-MD-VIRT].

   o  Multiple Administrations: Market fragmentation results from having
      different operators focused on a specific region.  This makes
      difficult to deploy new services, for example, virtual
      connectivity spanning multiple countries.

   o  Multi-domain Single Administrations: Technology fragmentation
      creates multi-domain single administration.  For example,
      different parts of a network could be created as different domains
      using separate technologies.

   This section summarizes, on the one hand, main standardization
   efforts delivering collections of norms and recommendations
   (architectures, frameworks, protocols), while on the other hand it
   also provides an overview of several projects formed to develop
   network services across multiple domains.

3.1.  Standardization Activities

3.1.1.  IETF

   SFC that span domains owned by single or multiple administrative
   entities are being proposed.  The Hierarchical Service Function
   Chaining (hSFC) [RFC8459], for example, defines an architecture to
   deploy SFC in large networks.  This RFC proposes to decompose the
   network into smaller domains (domains under the control of a single
   organization).  Another proposed initiative is [DRAFT-HH-MDSFC] that
   describes SFC crossing different domains owned by various
   organizations (e.g., ISPs) or by a single organization with
   administration partitions.  The proposed architecture uses a SFC
   eXchange Platform (SXP) to collect and exchange information
   (topology, service states, policies, etc.) between different
   organizations and it works both in centralized (Multiple SFC domains
   connected by a logical SXP) and distributed (SXP server as a broker)

Lachos, et al.          Expires January 14, 2021                [Page 4]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

   Another initiative is the Network Function Virtualization Research
   Group (NFVRG).  The draft "Multi-domain Network Virtualization"
   [DRAFT-MD-VIRT] envisions a complete end-to-end logical network as
   stitching services offered by multiple domains from multiple
   providers.  It also points to the need for creating solutions that
   enable the exchange of relevant information (resources and
   topologies) across different providers.

3.1.2.  ETSI

   The ETSI NFV ISG is paving the way toward viable architectural
   options supporting the efficient placement of functions in different
   administrative domains.  More specifically, the document
   [ETSI-NFV-IFA028] reports different NFV MANO architectural approaches
   with use cases related to network services provided using multiple
   administrative domains.  Besides, it gives a non-exhaustive list of
   key information to be exchanged between administrative domains
   (monitoring parameters, topology view, resource capabilities, etc.)
   and recommendations related to security to permit the correct and
   proper operation of the final service.

3.1.3.  MEF

   With its work on the Service Operations Specification MEF
   55 [MEF-SOE-MEF55], MEF has defined a reference architecture and
   framework for describing functional management entities (and
   interfaces between them) needed to support Lifecycle Service
   Orchestration (LSO).  This LSO architecture enables automated
   management and control of E2E connectivity services across multiple
   operator networks.  The automated service management includes
   fulfillment, control, performance, assurance, usage, security,
   analytics, and policy capabilities that make it possible, for
   example, expanding the footprint of service providers to interact
   with potentially several operators to manage and control the access
   portions of E2E services.

3.2.  Research projects

   Several projects include an architectural model integrating NFV
   management with SDN control capabilities to address the challenges
   towards flexible, dynamic, cost-effective, and on-demand service

   [H2020.5GEX] aims to integrate multiple administrations and
   technologies through the collaboration between operators in the
   context of emerging 5G networking. [VITAL][T-NOVA] follow a
   centralized approach where each domain advertises its capabilities to
   a federation layer which will act as a broker.  In order to avoid one

Lachos, et al.          Expires January 14, 2021                [Page 5]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

   network operator per country or regions, [H2020-5G-NORMA] proposes
   the use of management and control into a single virtual domain.
   Also, the 5G-Transformer project [H2020-5G-TRANSFORMER] is defining
   flexible slicing and federation of transport networking and computing
   resources across multiple domains.

4.  ALTO for Multi-domain SFC

   A "dialogue" between potential domains that will provide multi-domain
   SFC could be beneficial for a more efficient use of resources and
   increasing the SFC performance.  However, constrained knowledge of
   the network services and underlying network topology based only on
   localized views from the point of view of a single domain limits the
   potential and scope for multi-domain SFC.

   Note:  The examples used in this document are based on architectures
      and assumptions currently being proposed in the SFC
      WG [DRAFT-HH-MDSFC] and in the ALTO

   To enable a highly customized multi-domains SFC, [DRAFT-HH-MDSFC]
   proposes a SFC eXchange Platform to realize inter-domain
   communication between top-level control planes.  The SXP is a logical
   entity deployed in future Software-defined IXP (as a trusted third-
   party platform) or built by a single owner between different

   On a high level, the scope of the SXP contains two main tasks:

   o  Provide end-to-end visibility through the collection of topology
      information, service states, and policies from different domains.

   o  Compute inter-domain service function path to select the service
      function location from multiple candidate domains.

   The ALTO protocol [RFC7285] provides abstract network information in
   the form of map services that can be consumed by applications in
   order to become network-aware and to take optimized decisions
   regarding traffic flows.  Recently, ALTO is also being considered in
   multi-domain orchestration scenarios [DRAFT-ALTO-UNICORN]
   [DRAFT-ALTO-BROKER-MDO], in which an ALTO server can convey inter-
   domain network resource and topology information.

   In this context, the SXP can take advantage of multi-domain ALTO
   services to obtain important inter-domain information to "guide" the
   resource/service provider selection process in that the "best" domain
   or candidate domains (according to established policies) can be

Lachos, et al.          Expires January 14, 2021                [Page 6]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

   intelligently selected.  The following ALTO services can be

4.1.  Advantages of using ALTO

   ALTO (and customized ALTO extensions) can be used to offer
   aggregated/abstracted views on various types of information including
   domain-level topology, storage resources, computation resources,
   networking resources and PNF/VNF capabilities.  This generic
   representation contributing to a more simple and scalable solution
   for resource and service discovery in multi-domain, multi-technology

   In case of Multi-domain SFC, the following ALTO services could be

4.1.1.  Inter-domain info discovery with ALTO Property Map

   Each domain needs a global view of other potential candidate domains
   to know who can provide part of the SF in the SFC.  A brief list of
   information to be exchanged between different domains includes:

   o  Resource capabilities, applicable to both IT (computing and
      storage) and networking resources participant of the multi-domain
      SFC, to assist on the decision of SFs placement.

   o  Access information (e.g., URL) to the orchestrator entry points
      and Service Access Points (SAPs) for a corresponding network/

   The ALTO Property Map Service [DRAFT-ALTO-PM] can provide a clear
   global view of the resource information offered by other domains.
   This information allows discovering which candidate domains may be
   contacted to deliver the remaining requirements of a requested end-
   to-end service deployment.

4.1.2.  Inter-domain path computation with ALTO Cost Map

   Once the candidate domains are discovered, it is necessary to compute
   inter-domain service function path to select the service function
   location from those different candidate domains.

   The connectivity information among discovered domains can be
   retrieved by an ALTO Cost Map service, responding, for instance, a
   path vector with the AS-level topology distance between the source
   domain and candidate domains.  Moreover, path vector constraints (as
   described in the Multi-Cost Map [RFC8189]) can be applied to filter
   out the list of unqualified domains.

Lachos, et al.          Expires January 14, 2021                [Page 7]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

   In case of the Hybrid Hierarchical SFC architecture [DRAFT-HH-MDSFC],
   the SXP (or the Path Calculation Element in the top-level control
   plane) could use this information to compute multi-domain service
   function paths.

4.2.  Motivating Use Cases

4.2.1.  ALTO as part of the SFC eXchange Platform

   As mentioned earlier, [DRAFT-HH-MDSFC] defines a multi-domain SFC
   architecture that combines control planes to be deployed either into
   a large domain consisting of smaller sub-domains owned by the same
   organization or into multiple large domains with different ownership.
   Figure 1 shows a SXP connecting three different domains (AS1, AS2,
   AS3).  Each domain provides different SFs: AS1 -> SF1; AS2 -> SF2 and
   SF3; AS3 -> SF3.  The SXP includes an ALTO server component to
   provide abstract topology, resource, and service information for the
   high-level control plane in each domain.

Lachos, et al.          Expires January 14, 2021                [Page 8]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

                                 SFC eXchange
                            |     +--------+     |
                            |     | ALTO   |     |
                            |     | Server |     |
                            |     +--------+     |
                       +---->                    <-----+
                       |    +----------^---------+     |
                       |               |               |
                       |               |               |
                       |               |               |
                  +----v---+     +-----v--+     +------v-+
                  |Control |     |Control |     |Control |
                  |Plane   |     |Plane   |     |Plane   |
                  +--------+     +--------+     +--------+
                       |              |              |
                       |              |              |
                       |              |              |
                  +--------+     +--------+     +--------+
                  |Data    |     |Data    |     |Data    |
                  |Plane   -------Plane   -------Plane   |
                  +--------+     +--------+     +--------+
                     SF1            SF2            SF3


            Figure 1: ALTO as part of the SFC eXchange Platform

   Every domain has a local Information Base Element; this component can
   be used by the SXP to create hierarchical databases containing inter-
   domain resource and topology information.  This information source is
   used by the ALTO server to create two different ALTO Map Services:
   (i) Property Map and (ii) Cost Map.

   The Property Map includes a property value grouped by Autonomous
   System (AS), this value contains the supported network functions.
   Additional properties could be considered such as resource
   availability (e.g., CPUs, Memory, and Storage), orchestrator entry
   points, etc.  An example of the Property Map in our basic scenario

Lachos, et al.          Expires January 14, 2021                [Page 9]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

     |     | Capabilities | Entry Point | CPU | MEM | Storage | ... |
     | AS1 |    {SF1}     |  http://... | ... | ... |   ...   | ... |
     | AS2 |  {SF2, SF3}  |  http://... | ... | ... |   ...   | ... |
     | AS3 |    {SF3}     |  http://... | ... | ... |   ...   | ... |

                        Table 1: ALTO Property Map

   The Cost Map defines a path vector as an array of ASes, representing
   the AS-level topological distance for a given SFC request.  Table 2
   below shows a brief example of a service request and its inter-domain
   service function path response containing a list of potential domains
   to be traversed to deliver such service.

         |  SFC Request  | Multi-domain Service Function Path(s) |
         | SF1->SF2->SF3 |     1:{AS1:SF1->AS2:SF2->AS2:SF3}     |
         |               |     2:{AS1:SF1->AS2:SF2->AS3:SF3}     |

                          Table 2: ALTO Cost Map

4.2.2.  Resource Orchestration for Multi-Domain, Geo-Distributed Data

   In addition to commercial SFC, ALTO is also used as a core
   information model for collaborative data science networks.  The
   document [DRAFT-ALTO-UNICORN] presents the design of Unicorn, a
   unified resource orchestration framework for multi-domain, geo-
   distributed data analytics, currently being developed and deployed in
   the CMS network, one of the largest scientific experiments in the LHC

   ALTO is well suited as a fundamental component in Unicorn for
   providing a generic representation that (1) allows different types of
   data analytics jobs to accurately describe their resource
   requirements and (2) allows member networks to provide accurate
   information on different types of resources they own and at the same
   time maintain their privacies.

Lachos, et al.          Expires January 14, 2021               [Page 10]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

          .-------------.           .-------------.
          |Application 1|    ...    |Application N|
          '-------------'           '-------------'
                \                         /
 .- - - - - - - -\- - - - - - - - - - - -/- - - - - - - - - - - - - - -.
 |  Unicorn       \                     /                              |
 |               .-----------------------.                             |
 |               | Resource Orchestrator |     .----------------------.|
 |               |                       |     |Distributed Hash Table||
 |               |     .-----------.     |---- |   of Computing and   ||
 |               |     |ALTO Client|     |     |   Storage Resources  ||
 |               |     '-----------'     |     '----------------------'|
 |               '-----------------------'                             |
 |             /            |            \                             |
 |            /             |             \                            |
 |   .-------------.  .-----------.      .-------------.               |
 |   |ALTO Server 1|  | Execution |      |ALTO Server M|               |
 |   '-------------'  |  Agents   |      '-------------'               |
 |          |         '-----------'            |                       |
 |          |          /           \           |                       |
 |  .----------------./             \ .----------------.               |
 |  |     Site 1     |       . .      |     Site N     |               |
 |  '----------------'                '----------------'               |
 '- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -'

                    Figure 2: Architecture of Unicorn.

   Figure 2 presents the architecture of Unicorn.  Specifically, for
   each member network, one or more ALTO servers are deployed to provide
   accurate, yet privacy-preserving information of different types of
   resources owned by the corresponding network.  Examples of such
   information include the link bandwidth between endpoints, the memory
   I/O bandwidth and the CPU utilization at computing endpoints and the
   storage space at storage endpoints.  In addition to the basic ALTO
   services defined in [RFC7285], The ALTO servers in Unicorn also
   provide ALTO extension services such as the ALTO Multi-Cost Service
   [RFC8189], the ALTO Server-Sent Event Service [DRAFT-ALTO-INCR-UPD]
   and the ALTO Path Vector Service [DRAFT-ALTO-PV] to provide fine-
   grained resource information.

   Because the ALTO Path Vector service may expose additional private
   information of each network, Unicorn develops an obfuscating protocol
   which ensures that nor the orchestrator or any member networks can
   associate any path vector information with a corresponding network.

   To better address the scalability issue of multi-domain resource
   discovery, Unicorn also develops a proactive full-mesh discovery

Lachos, et al.          Expires January 14, 2021               [Page 11]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

   mechanism, which precomputes network-level ALTO path vector
   information and performs projection using such information to compute
   the fine-grained resource information in response to orchestrator's
   resource discovery requests.

   Details of the obfuscating protocol and the proactive full-mesh
   discovery mechanism developed in Unicorn can be found in the
   [DRAFT-ALTO-UNICORN] document.

5.  IANA Considerations

   This document includes no request to IANA.

6.  Security Considerations

   The ALTO base protocol has an extensive discussion on potential
   security and privacy issues.  Using the ALTO base protocol to support
   multi-domain SFC will not raise new security and privacy issue.
   However, the information provided by the ALTO base protocol are
   considered coarse-grained in several recent use cases.  As a result,
   several ALTO extension services have been designed to provide fine-
   grained network information to the application.  Using these ALTO
   extension services for multi-domain SFC would raise new security and
   privacy concerns.  Next we list these issues on a per extension

   The ALTO unified property extension [DRAFT-ALTO-PM] generalizes the
   concept of endpoint properties to other entity domains, such as
   abstract network element.  The properties of these entities may
   contain sensitive service-function-specific information.  Exposing
   such information may discourage networks to provide fine-grained
   information to support multi-domain SFC.

   The ALTO performance cost metrics extension [DRAFT-ALTO-METRICS]
   proposes a set of ALTO cost metrics derived from traffic engineering
   tools and protocols.  It is stated in this extension that "sharing
   network TE metric values in numerical mode requires full mutual
   confidence between the entities managing the ALTO Server and Client."
   In multi-domain SFC use case, such mutual confidence is needed not
   only between ALTO server and client, but also among all networks, and
   third-parties such as broker and a global orchestrator.  How to
   achieve such mutual confidence in multi-domain SFC use case requires
   further investigation.

   The ALTO path vector extension [DRAFT-ALTO-PV] allows ALTO clients to
   query network information such as capacity region for a given set of
   flows.  Several related studies have shared concerns that this
   extension may reveal more network internal structures than the more

Lachos, et al.          Expires January 14, 2021               [Page 12]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

   abstract single-node abstraction used in the ALTO base protocol.  In
   multi-domain SFC, this concern will further be amplified as third-
   party participants may access such information.  The recent designed
   Unicorn system proposes an obfuscating protocol that prevent the
   receiver of the capacity region information from associating this
   region to any network.  This protocol sheds light for addressing the
   privacy issue brought by the ALTO path vector extension.

   The ALTO cost calendar [DRAFT-ALTO-CALENDAR] and the ALTO incremental
   update [DRAFT-ALTO-INCR-UPD] extensions allows the ALTO client to get
   temporal network information.  The intention of these extensions is
   to allow applications to make flexible decisions on when to use
   network information.  However, both extensions expose temporal policy
   and traffic information of network so that a user may know when the
   network is most vulnerable for overloading.  This issue need to be
   carefully addressed in order for both extensions to be used for
   multi-domain SFC.

7.  Summary and Outlook

   This document introduced initiatives and solutions being proposed in
   the context of SFC traversing different domains.  It is also provided
   initial arguments why ALTO is a meaningful protocol in such multi-
   domain scenario, and it presented use case examples about the how
   ALTO can be used to advertise and discover abstract topology,
   resource and service information from different domains, and then
   compute inter-domain service function paths.

   The overall objective of this document is to arouse discussions in
   the SFC WG in order to assess the suitability of the ALTO as a useful
   protocol for multi-domain SFC scenarios.  The result of such
   discussions will be captured in future versions of this draft.

8.  Acknowledgments

   This work is supported by the Innovation Center of Ericsson S.A.,
   Brazil (grant agreement UNI.64).

   Many thanks to Sabine Randriamasy, and Lyle Bertz for their feedback
   on this draft.

9.  References

9.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997,

Lachos, et al.          Expires January 14, 2021               [Page 13]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

   [RFC7285]  Alimi, R., Ed., Penno, R., Ed., Yang, Y., Ed., Kiesel, S.,
              Previdi, S., Roome, W., Shalunov, S., and R. Woundy,
              "Application-Layer Traffic Optimization (ALTO) Protocol",
              RFC 7285, DOI 10.17487/RFC7285, September 2014,

   [RFC7665]  Halpern, J., Ed. and C. Pignataro, Ed., "Service Function
              Chaining (SFC) Architecture", RFC 7665,
              DOI 10.17487/RFC7665, October 2015,

   [RFC8189]  Randriamasy, S., Roome, W., and N. Schwan, "Multi-Cost
              Application-Layer Traffic Optimization (ALTO)", RFC 8189,
              DOI 10.17487/RFC8189, October 2017,

9.2.  Informative References

              Perez, D. and C. Rothenberg, "ALTO-based Broker-assisted
              Multi-domain Orchestration", draft-lachosrothenberg-alto-
              brokermdo-03 (work in progress), March 2020.

              Randriamasy, S., Yang, Y., WU, Q., Lingli, D., and N.
              Schwan, "Application-Layer Traffic Optimization (ALTO)
              Cost Calendar", draft-ietf-alto-cost-calendar-21 (work in
              progress), March 2020.

              Roome, W. and Y. Yang, "ALTO Incremental Updates Using
              Server-Sent Events (SSE)", draft-ietf-alto-incr-update-
              sse-22 (work in progress), March 2020.

              WU, Q., Yang, Y., Dhody, D., Randriamasy, S., and L.
              Contreras, "ALTO Performance Cost Metrics", draft-ietf-
              alto-performance-metrics-09 (work in progress), March

              Roome, W., Randriamasy, S., Yang, Y., Zhang, J., and K.
              Gao, "Unified Properties for the ALTO Protocol", draft-
              ietf-alto-unified-props-new-11 (work in progress), March

Lachos, et al.          Expires January 14, 2021               [Page 14]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

              Gao, K., Randriamasy, S., Yang, Y., and J. Zhang, "ALTO
              Extension: Path Vector", draft-ietf-alto-path-vector-10
              (work in progress), March 2020.

              Xiang, Q., Zhang, J., Le, F., Yang, Y., and H. Newman,
              "Resource Orchestration for Multi-Domain, Exascale, Geo-
              Distributed Data Analytics", draft-xiang-alto-multidomain-
              analytics-03 (work in progress), March 2020.

              Li, G., Li, G., Xu, Q., Zhou, H., and B. Feng, "Hybrid
              Hierarchical Multi-Domain Service Function chaining",
              draft-li-sfc-hhsfc-08 (work in progress), March 2020.

              Bernardos, C., Contreras, L., Vaishnavi, I., Szabo, R.,
              Li, X., Paolucci, F., Sgambelluri, A., Martini, B.,
              Valcarenghi, L., Landi, G., Andrushko, D., and A. Mourad,
              "Multi-domain Network Virtualization", draft-bernardos-
              nfvrg-multidomain-05 (work in progress), September 2018.

              ETSI, "Report on architecture options to support multiple
              administrative domains V3.1.1", Jan 2018,

              H2020, "5G-NORMA -- 5G Novel Radio Multiservice adaptive
              network Architecture", 2015, <https://5gnorma.5g-ppp.eu/>.

              H2020, "5G-Transformer -- 5G Mobile Transport Platform for
              Vertical", 2017, <http://5g-transformer.eu/>.

              Bernardos, C., Dugeon, O., Galis, A., Morris, D., Simon,
              C., and R. Szabo, "5G Exchange (5GEx)--Multi-domain
              Orchestration for Software Defined Infrastructures",
              focus vol. 4, no.5, p.2, 2015.

              Metro Ethernet Forum, "Lifecycle Service Orchestration
              (LSO): Reference Architecture and Framework", Mar 2016,

Lachos, et al.          Expires January 14, 2021               [Page 15]

Internet-Draft         Multi-domain SFC with ALTO              July 2020

   [T-NOVA]   FP7 project T-NOVA, "T-NOVA Project, Network Functions as
              a Service over Virtualised Infrastructures", 2014,

   [VITAL]    VITAL PROJECT H2020, "VITAL -- VIrtualized hybrid
              satellite-TerrestriAl systems for resilient and fLexible
              future networks", 2015, <http://www.ict-vital.eu/>.

Authors' Addresses

   Danny Alex Lachos Perez
   University of Campinas
   Av. Albert Einstein 400
   Campinas, Sao Paulo  13083-970

   Email: dlachosp@dca.fee.unicamp.br
   URI:   https://intrig.dca.fee.unicamp.br/danny-lachos/

   Qiao Xiang
   Tongji/Yale University
   51 Prospect Street
   New Haven, CT

   Email: qiao.xiang@cs.yale.edu

   Christian Esteve Rothenberg
   University of Campinas
   Av. Albert Einstein 400
   Campinas, Sao Paulo  13083-970

   Email: chesteve@dca.fee.unicamp.br
   URI:   https://intrig.dca.fee.unicamp.br/christian/

   Y. Richard Yang
   Tongji/Yale University
   51 Prospect St
   New Haven, CT

   Email: yang.r.yang@gmail.com

Lachos, et al.          Expires January 14, 2021               [Page 16]