Resource Orchestration for Multi-Domain Data Analytics

Document Type Active Internet-Draft (individual)
Last updated 2017-07-03
Stream (None)
Intended RFC status (None)
Formats plain text pdf html bibtex
Stream Stream state (No stream defined)
Consensus Boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date
Responsible AD (None)
Send notices to (None)
ALTO WG                                                         Q. Xiang
Internet-Draft                                    Tongji/Yale University
Intended status: Informational                                 H. Newman
Expires: January 4, 2018              California Institute of Technology
                                                            G. Bernstein
                                                       Grotto Networking
                                                                   H. Du
                                                  Tongji/Yale University
                                                                  K. Gao
                                                     Tsinghua University
                                                               A. Mughal
                                                               J. Balcas
                                      California Institute of Technology
                                                                J. Zhang
                                                       Tongji University
                                                                 Y. Yang
                                                  Tongji/Yale University
                                                            July 3, 2017

         Resource Orchestration for Multi-Domain Data Analytics


   Data-intensive analytics is entering the era of multi-domain,
   geographically-distributed, collaborative computing, where different
   organizations contribute various resources to collaboratively
   collect, share and analyze extremely large amounts of data.  Examples
   of this paradigm include the Compact Muon Solenoid (CMS) and A
   Toroidal LHC ApparatuS (ATLAS) experiments of the Large Hadron
   Collider (LHC) program.  Massive datasets continue to be acquired,
   simulated, processed and analyzed by globally distributed science
   networks in these collaborations.  Applications that manage and
   analyze such massive data volumes can benefit substantially from the
   information about networking, computing and storage resources from
   each member's site, and more directly from network-resident services
   that optimize and load balance resource usage among multiple data
   transfers and analytics requests, and achieve a better utilization of
   multiple resources in clusters.

   The Application-Layer Traffic Optimization (ALTO) protocol can
   provide via extensions the network information about different
   clusters/sites, to both users and proactive network management
   services where applicable, with the goal of improving both
   application performance and network resource utilization.  In this
   document, we propose that it is feasible to use existing ALTO
   services to provides not only network information, but also

Xiang, et al.            Expires January 4, 2018                [Page 1]
Internet-Draft        ExaScale Network Optimization            July 2017

   information about computation and storage resources in data analytics
   networks.  We introduce a uniform resource orchestration framework
   (Unicorn), which achieves an efficient multi-resource allocation to
   support low-latency dataset transfer and data intensive analytics for
   collaborative computing.  It collects cluster information from
   multiple ALTO services utilizing topology extensions and leverages
   emerging SDN control capabilities to orchestrate the resource
   allocation for dataset transfers and analytics tasks, leading to
   improved transfer and analytics latency as well as more efficient
   utilization of multi-resources in sites.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on January 4, 2018.

Copyright Notice

   Copyright (c) 2017 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   ( in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
Show full document text