Unicorn: Resource Orchestration for Multi-Domain, Geo-Distributed Data Analytics
draft-xiang-alto-multidomain-analytics-02

Document Type Active Internet-Draft (individual)
Last updated 2018-07-02
Stream (None)
Intended RFC status (None)
Formats plain text pdf html bibtex
Stream Stream state (No stream defined)
Consensus Boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date
Responsible AD (None)
Send notices to (None)
ALTO WG                                                         Q. Xiang
Internet-Draft                                    Tongji/Yale University
Intended status: Informational                                     F. Le
Expires: January 3, 2019                                             IBM
                                                                 Y. Yang
                                                  Tongji/Yale University
                                                               H. Newman
                                      California Institute of Technology
                                                                   H. Du
                                                       Tongji University
                                                            July 2, 2018

 Unicorn: Resource Orchestration for Multi-Domain, Geo-Distributed Data
                               Analytics
             draft-xiang-alto-multidomain-analytics-02.txt

Abstract

   As the data volume increases exponentially over time, data analytics
   is transiting from a single-domain network to a multi-domain, geo-
   distributed network, where different member networks contribute
   various resources, e.g., computation, storage and networking
   resources, to collaboratively collect, share and analyze extremely
   large amounts of data.  Such a network calls for a resource
   orchestration framework that emphasizes the performance
   predictability of data analytics jobs, the high utilization of
   resources, and the autonomy and privacy of member networks.

   This document presents the design of Unicorn, a unified resource
   orchestration framework for multi-domain, geo-distributed data
   analytics, which uses the Application-Layer Traffic Optimization
   (ALTO) protocol as the key component for (1) allows member networks
   to provide accurate information on different types of resources; (2)
   keeps the private information of member networks; and (3) allows data
   analytics jobs to accurately describe their requirements of different
   types of resources.  As a part of Unicorn, an ALTO extension for
   privacy-preserving interdomain information aggregation is also
   presented.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute

Xiang, et al.            Expires January 3, 2019                [Page 1]
Internet-Draft               Unicorn Design                    July 2018

   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on January 3, 2019.

Copyright Notice

   Copyright (c) 2018 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Requirements Language . . . . . . . . . . . . . . . . . . . .   4
   3.  Changes Since Version -01 . . . . . . . . . . . . . . . . . .   4
   4.  Characteristics of Multi-Domain, Geo-Distributed Data
       Analytics . . . . . . . . . . . . . . . . . . . . . . . . . .   4
     4.1.  Dynamic Data Analytics Workload . . . . . . . . . . . . .   4
     4.2.  Dynamic Resource Availability . . . . . . . . . . . . . .   5
   5.  Design Requirements . . . . . . . . . . . . . . . . . . . . .   6
   6.  Review of Resource Orchestration Designs for Data Analytics .   6
     6.1.  Centralized resource-graph-based orchestration  . . . . .   7
     6.2.  Centralized ClassAds-based orchestration  . . . . . . . .   7
     6.3.  Distributed opportunistic orchestration . . . . . . . . .   7
     6.4.  Inadequacy of Existing Designs for Multi-Domain, Geo-
           Distributed Data Analytics  . . . . . . . . . . . . . . .   7
   7.  Unicorn Design  . . . . . . . . . . . . . . . . . . . . . . .   8
     7.1.  Choosing ALTO as the Resource Information Model . . . . .   8
     7.2.  Architecture of Unicorn . . . . . . . . . . . . . . . . .   9
Show full document text