A Massive Data Migration Framework
draft-yangcan-ietf-data-migration-standards-01

Document Type Active Internet-Draft (individual)
Last updated 2018-12-05
Stream (None)
Intended RFC status (None)
Formats plain text pdf xml html bibtex
Stream Stream state (No stream defined)
Consensus Boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date
Responsible AD (None)
Send notices to (None)
Internet Engineering Task Force                             C. Yang, Ed.
Internet-Draft                                               Y. Liu, Ed.
Intended status: Standards Track    South China University of Technology
Expires: June 8, 2019                                            C. Chen
                                                                  Inspur
                                                                 G. Chen
                                                                    GSTA
                                                                  Y. Wei
                                                                  Huawei
                                                        December 5, 2018

                   A Massive Data Migration Framework
             draft-yangcan-ietf-data-migration-standards-01

Abstract

   This document describes a standardized framework for implementing the
   massive data migration between traditional databases and big-data
   platforms on the cloud via Internet, especially for an instance of
   Hadoop data architecture.  The main goal of the framework is to
   provide more concise and friendly interfaces for users more easily
   and quickly migrate the massive data from a relational database to a
   distributed platform for a variety of requirements, in order to make
   full use of distributed storage resource and distributed computing
   capability to solve the bottleneck problems of both storage and
   computing performance in traditional enterprise-level applications.
   This document covers the fundamental architecture, data elements
   specification, operations, and interface related to massive data
   migration.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on June 8, 2019.

Yang, et al.              Expires June 8, 2019                  [Page 1]
Internet-Draft          Data Migration Standards           December 2018

Copyright Notice

   Copyright (c) 2018 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Definitions and Terminology . . . . . . . . . . . . . . . . .   4
   3.  Specific Framework Implementation Standards . . . . . . . . .   6
     3.1.  System Architecture Diagram . . . . . . . . . . . . . . .   6
     3.2.  Source and Target of Migration  . . . . . . . . . . . . .   7
       3.2.1.  The Data Sources of Migration . . . . . . . . . . . .   7
       3.2.2.  The Connection Testing of Relational Data Sources . .   7
       3.2.3.  The Target Storage Container of Data Migration  . . .   8
       3.2.4.  Specifying Target Cloud Platform  . . . . . . . . . .   8
       3.2.5.  Data Migration to third-party Web Applications  . . .   8
     3.3.  Type of Migrated Database . . . . . . . . . . . . . . . .   8
     3.4.  Scale of Migrated Table . . . . . . . . . . . . . . . . .   9
       3.4.1.  Full Table Migration  . . . . . . . . . . . . . . . .   9
       3.4.2.  Single Table Migration  . . . . . . . . . . . . . . .   9
       3.4.3.  Multi-table migration . . . . . . . . . . . . . . . .   9
     3.5.  Split-by  . . . . . . . . . . . . . . . . . . . . . . . .  10
       3.5.1.  Single Column . . . . . . . . . . . . . . . . . . . .  10
       3.5.2.  Multiple Column . . . . . . . . . . . . . . . . . . .  11
       3.5.3.  Non-linear Segmentation . . . . . . . . . . . . . . .  11
     3.6.  Conditional Query Migration . . . . . . . . . . . . . . .  11
     3.7.  Dynamic Detection of Data Redundancy  . . . . . . . . . .  11
     3.8.  Data Migration with Compression . . . . . . . . . . . . .  12
     3.9.  Updating Mode of Data Migration . . . . . . . . . . . . .  12
       3.9.1.  Appending Migration . . . . . . . . . . . . . . . . .  12
Show full document text