Traffic Optimization for ExaScale Science Applications
draft-xiang-alto-exascale-network-optimization-00

Abstract

Massive datasets continue to be acquired, simulated, processed and analyzed by globally distributed scientific collaborations, and the volume of this data is growing exponentially. These datasets need to be exchanged through a global network infrastructure. Applications that manages the transfer of such massive data volumes can benefit substantially from the utilization of network information, and more directly from network-resident services that optimize and load balance network usage among multiple transfer requests, and coordinate the network use with the use of other resources such as computing and storage. The Application-Layer Traffic Optimization (ALTO) protocol can provide via extensions such network information, to both users and proactive network management services where applicable, with the goal of improving both application performance and network resource utilization, leading to greater overall efficiency of the science programs' workflows. This document introduces an Exascale Dataset Transfer Orchestrator (EDTO), which is a data transfer scheduling service for exascale science networks. EDTO provides simple APIs for users to submit, update and delete data transfer requests and to monitor the status of each transfer, along with local and global network and site state information in real-time. EDTO collects network information from multiple ALTO services utilizing proposed topology extensions and leverages emerging SDN control capabilities to orchestrate the scheduling of multiple large dataset transfers, leading to improved data transfer latency and reliability as well as more efficient utilization of limited network resources.

Authors

Qiao Xiang
Harvey Newman
Greg M. Bernstein
Azher Mughal
Justas Balcas

(Note: The e-mail addresses provided for the authors of this Internet-Draft may no longer be valid.)