[Search] [pdf|bibtex] [Tracker] [Email] [Nits]

Versions: 00                                                            
rtgwg                                                            L. Geng
Internet-Draft                                                    P. Liu
Intended status: Informational                              China Mobile
Expires: May 3, 2021                                           P. Willis
                                                        October 30, 2020

Dynamic-Anycast in Compute First Networking (CFN-Dyncast) Use Cases and
                           Problem Statement


   Service providers are exploring the edge computing to achieve better
   response time, control over data and carbon energy saving by moving
   the computing services towards the edge of the network in scenarios
   of 5G MEC (Multi-access Edge Computing), virtualized central office,
   and others.  Providing services by sharing computing resources from
   multiple edges is emerging and becoming more and more useful for
   computationally intensive tasks.  The service nodes attached to
   multiple edges normally have two key features, service equivalency
   and service dynamism.  Ideally they should serve the service in a
   computational balanced way.  However lots of approaches dispatch the
   service in a static way, e.g., to the geographically closest edge,
   and they may cause unbalanced usage of computing resources at edges
   which further degrades user experience and system utilization.  This
   draft provides an overview of scenarios and problems associated.

   Networking taking account of computing resource metrics as one of its
   top parameters is called Compute First Networking (CFN) in this
   document.  The document identifies several key areas which require
   more investigations in architecture and protocol to achieve the
   balanced computing and networking resource utilization among edges in

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any

Geng, et al.               Expires May 3, 2021                  [Page 1]

Internet-Draft     CFN-Dyncast Use Cases and Problems       October 2020

   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on May 3, 2021.

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Definition of Terms . . . . . . . . . . . . . . . . . . . . .   4
   3.  Main Use-Cases  . . . . . . . . . . . . . . . . . . . . . . .   4
     3.1.  Cloud Based Recognition in Augmented Reality (AR) . . . .   4
     3.2.  Connected Car . . . . . . . . . . . . . . . . . . . . . .   5
     3.3.  Cloud Virtual Reality (VR)  . . . . . . . . . . . . . . .   5
   4.  Requirements  . . . . . . . . . . . . . . . . . . . . . . . .   6
   5.  Problems Statement  . . . . . . . . . . . . . . . . . . . . .   6
     5.1.  Anycast based service addressing methodology  . . . . . .   7
     5.2.  Flow affinity . . . . . . . . . . . . . . . . . . . . . .   7
     5.3.  Computing Aware Routing . . . . . . . . . . . . . . . . .   8
   6.  Summary . . . . . . . . . . . . . . . . . . . . . . . . . . .   8
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
   9.  Informative References  . . . . . . . . . . . . . . . . . . .   9
   Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .   9
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   9

1.  Introduction

   Edge computing aims to provide better response times and transfer
   rate, with respect to Cloud Computing, by moving the computing
   towards the edge of the network.  Edge computing can be built on
   industrial PCs, embedded systems, gateways and others.  They are put
   close to the end user.  There is an emerging requirement that
   multiple edge sites (called edges too in this document) are deployed

Geng, et al.               Expires May 3, 2021                  [Page 2]

Internet-Draft     CFN-Dyncast Use Cases and Problems       October 2020

   at different locations to provide the service.  There are millions of
   home gateways, thousands of base stations and hundreds of central
   offices in a city that can serve as candidate edges for hosting
   service nodes.  Depending on the location of the edge and its
   capacity, each edge has different computing resources to be used for
   a service.  At peak hour, computing resources attached to a client's
   closest edge site may not be sufficient to handle all the incoming
   service demands.  Longer response time or even demand dropping can be
   experienced by the user.  Increasing the computing resources hosted
   on each edge site to the potential maximum capacity is neither
   feasible nor economical.

   Some user devices are purely battery-driven.  Offloading the
   computation intensive processing to the edge can save the battery.
   Moreover the edge may use the data set (for the computation) that may
   not exist on the user device because of the size of data pool or data
   governance reasons.

   At the same time, with the new technologies such as serverless
   computing and container based virtual functions, service node on an
   edge can be easily created and terminated in a sub-second scale.  It
   makes the available computing resources for a service change
   dramatically over time at an edge.

   DNS-based load balancing usually configures a domain in Domain Name
   System (DNS) such that client requests to the domain are distributed
   across a group of servers.  It usually provides several IP addresses
   for a domain name.  The traditional techniques to manage the overall
   load balancing process of clients issuing requests including choose-
   the-closest or round-robin.  The are relatively static which may
   cause the unbalanced workload distribution in terms of network load
   and computational load.

   There are some dynamic ways which tries to distribute the request to
   the server with the best available resources and minimal load.  They
   usually require L4-L7 handling of the packet processing.  It is not
   an efficient approach for huge number of short connections.  At the
   same time, such approaches can hardly get network status in real
   time.  Therefore the choice of the service node is almost entirely
   determined by the computing status, rather than the comprehensive
   consideration of both computing and network.

   Networking taking account of computing resource metrics as one of its
   top parameters is called Compute First Networking (CFN) in this
   document.  Edge site can interact with each other to provide network-
   based edge computing service dispatching to achieve better load
   balancing in CFN.  Both computing load and network status are network
   visible resources.

Geng, et al.               Expires May 3, 2021                  [Page 3]

Internet-Draft     CFN-Dyncast Use Cases and Problems       October 2020

   A single service has multiple instances attached to multiple edge
   computing sites is conceptually like anycast in network language.
   Because of the dynamic and anycast aspects of the problem, jointly
   with the CFN deployment, we generally refer to it in this document as
   CFN-Dyncast, as for Compute First Networking Dynamic Anycast.  This
   draft describes usage scenarios, problem space and key areas of CFN-

2.  Definition of Terms

   CFN: Compute First Networking.  Networking architecture taking
   account of computing resource metrics as one of its top parameters to
   achieve flexible load management and performance optimizations in
   terms of both network and computing resources.

   CFN-Dyncast: Compute First Networking Dynamic Anycast.  The dynamic
   and anycast aspects of the architecture in a CFN deployment.

3.  Main Use-Cases

   This section presents several typical scenarios which require
   multiple edge sites to interconnect and to co-ordinate at network
   layer to meet the service requirements and ensure user experience.

3.1.  Cloud Based Recognition in Augmented Reality (AR)

   In AR environment, the end device captures the images via cameras and
   sends out the computing intensive service demand.  Normally service
   nodes at the edge are responsible for tasks with medium computational
   complexity or low latency requirement like object detection, feature
   extraction and template matching, and service nodes at cloud are
   responsible for the most intensive computational tasks like object
   recognition or latency non-sensitive tasks like AI based model
   training.  The end device hence only handles the tasks like target
   tracking and image display, thereby reducing the computing load of
   the client.

   The computing resource for a specific service at the edge can be
   instantiated on-demand.  Once the task is completed, this resource
   can be released.  The lifetime of such "function as a service" can be
   on a millisecond scale.  Therefore computing resources on the edges
   have distributed and dynamic natures.  A service demand has to be
   sent to and served by an edge with sufficient computing resource and
   a good network path.

Geng, et al.               Expires May 3, 2021                  [Page 4]

Internet-Draft     CFN-Dyncast Use Cases and Problems       October 2020

3.2.  Connected Car

   In auxiliary driving scenarios, to help overcome the non-line-of-
   sight problem due to blind spot or obstacles, the edge node can
   collect the comprehensive road and traffic information around the
   vehicle location and perform data processing, and then the vehicles
   in high security risk can be signaled.  It improves the driving
   safety in complicated road conditions, like at the intersections.
   The video image information captured by the surveillance camera is
   transmitted to the nearest edge node for processing.  Warnings can be
   sent to the cars driving too fast or under other invisible dangers.

   When the local edge node is overloaded, the service demand sent to it
   will be queued and the response from the auxiliary driving will be
   delayed, and it may lead to traffic accidents.  Hence, in such cases,
   delay-insensitive services such as in-vehicle entertainment should be
   dispatched to other light loaded nodes instead of local edge nodes,
   so that the delay-sensitive service is preferentially processed
   locally to ensure the service availability and user experience.

3.3.  Cloud Virtual Reality (VR)

   Cloud VR introduces the concept and technology of cloud computing and
   rendering into VR applications.  Edge cloud helps encode/decode and
   rendering in this scenario.  The end device usually only uploads the
   posture or control information to the edge and then VR contents are
   rendered in edge cloud.  The video and audio outputs generated from
   edge cloud are encoded, compressed, and transmitted back to the end
   device or further transmitted to central data center via high
   bandwidth network.

   Cloud VR services have high requirements on both network and
   computing.  For example, for an entry-level Cloud VR (panoramic 8K 2D
   video) with 110-degree Field of View (FOV) transmission, the typical
   network requirements are bandwidth 40Mbps, RTT 20ms, packet loss rate
   is 2.4E-5; the typical computing requirements are 8K H.265 real-time
   decoding, 2K H.264 real-time encoding.

   Edge site may use CPU or GPU for encode/decode.  GPU usually has
   better performance but CPU is more simple and straight forward for
   use.  Edges have different computing resources in terms of CPU and
   GPU physically deployed.  Available remaining resource determines if
   a gaming instance can be started.  The instance CPU, GPU and memory
   utilization has a high impact on the processing delay on encoding,
   decoding and rendering.  At the same time, the network path quality
   to the edge site is a key for user experience on quality of audio/
   video and game command response time.

Geng, et al.               Expires May 3, 2021                  [Page 5]

Internet-Draft     CFN-Dyncast Use Cases and Problems       October 2020

   Cloud VR service brings challenging requirements on both network and
   computing so that the edge node to serve a service demand has to be
   carefully selected to make sure it has sufficient computing resource
   and good network path.

4.  Requirements

   This document mainly targets at the typical edge computing scenarios
   with two key features, service equivalency and service dynamism.

   o  Service equivalency: A service is provided by one or more service
      instances, providing an equivalent service functionality to
      clients, while the existence of several instances is (possibly
      across multiple edges) is to ensure better scalability and

   o  Service dynamism: A single instance has very dynamic resources
      over time to serve a service demand.  Its dynamism is affected by
      computing resource capability and load, network path quality, and
      etc.  The balancing mechanisms should adapt to the service
      dynamism quickly and seamlessly.  Failover kind of switching is
      not desired.

5.  Problems Statement

   A service demand should be routed to the most suitable edge and
   further to the service instance in real time among the multiple edges
   with service equivalency and dynamism.  Existing mechanisms use one
   or more of the following ways and each of them has issues associated.

   o  Use the least network cost as metric to select the edge.  Issue:
      Computing information is a key to be considered in edge computing,
      and it is not included here.

   o  Use geographical location deduced from IP prefix, pick the closest
      edge.  Issue: Edges are not so far apart in edge computing
      scenario.  Either hard to be deduced from IP address or the
      location is not the key distinguisher.

   o  Health check in an infrequent base (>1s) to reflect the service
      node status, and switch when fail-over.  Issue: Health check is
      very different from computing status information of service
      instance.  It is too coarse granularity.

   o  Application layer randomly picks or uses round-robin way to pick a
      service node.  Issue: It may share the load across multiple
      service instances in terms of the computing capacity, the network
      cost variance is barely considered.  Edges can be deployed in

Geng, et al.               Expires May 3, 2021                  [Page 6]

Internet-Draft     CFN-Dyncast Use Cases and Problems       October 2020

      different cities which are not equal cost paths to a client.
      Therefore network status is also a major concern.

   o  Global resolver and early binding (DNS-based load balancing):
      Client queries a global resolver or load balancer first and gets
      the exact server's address.  And then steer traffic using that
      address as binding address.  It is called early binding because an
      explicit binding address query has to be performed before sending
      user data.  Issue: Firstly, it clashes with the service dynamism.
      Current resolver does not have the capability of such high
      frequent change of indirection to new instance based on the
      frequent change of each service instance.  Secondly, edge
      computing flow can be short.  One or two round trip would be
      completed.  Out-of-band query for specific server address has high
      overhead as it takes one more round trips.  As discussed in
      section 5.4 of [I-D.sarathchandra-coin-appcentres], the flexible
      re-routing to appropriate service instances out of a pool of
      available ones faces significant challenges when utilizing DNS for
      this purpose.

   o  Traditional anycast.  Issue: Only works for single request/reply
      communication.  No flow affinity guaranteed.

   A network based dynamic anycast (Dyncast) architecture aims to
   address the following points in CFN (CFN-Dyncast).

5.1.  Anycast based service addressing methodology

   A unique service identifier is used by all the service instances for
   a specific service no matter which edge it attaches to.  An anycast
   like addressing and routing methodology among multiple edges makes
   sure the data packet potentially can reach any of the edges with the
   service instance attached.  At the same time, each service instance
   has its own unicast address to be used by the attaching edge to
   access the service.  From service identifier (an anycast address) to
   a specific unicast address, the discovery and mapping methodology is
   required to allow in-band service instance and edge selection in real
   time in network.

5.2.  Flow affinity

   The traditional anycast is normally used for single request single
   response style communication as each packet is forwarded individually
   based on the forwarding table at the time.  Packets may be sent to
   different places when the network status changes.  CFN in edge
   computing requires multiple request multiple response style
   communication between the client and the service node.  Therefore the

Geng, et al.               Expires May 3, 2021                  [Page 7]

Internet-Draft     CFN-Dyncast Use Cases and Problems       October 2020

   data plane must maintain flow affinity.  All the packets from the
   same flow should go to the same service node.

5.3.  Computing Aware Routing

   Given that the current state of the art for routing is based on the
   network cost, computing resource and/or load information is not
   available or distributed at the network layer.  At the same time,
   computing resource metrics are not well defined and understood by the
   network.  They can be CPU/GPU capacity and load, number of sessions
   currently serving, latency of service process expected and the
   weights of each metric.  Hence it is hard to make the best choice of
   the edge based on both computing and network metrics at the same

   Computing information metric representation has to be agreed on by
   the participated edges and metrics are to be exchanged among them.

   Network cost in current routing system does not change very
   frequently.  However computing load is highly dynamic information.
   It changes rapidly with the number of sessions, CPU/GPU utilization
   and memory space.  It has to be determined at what interval or event
   such information needs to be distributed among edges.  More frequent
   distribution more accurate synchronization, but also more overhead.

   Choosing the least path cost is the most common rule in routing.
   However, the logic does not work well in computing aware routing.
   Choosing the least computing load may result in oscillation.  The
   least load edge can quickly be flooded by the huge number of new
   computing demands and soon become overloaded.  Tidal effect may

   Depending on the usage scenario, computing information can be carried
   in BGP, IGP or SDN-like centralized way.  More investigations in
   those solution spaces is to be elaborated in other documents.  It is
   out of scope of this draft.

6.  Summary

   This document presents the CFN-Dyncast problem statement.  CFN-
   Dyncast aims at leveraging the resources mobile providers have
   available at the edge of their networks.  However, CFN-Dyncast aims
   at taking into account as well the dynamic nature of service demands
   and the availability of network resources so as to satisfy service
   requirements and load balance among service instances.

   This also document illustrate some use cases problems and list the
   requirements for CFN-Dyncast.  CFN-Dyncast architecture should

Geng, et al.               Expires May 3, 2021                  [Page 8]

Internet-Draft     CFN-Dyncast Use Cases and Problems       October 2020

   addresses how to distribute the computing resource information at the
   network layer and how to assure flow affinity in an anycast based
   service addressing environment.

7.  Security Considerations


8.  IANA Considerations

   No IANA action is required so far.

9.  Informative References

              Trossen, D., Sarathchandra, C., and M. Boniface, "In-
              Network Computing for App-Centric Micro-Services", draft-
              sarathchandra-coin-appcentres-03 (work in progress),
              October 2020.


   The author would like to thank Yizhou Li, Luigi IANNONE and Dirk
   Trossen for their valuable suggestions to this document.

Authors' Addresses

   Liang Geng
   China Mobile

   Email: gengliang@chinamobile.com

   Peng Liu
   China Mobile

   Email: liupengyjy@chinamobile.com

   Peter Willis

   Email: peter.j.willis@bt.com

Geng, et al.               Expires May 3, 2021                  [Page 9]