[Search] [txt|pdf|bibtex] [Tracker] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01 02 03 04                                                
Network Working Group                                    L. Dunbar
Internet Draft                                   J. Kaippallimalil
Intended status: Standard                                Futurewei
Expires: August 22, 2021
                                                 February 22, 2021



            IPv6 Solution for 5G Edge Computing Sticky Service
           draft-dunbar-6man-5g-edge-compute-sticky-service-02

Abstract

   This draft describes an IPv6 solution that enables packets from an
   application on a UE (User Equipment) sticking to the same ANYCAST
   server location when the UE moves from one 5G cell site to another.

   The proposed solution expands the Application-aware ID and Service-
   Para options specified by [APN6] by including the ANYCAST Sticky
   Service location information in the IPv6 Hop-by-Hop or Destination
   Optional Header.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79. This document may not be modified,
   and derivative works of it may not be created, except to publish it
   as an RFC and to translate it into languages other than English.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt




xxx, et al.            Expires August 22, 2021            [Page 1]


Internet-Draft     IPv6 for 5G Edge Sticky Service

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   This Internet-Draft will expire on April 7, 2021.

Copyright Notice

   Copyright (c) 2021 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

1. Introduction.................................................. 3
      1.1. 5G Edge Computing Background.......................... 3
      1.2. 5G Edge Computing Network Properties.................. 3
      1.3. Problem #1: ANYCAST in 5G EC Environment.............. 5
      1.4. Problem #2: sticking to original App Server........... 6
      1.5. Problem #3: Application Server Relocation............. 6
2. Conventions used in this document............................. 7
3. Sticky Egress node to an ANYCAST Server....................... 8
4. End-Node based Sticky Service Solution........................ 9
      4.1. Sticky Egress Address Discovery...................... 10
      4.2. Sticky-Dst-SubTLV in Destination Extension Header.... 11
      4.3. Expected behavior at the UE.......................... 12
      4.4. Processing at the Ingress router..................... 12
5. Tunnel based Sticky Service Solutions........................ 13
      5.1. Ingress and Egress Routers Processing Behavior....... 13
      5.2. Scenario 1: Without Communication with 5G system..... 15
      5.3. Scenario 2: With communication with 5G system........ 15
6. Expanding APN6 for Sticky Service information................ 16
      6.1. Sticky Service ID encoded in the Application-aware ID 16
      6.2. Sticky Service Sub-TLV encoded in APN6 Service-para
      option.................................................... 16
7. Manageability Considerations................................. 17
8. Security Considerations...................................... 17
9. IANA Considerations.......................................... 17


Dunbar, et al.         Expires August 22, 2021            [Page 2]


Internet-Draft     IPv6 for 5G Edge Sticky Service

10. References.................................................. 17
      10.1. Normative References................................ 17
      10.2. Informative References.............................. 17
11. Acknowledgments............................................. 18

1. Introduction

  1.1. 5G Edge Computing Background

   As described in [5G-EC-Metrics], one Application in 5G Edge Computing
   environment can have multiple Application Servers hosted in different
   Edge Computing data centers that are close in proximity. Those Edge
   Computing (mini) data centers are usually very close to, or co-
   located with, 5G base stations, with the goal to minimize latency and
   optimize the user experience.

   When a UE (User Equipment) initiates application packets using the
   destination address from a DNS reply or from its own cache, the
   packets from the UE are carried in a PDU session through 5G Core
   [5GC] to the 5G UPF-PSA (User Plan Function - PDU Session Anchor).
   The UPF-PSA decapsulate the 5G GTP outer header and forwards the
   packets from the UEs to the Ingress router of the Edge Computing (EC)
   Local Data Network (LDN). The LDN for 5G EC, which is the IP Networks
   from 5GC perspective, is responsible for forwarding the packets to
   the intended destinations.

   When the UE moves out of coverage of its current gNB (next generation
   Node B) (gNB1), handover procedures are initiated and the 5G SMF
   (Session Management Function) also selects a new UPF-PSA. The
   standard handover procedures described in 3GPP TS 23.501 and TS
   23.502 are followed. When the handover process is complete, the UE
   has a new IP address and the IP point of attachment is to the new
   UPF-PSA. 5GC may maintain a path from the old UPF to new the UPF for
   a short period of time for SSC [Session and Service Continuity] mode
   3 to make the handover process more seamless.

  1.2. 5G Edge Computing Network Properties

   In this document, 5G Edge Computing Network refers to multiple
   Local IP Data Networks (LDN) in one region that interconnect
   the Edge Computing mini-data centers. Those IP LDN networks are
   the N6 interfaces from 3GPP 5G perspective.

   The ingress routers to the 5G Edge Computing Network are the
   routers directly connected to 5G UPFs. The egress routers to


Dunbar, et al.         Expires August 22, 2021            [Page 3]


Internet-Draft     IPv6 for 5G Edge Sticky Service

   the 5G Edge Computing Network are the routers that have direct
   link to the Edge Computing servers. The servers and the egress
   routers are co-located. Some of those mini Edge Computing Data
   centers may have Virtual switches or Top of Rack switches
   between the egress routers and the servers. But transmission
   delay between the egress routers and the Edge Computing servers
   is so small, therefore can be considered negligible in this
   document.

   When one mini data center has multiple Edge Computing Servers
   attached to one App Layer Load Balancer, only the App Layer
   Load Balancer is visible to the 5G Edge Computing Network. How
   the App Layer Load balancer manages the individual servers is
   out of the scope of the network layer.

   The Edge Computer Services are specially managed services that
   need to utilize the network topology and balance among multiple
   mini Edge Computing Data Centers with the same ANYCAST address.
   UEs can access many services that are not part of the
   registered 5G Edge Computing Services.



























Dunbar, et al.         Expires August 22, 2021            [Page 4]


Internet-Draft     IPv6 for 5G Edge Sticky Service

   +--+
   |UE|---\+---------+                 +------------------+
   +--+    |  5G     |    +---------+  |   S1: aa08::4450 |
   +--+    | Site +--++---+         +----+                |
   |UE|----|  A   |PSA| Ra|         | R1 | S2: aa08::4460 |
   +--+    |      +---+---+         +----+                |
  +---+    |         |  |           |  |   S3: aa08::4470 |
  |UE1|---/+---------+  |           |  +------------------+
  +---+                 |IP Network |       L-DN1
                        |(3GPP N6)  |
     |                  |           |  +------------------+
     | UE1              |           |  |  S1: aa08::4450  |
     | moves to         |          +----+                 |
     | Site B           |          | R3 | S2: aa08::4460  |
     v                  |          +----+                 |
                        |           |  |  S3: aa08::4470  |
                        |           |  +------------------+
                        |           |      L-DN3
   +--+                 |           |
   |UE|---\+---------+  |           |  +------------------+
   +--+    |  5G     |  |           |  |  S1: aa08::4450  |
   +--+    | Site +--++-+--+        +----+                |
   |UE|----|  B   |PSA| Rb |        | R2 | S2: aa08::4460 |
   +--+    |      +--++----+        +----+                |
   +--+    |         |  +-----------+  |  S3: aa08::4470  |
   |UE|---/+---------+                 +------------------+
   +--+                                     L-DN2
               Figure 1: App Servers in different edge DCs



  1.3. Problem #1: ANYCAST in 5G EC Environment

   Increasingly, ANYCAST is used extensively by various application
   providers and CDNs because it is possible to dynamically load balance
   across multiple locations of the same address based on network
   conditions. BGP is an integral part in the way IP anycast usually
   functions. Within BGP routing there are multiple routes for the same
   IP address which are pointing to different locations.

   Application Server location selection using Anycast address leverages
   the proximity information present in the network (routing) layer and
   eliminates the single point of failure and bottleneck at the DNS
   resolvers and application layer load balancers. Another benefit of
   using ANYCAST address is removing the dependency on UEs that use



Dunbar, et al.         Expires August 22, 2021            [Page 5]


Internet-Draft     IPv6 for 5G Edge Sticky Service

   their cached IP addresses instead of querying DNS when they move to a
   new location.

   But, having multiple locations for the same ANYCAST address in 5G
   Edge Computing environment can be problematic because all those edge
   computing Data Centers can be close in proximity.  There might not be
   any difference in the routing cost to reach the Application Servers
   in different Edge DCs.   Same routing cost to multiple ANYCAST
   locations can cause packets from one flow to be forwarded to
   different locations, which can cause service glitches.

  1.4. Problem #2: sticking to original App Server

   When a UE moves to a new location but continues the same application
   flow, routers at the new location might choose the App Server closer
   to the new location. As shown in the figure below, when the UE1 in
   5G-site-A moves to the 5G-Site-B, the router directly connected to 5G
   PSA2 might forward the packets destined towards the S1: aa08::4450 to
   the server located in L-DN2 because L-DN2 has the lowest cost based
   on routing.

   This is not the desired behavior for some services, which are called
   Sticky Services in this document.

   Even for some advanced applications with built-in mechanisms to re-
   sync the communications at the application layer after switching to a
   new location, service glitches are very often experienced by users.

   It worth noting that not all services need to be sticky. We assume
   only a subset of services are, and the Network is informed of the
   services that need to be sticky, usually by requests from application
   developers or controllers.

   This document describes an IPv6 based network layer solution to stick
   the packets belonging to the same flow of a UE to its original App
   Server location after the UE is anchored to a new UPF-PSA.

 1.5. Problem #3: Application Server Relocation

   When an Application Server is added to, moved, or deleted from a 5G
   Edge Computing Data Center, the routing protocol has to propagate the
   changes to 5G PSA or the PSA adjacent routers.  After the change, the
   cost associated with the site [5G-EC-Metrics] might change as well.

   Note: for the ease of description, the Edge Computing Server,
   Application Server, or App Server are used interchangeably throughout
   this document.


Dunbar, et al.         Expires August 22, 2021            [Page 6]


Internet-Draft     IPv6 for 5G Edge Sticky Service



 2. Conventions used in this document


   APN6        Application aware network using IPv6. The term
               "Application" has very broad meanings.  In this document
               the term "Application" refers to any applications that
               use ANYCAST servers in the 5G Edge Computing
               Environment.

   A-ER:       Egress Router to an Application Server, [A-ER] is used
               to describe the last router that the Application Server
               is attached. For 5G EC environment, the A-ER can be the
               gateway router to a (mini) Edge Computing Data Center.

   Application Server: An application server is a physical or virtual
               server that host the software system for the
               application.

   Application Server Location: Represent a cluster of servers at one
               location serving the same Application. One application
               may have a Layer 7 Load balancer, whose address(es) are
               reachable from external IP network, in front of a set of
               application servers. From IP network perspective, this
               whole group of servers are considered as the Application
               server at the location.

   Edge Application Server: used interchangeably with Application Server
               throughout this document.

   EC:                  Edge Computing

   Edge Hosting Environment: An environment providing support required
               for Edge Application Server's execution.

               NOTE: The above terminologies are the same as those used
               in 3GPP TR 23.758

   Edge DC:  Edge Data Center, which provides the Edge Computing Hosting
               Environment. It might be co-located with 5G Base Station
               and not only host 5G core functions.

   gNB                  next generation Node B



Dunbar, et al.         Expires August 22, 2021            [Page 7]


Internet-Draft     IPv6 for 5G Edge Sticky Service

   L-DN:       Local Data Network

   PSA:        PDU Session Anchor (UPF)

   SSC:        Session and Service Continuity

   UE:                  User Equipment

   UPF:        User Plane Function


   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

 3. Sticky Egress node to an ANYCAST Server

   From Local Data Network perspective, achieving Sticky Service for a
   flow from a UE to a specific ANYCAST server is same as delivering the
   packets of the flow to the same egress router, e.g. R1 in Figure 1,
   to which the ANYCAST Server is attached. The egress router, say R1 in
   Figure 1, should deliver the packets destined towards the ANYCAST
   address to its directly attached server even through the same address
   is also reachable from other routers, unless the directly attached
   server is no longer reachable due to Server or Link failure.

   The egress router, e.g. R1, to which the Edge Computing server is
   directly attached, is called the Sticky Egress node to the Edge
   Computing Server at the location. The Sticky Egress node has a
   unicast address.

   When there are multiple Edge Computing Servers with the same ANYCAST
   address located in different mini Edge Computing Data Centers, each
   location is identified by its Sticky Egress nodes unicast
   address(es). To the LDN's ingress routers, e.g. Ra or Rb in the
   Figure 1, achieving sticky service is to send the packets belonging
   to the same flow to the same Sticky Egress node's unicast address.

   From Local Data Network perspective, the Sticky Egress nodes' unicast
   addresses are the Next Hops (i.e. R1, R2, and R3) to reach the Edge
   Computing server ANYCAST address.

   The Flow Affinity feature, which is supported by most commercial
   routers today, can ensure packets belonging to one flow be forwarded



Dunbar, et al.         Expires August 22, 2021            [Page 8]


Internet-Draft     IPv6 for 5G Edge Sticky Service

   along the same path to the same egress router which then delivers the
   packets to the attached Edge Computing Server.

   Editor's note: for IPv6 traffic, Flow Affinity can be supported by
   the routers of the Local Data Network (LDN) forwarding the packets
   with the same Flow Label in the packets' IPv6 Header along the same
   path towards the same egress router. For IPv4 traffic, 5 tuples in
   the IPv4 header can be used to achieve the Flow Affinity

   The solution described in this document is to ensure a flow from a UE
   to be forwarded to the same egress router of the App Server after the
   UE moves to a new 5G Site which result in the UE being assigned a new
   IP address and attached to a new access router.

4. End-Node based Sticky Service Solution

   The End-Node based Sticky Service solution needs IPv6 end nodes to
   insert Destination Option header to the IPv6 header. Section 5
   describes a Sticky Service solution that do not require end nodes
   performing anything. Here are some assumptions for the End-Node based
   Sticky Service solution:

     - There is an EdgeCompute-Controller that can exchange information
        with the Edge Computing servers.
     - Network is aware of the Sticky Services by the Sticky Service
        Identifiers, which can be ANYCAST addresses or regular IP
        addresses. If an Edge Computing service needs to be sticky in
        the 5G Edge Computing environment, the service ID is registered
        with the 5G Edge Computing controller.

   Here is the overview of the End-Node based Sticky Service solution:
     - Each ANYCAST Edge Computing server either learns or is informed
        of the unicast Sticky Egress address (Section 3). The goal of
        the network is to deliver packets belonging to one flow to the
        same Sticky Egress address for the ANYCAST address. Section 4.1
        describes how Edge Computing Servers discover their
        corresponding Sticky Egress unicast addresses.
     - When an Edge Computing server sends data packets to a UE (or
        client), it inserts the Sticky-Dst-SubTLV (described in Section
        4.2) into the packets' Destination Option Header.
     - UE (or client) needs to copy the Destination Option Header from
        the received packet to the next packet's Destination Header if
        the next packet belongs to the same flow as the previous packet.




Dunbar, et al.         Expires August 22, 2021            [Page 9]


Internet-Draft     IPv6 for 5G Edge Sticky Service

     - If the following conditions are true, the ingress router
        encapsulates the packet from the UE in a tunnel whose outer
        destination address is set to the Sticky Egress Address
        extracted from the packet's Sticky-Dst-SubTLV:
          o The destination of the packet from the UE side matches
             with one of the Sticky Service ACLs configured on the
             ingress router of the LDN,
          o the packet header has the Destination Option present with
             Sticky-Dst-SubTLV.


     - Else (i.e. one of the conditions above is not true), the ingress
        node uses its own algorithm, such as the least cost as described
        in [5G-EC-Metrics], to select the optimal Sticky Egress address
        for forwarding the packet.


 4.1. Sticky Egress Address Discovery

   To an App server with ANYCAST address, the Sticky Egress address is
   same as its default Gateway address.

   To prevent malicious UEs (or clients) sending DDOS attacks to routers
   within 5G EC LDN, e.g. the Sticky Egress address that is encoded in
   the Destination option header in the packets sent back to the UEs (or
   clients), a proxy Sticky Egress address can be encoded in the
   Destination option header. The proxy Sticky Egress address is only
   recognizable by the 5G EC LDN ingress nodes, i.e. the Ra and Rb in
   the Figure 1, but not routable in other networks. The LDN ingress
   routers can translate the proxy Sticky Egress to a routable address
   for the Sticky Egress node after the source addresses of the packets
   are authenticated.














Dunbar, et al.         Expires August 22, 2021           [Page 10]


Internet-Draft     IPv6 for 5G Edge Sticky Service

 4.2. Sticky-Dst-SubTLV in Destination Extension Header

   A new Sticky-Dst-SubTLV is specified as below, which can be inserted
   into the IPv6 Destination Options header. The IPv6 Destination Option
   Header is specified by [RFC8200] as having a Next Header value of 60:

    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Next Header  |  Hdr Ext Len  |                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
    |                                                               |
    |                     Sticky-Dst-SubTLV                         |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+



     Sticky-Dst-SubTLV is specified as:

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Sticky-Type  |          Len  | AFI           | Reserved      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |Sticky Egress address (IPv4 or IPv6) for reaching the ANYCAST  |
   ~                                                               ~
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


     Sticky-Type = 1: indicate the Sticky Egress unicast address at
     encoded in the Sticky-Dst-SubTLV.





















Dunbar, et al.         Expires August 22, 2021           [Page 11]


Internet-Draft     IPv6 for 5G Edge Sticky Service


 4.3. Expected behavior at the UE

   For edge computing services that need sticky service while UEs move
   across multiple 5G sites, the UEs (the TCP/IP layer of the UEs' OS to
   be precise) need to extract the Destination Extension Header field
   from packets received from the App Server and inserts the extracted
   Destination Extension Header into the subsequent packets belonging to
   the same flow.

   Granted, it might take some time for IPv6 TCP/IP Layer to adopt the
   practice of copying the IPv6 Destination Extension Header field from
   the received packets to the subsequent packets belonging to the same
   flow. However, once the egress routers and ingress routers for 5G
   local data network support the feature, more and more Edge Computing
   services would want to utilize this special feature by adding this
   step.

   Section 4 describes the network layer processing if UEs do not
   perform the steps described here.

 4.4. Processing at the Ingress router

     - An Ingress router is configured with an ACL for filtering out
        the applications that need sticky service.

        Note, not all applications need sticky service. Using ACL can
        greatly reduce the processing on the routers.

     - When an Ingress router receives a packet from the 5G side that
        matches the ACL, the Ingress router extracts the Sticky-Dst-
        SubTLV from the packet IPv6 header if the field exists in the
        packet header.
     - Encapsulate the packet with the tunnel type that supported by
        the original Sticky Egress node, using the extracted Sticky
        Egress address in the destination field of the outer header, and
        forward the packet.

        Note: if the proxy Sticky Egress address is encoded in the
        Sticky-Dst-SubTLV, the ingress router needs to translate
        the proxy Sticky Egress address to a routable address.

     If none of the above conditions are met, the ingress router uses
     its own algorithm to select the optimal Sticky Egress node to
     forward the packet.



Dunbar, et al.         Expires August 22, 2021           [Page 12]


Internet-Draft     IPv6 for 5G Edge Sticky Service



 5. Tunnel based Sticky Service Solutions

   Before all UEs can change their processing behavior as described in
   the Section 4, a Tunnel based Sticky Service solution can be used.
   This solution doesn't depend on UE behavior, therefore, more
   processing on the LDN ingress routers is needed.
 5.1. Ingress and Egress Routers Processing Behavior

   The solution assumes that both ingress routers and egress routers
   support at least one type of tunnel and are configured with ACLs to
   filter out packets whose destination or source addresses match with
   the Sticky Service Identifier. The solution also assumes there are
   only limited number of Sticky Services to be supported.
   An ingress router needs to build a Sticky-Service-Table, with the
   minimum following attributes. The Sticky-Service-Table is initialized
   to be empty.
     - Sticky Service ID
     - Flow Label
     - Sticky Egress address
     - Timer

   Editor's Note:
     When a UE moves from one 5G Site to another, the same UE will have
     a new IP address. "Flow Label + Sticky Service ID" stays the same
     when a UE is anchored to a new PSA. Therefore, this solution use
     "Flow Label + Sticky Service ID" to identify a sticky flow. Since
     the chance of different UEs sending packets to the same ANYCAST
     address using the same Flow Label is very low, it is with high
     probability that "Flow Label + Sticky Service ID" can uniquely
     identify a flow. When multiple UEs using the same Flow Label
     sending packets to the same ANYCAST address, the solution described
     in this section will stick the flows to the same ANYCAST server
     attached to the Sticky Egress router. This behavior doesn't cause
     any harm.

   Each entry in the Sticky-Service-Table has a Timer because a sticky
   service is no longer sticky if there are no packets of the same flow
   destined towards the service ID for a period of time. The Timer
   should be larger than a typical TCP session Timeout value. An entry
   is automatically removed from the Sticky-Service-Table when its timer
   expires.


Dunbar, et al.         Expires August 22, 2021           [Page 13]


Internet-Draft     IPv6 for 5G Edge Sticky Service

   Note: since there are only small number of Sticky services, the
   Sticky-Service-Table is not very large.
   When an ingress router receives a packet from a UE matching with one
   of the Sticky Service ACLs and there is no entry in the Sticky-
   Service-Table matching the Flow Label and the Sticky Service ID, the
   ingress router considers the packet to be the first packet of the
   flow. There is no need to sticking the packet to any location. The
   ingress router uses its own algorithm to select the optimal egress
   node as the Sticky Egress address for the ANYCAST address,
   encapsulates the packet with a tunnel that is supported by the egress
   node. The tunnel's destination address is set to the egress node
   address.

   When an egress router receives a packet from an attached host with
   the packet's source address matching with one of the Sticky Service
   IDs, the egress router encapsulates the packet with a tunnel that is
   supported by the ingress router and the tunnel's destination address
   is set to the ingress router address. An Egress router learns the
   ingress router address for a UE IP address via BGP UPDATE messages.

   When an ingress router receives a packet in a tunnel from any egress
   router and the packet's source address matches with a Sticky Service
   ID, the egress router address is set as the Sticky Egress address for
   the Sticky Service ID. The ingress router adds the entry of "Sticky-
   Service-ID + Flow Label + the associated Sticky Egress address +
   Timer" to the Sticky-Service-Table if the entry doesn't exist yet in
   the table. If the entry exists, the ingress router refreshes the
   Timer of the entry in the table.

   When the ingress router receives the subsequent packets of a flow
   from the 5G side matching with an Sticky Service ID and the Sticky-
   Service ID exists in the Sticky-Service-Table, the ingress router
   uses the Sticky Egress address found in the Sticky-Service-Table to
   encapsulate the packet and refresh the Timer of the entry. If the
   Sticky-Service ID doesn't exist in the table, the ingress router
   considers the packet as the first packet of a flow.

   The subsequent sections describe how ingress nodes prorogate their
   Sticky-Service-Table to their neighboring ingress nodes. The
   propagation is for neighboring ingress nodes to be informed of the
   Sticky Egress address to a sticky service if a UE moves to a new
   neighboring 5G site resulting in anchoring to a new ingress node.







Dunbar, et al.         Expires August 22, 2021           [Page 14]


Internet-Draft     IPv6 for 5G Edge Sticky Service

 5.2. Scenario 1: Without Communication with 5G system

   When a UE moves to a very far away 5G site, say a different
   geographic region, the benefit of sticking to the original ANYCAST
   server is out weighted by network delay. Then, there is no point
   sending packets to the Sticky Egress node if the ingress router very
   far away. Therefore, it is necessary for each ingress router to have
   a group of neighboring ingress routers that are not too far away from
   the potential Sticky Egress nodes selected by the ingress router.
   This group of ingress routers is called the Neighboring Ingress
   Group. Each ingress router can either automatically discover its
   Neighboring Ingress Group by routing protocols or is configured by
   its controller. It is out of the scope of this document on how
   ingress nodes discover its Neighboring Ingress Group.

   Each ingress node needs to periodically advertise its Sticky-Service-
   Table to the routers within its Neighboring Ingress Group.

   Upon receiving the Sticky-Service-Table from routers in its
   Neighboring Ingress Group, each ingress router merges the entries
   from the received Sticky-Service-Table to its own.

   The ingress and the egress nodes perform the same actions as
   described in Section 5.1.

 5.3. Scenario 2: With communication with 5G system

   In this scenario, there is communication with 5G System and network
   get notified by a UE is anchored to a new PSA.

   When a UE is re-anchoring from PSA1 to PSA2, 5GC EC management system
   sends a notification to the router that is directly connected to
   PSA1. The notification includes the address of the new PSA that the
   UE is to be anchored, i.e. the PSA2, and the UE's new IP address.

   In this scenario, the Sticky Service can be uniquely identified by
   "Sticky Service ID" + "UE address". the Sticky-Service-Table should
   include the following attributes:

     - Sticky Service ID
     - UE address
     - Sticky Egress address
     - Timer

   Upon receiving the notification from the 5G EC management system, the
   ingress router (i.e. the one directly connected to the old PSA) sends
   the specific entry of the Sticky-Service Table, i.e. "Sticky Service


Dunbar, et al.         Expires August 22, 2021           [Page 15]


Internet-Draft     IPv6 for 5G Edge Sticky Service

   ID" + UE address + Sticky Egress + Timer to the router directly
   connected to the new PSA.

   Upon receiving the entry, the ingress router merges the entry into
   its own Sticky-Service-Table.

   The ingress and egress router processing are the same as described in
   Section 5.1 except a flow is now uniquely identified by the "Sticky
   Service ID" + "UE address" instead of "Sticky Service ID" + "Flow
   Label".

 6. Expanding APN6 for Sticky Service information

   The Application-aware ID and Service-Para Option described
   [APN6] can be expanded to include the sticky service
   information.

 6.1. Sticky Service ID encoded in the Application-aware ID

    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Sticky Level  |StickyServiceID| Reserved      | Flow ID       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Sticky Level: represent how important for an application to stick to
   its ANYCAST servers. Some applications may prefer one flow sticking
   to the original ANYCAST server, but not required. Some applications
   may require the stickiness.

   StickyServiceID: the ANYCAST address of the application servers.

   The Reserved field can be used for future to identifier the 5G access
   domain for the flow.

   Flow ID: the identifier for the flow that needs to stick to a
   specific ANYCAST server.


 6.2. Sticky Service Sub-TLV encoded in APN6 Service-para option

   The Sticky-Dst-SubTLV described in the Section 4.2 of this document
   can be included in the Service-Para Sub-TLVs field.







Dunbar, et al.         Expires August 22, 2021           [Page 16]


Internet-Draft     IPv6 for 5G Edge Sticky Service

 7. Manageability Considerations

     To be added.

 8. Security Considerations


   To be added.

 9. IANA Considerations

       To be added.

 10. References


 10.1. Normative References

   [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
             Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC4364] E. rosen, Y. Rekhter, "BGP/MPLS IP Virtual Private networks
             (VPNs)", Feb 2006.

   [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119
             Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May
             2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC8200] s. Deering R. Hinden, "Internet Protocol, Version 6 (IPv6)
             Specification", July 2017


 10.2. Informative References

   [3GPP-EdgeComputing] 3GPP TR 23.748, "3rd Generation Partnership
             Project; Technical Specification Group Services and System
             Aspects; Study on enhancement of support for Edge
             Computing in 5G Core network (5GC)", Release 17 work in
             progress, Aug 2020.

   [5G-EC-Metrics] L. Dunbar, H. Song, J. Kaippallimalil, "IP Layer
             Metrics for 5G Edge Computing Service", draft-dunbar-ippm-
             5g-edge-compute-ip-layer-metrics-00, work-in-progress, Oct
             2020.



Dunbar, et al.         Expires August 22, 2021           [Page 17]


Internet-Draft     IPv6 for 5G Edge Sticky Service

   [APN6]   Z. Li, et al, "Application-aware IPv6 Networking (APN6)
             Encapsulation", draft-li-6man-app-aware-ipv6-network-03,
             work-in-progress, Feb 2021.

   [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent
             Address Family Identifier (SAFI) and the BGP Tunnel
             Encapsulation Attribute", April 2009.

   [BGP-SDWAN-Port] L. Dunbar, H. Wang, W. Hao, "BGP Extension for SDWAN
             Overlay Networks", draft-dunbar-idr-bgp-sdwan-overlay-ext-
             03, work-in-progress, Nov 2018.

   [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. Majumdar,
             "BGP UPDATE for SDWAN Edge Discovery", draft-dunbar-idr-
             sdwan-edge-discovery-00, work-in-progress, July 2020.

   [Tunnel-Encap] E. Rosen, et al "The BGP Tunnel Encapsulation
             Attribute", draft-ietf-idr-tunnel-encaps-10, Aug 2018.



 11. Acknowledgments

   Acknowledgements to Ron Bonica and Donald Eastlake for their review
   and contributions.

   This document was prepared using 2-Word-v2.0.template.dot.

Authors' Addresses


   Linda Dunbar
   Futurewei
   Email: ldunbar@futurewei.com

   John Kaippallimalil
   Futurewei
   Email: john.kaippallimalil@futurewei.com









Dunbar, et al.         Expires August 22, 2021           [Page 18]