Skip to main content

Mobile User Plane Evolution
draft-zzhang-dmm-mup-evolution-09

Document Type Active Internet-Draft (candidate for dmm WG)
Authors Zhaohui (Jeffrey) Zhang , Keyur Patel , Luis M. Contreras , Kashif Islam , Jari Mutikainen , Tianji Jiang , Luay Jalil , Ori Prio Sejati , Shay Zadok
Last updated 2024-07-02
RFC stream Internet Engineering Task Force (IETF)
Intended RFC status (None)
Formats
Additional resources Mailing list discussion
Stream WG state Call For Adoption By WG Issued
Document shepherd (None)
IESG IESG state I-D Exists
Consensus boilerplate Unknown
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-zzhang-dmm-mup-evolution-09
dmm                                                             Z. Zhang
Internet-Draft                                          Juniper Networks
Intended status: Informational                                  K. Patel
Expires: 3 January 2025                                           Arrcus
                                                            L. Contreras
                                                              Telefonica
                                                                K. Islam
                                                                  Redhat
                                                           J. Mutikainen
                                                              NTT Docomo
                                                                T. Jiang
                                                            China Mobile
                                                                L. Jalil
                                                                 Verizon
                                                               O. Sejati
                                                               XL Axiata
                                                                S. Zadok
                                                                Broadcom
                                                             2 July 2024

                      Mobile User Plane Evolution
                   draft-zzhang-dmm-mup-evolution-09

Abstract

   This document describes evolution of mobile user plane in 5G,
   including distributed User Plane Functions (UPFs) and alternative
   user plane implementations that some vendors/operators are promoting
   without changing 3GPP architecture/signaling, and further discusses
   potentially integrating UPF and Access Node (AN) in 6G mobile
   networks.

   This document is not an attempt to do 3GPP work in IETF.  Rather, it
   discusses potential integration of IETF/wireline and 3GPP/wireless
   technologies - first among parties who are familiar with both areas
   and friendly with IETF/wireline technologies.  If the ideas in this
   document are deemed reasonable, feasible and desired among these
   parties, they can then be brought to 3GPP for further discussions.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

Zhang, et al.            Expires 3 January 2025                 [Page 1]
Internet-Draft                MUP Evolution                    July 2024

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 3 January 2025.

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Current User Plane in 5G  . . . . . . . . . . . . . . . . . .   3
   2.  MUP Evolution in 5G . . . . . . . . . . . . . . . . . . . . .   4
     2.1.  Distributed UPFs  . . . . . . . . . . . . . . . . . . . .   4
       2.1.1.  Advantages of Distributed PSA UPFs  . . . . . . . . .   6
       2.1.2.  Enablers of Distributed PSA UPFs  . . . . . . . . . .   7
     2.2.  Alternative Transport Options for 5G  . . . . . . . . . .   8
       2.2.1.  GTP vs. SRv6 vs. MPLS tunneling . . . . . . . . . . .   8
       2.2.2.  Routing Based UPF-Lite  . . . . . . . . . . . . . . .   8
   3.  MUP Evolution for 6G  . . . . . . . . . . . . . . . . . . . .   9
     3.1.  UPF Distribution and RAN Decomposition  . . . . . . . . .  10
     3.2.  Integrated AN/UP Function (ANUP)  . . . . . . . . . . . .  10
     3.3.  ANUP Potential Use-case: 5G-A Satellite Services  . . . .  12
     3.4.  ANUP-like Feature in 4G: Local IP Access (LIPA) . . . . .  13
   4.  ANUP: Advanced Technical Considerations . . . . . . . . . . .  14
     4.1.  Separate AN/UP Functions  . . . . . . . . . . . . . . . .  14
     4.2.  Simplified/reduced Signaling and optimized data plane . .  15
     4.3.  Microservice architecture . . . . . . . . . . . . . . . .  16
     4.4.  Increased burden on previously simple AN  . . . . . . . .  16
     4.5.  Use of ULCL I-UPF for MEC Purpose . . . . . . . . . . . .  17
     4.6.  VPN PE Function in AN/ANUP  . . . . . . . . . . . . . . .  18

Zhang, et al.            Expires 3 January 2025                 [Page 2]
Internet-Draft                MUP Evolution                    July 2024

     4.7.  QoS Handling  . . . . . . . . . . . . . . . . . . . . . .  19
     4.8.  NAT . . . . . . . . . . . . . . . . . . . . . . . . . . .  19
   5.  ANUP Implications: IETF to 3GPP . . . . . . . . . . . . . . .  20
     5.1.  User-plane/UP vs. Control-plane/CP  . . . . . . . . . . .  20
     5.2.  Impacts & Intentions to 5G/6G CP  . . . . . . . . . . . .  20
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .  20
   7.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  21
   8.  Informative References  . . . . . . . . . . . . . . . . . . .  21
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  22

1.  Current User Plane in 5G

   Mobile User Plane (MUP) in 5G [_3GPP-23.501] has two distinct parts:
   the Access Network part between UE and AN/gNB, and the Core Network
   part between AN/gNB and UPF.

                              N3              N9             N6
       UE          AN(gNB)    |    I-UPF      |  PSA UPF     |
   +---------+                |               |              |
   |App Layer|                |               |    routing   |    __
   +---------+                |               |+--/---+---\-+|   (  )
   |PDU Layer|      relay     |    relay      || PDU  |     ||  (    )
   +---------+ +---/--+--\---+|+---/--+--\---+|+------+IP+L2|| (      )
   |         | |      |GTP-U |||GTP-U |GTP-U |||GTP-U |     || (  DN  )
   | 5G-AN   | |5G-AN +------+||------+------+||------+  or || (      )
   |         | |      |UDP+IP|||UDP+IP|UDP+IP|||UDP+IP|     ||  (    )
   | Proto   | |Proto +------+||------+------+||------+Ether||   (  )
   |         | |      |  L2  |||  L2  |  L2  |||  L2  |     ||    --
   | Layers  | |Layers+------+||------+------+||------+-----+|
   |         | |      |  L1  |||  L1  |  L1  |||  L1  |  L1 ||
   +---------+ +------+------+|+------+------+|+------+-----+|
                              |               |              |

   For the core network (CN) part, N3 interface extends the PDU layer
   from AN/gNB towards the PSA UPF, optionally through I-UPFs and in
   that case N9 interface is used between I-UPF and PSA UPF.
   Traditionally, UPFs are deployed at central locations and the N3/N9
   tunnels extend the PDU layer to them.  The N3/N9 interface uses GTP-U
   tunnels that are typically over a VPN over a transport
   infrastructure.  While N6 is a 3GPP defined interface, it is for
   reference only and there is no tunneling or specification involved.
   It is simply a direct IP (in case of IP PDU session) or Ethernet (in
   case of Ethernet PDU session) connection to the DN.

   At the AN/gNB, relay is done between the radio layer and the GTP-U
   layer.  At the PSA UPF, routing/switching is done for IP/Ethernet
   before GTP-U encapsulation (for downlink traffic) or after GTP-U
   decapsulation (for uplink traffic).

Zhang, et al.            Expires 3 January 2025                 [Page 3]
Internet-Draft                MUP Evolution                    July 2024

2.  MUP Evolution in 5G

2.1.  Distributed UPFs

   With MEC, ULCL UPFs are deployed closer to gNBs, while centralized
   PSA UPFs are still used to provide persistent IP addresses to UEs.

   In fact, even PSA UPFs could be distributed closer to gNBs and then
   the N3 interface becomes very simple – over a direct or short
   transport connection between gNB and UPF (or even an internal
   connection if the gNB and UPF are hosted on the same server).  On the
   other hand, since the UPF to DN connection is direct, the DN becomes
   a VPN (e.g., IP VPN in case of IP PDU sessions or EVPN in case of
   Ethernet PDU sessions) over a transport infrastructure, most likely
   the same transport infrastructure for the VPN supporting the N3/N9
   tunneling in centralized PSA UPF case, as shown in the following
   picture:

Zhang, et al.            Expires 3 January 2025                 [Page 4]
Internet-Draft                MUP Evolution                    July 2024

                             N3             N6
       UE1         AN1/gNB1   |  PSA UPF1    |
   +---------+                |              |
   |App Layer|                |    routing   |
   +---------+                |+--/---+---\-+|
   |PDU Layer|      relay     || PDU  |     ||      PE1
   +---------+ +---/--+--\---+|+------+IP+L2||    +----+--+
   |         | |      |GTP-U |||GTP-U |     ||----+VRF1|  |
   | 5G-AN   | |5G-AN +------+||------+  or ||    +----+  |
   |         | |      |UDP+IP|||UDP+IP|     ||    |VRFn|  |
   | Proto   | |Proto +------+||------+Ether||    +----+--+
   |         | |      |  L2  |||  L2  |     ||   (         )
   | Layers  | |Layers+------+||------+-----+|  (           )
   |         | |      |  L1  |||  L1  |  L1 ||  ( Transport  )
   +---------+ +------+------+|+------+-----+|  (            )
                              |              |  ( Network    )  PE3
                              |              |  (            +--+----+
       UE2         AN2/gNB2   |  PSA UPF2    |  (            |  |VRF1|
   +---------+                |              |  (            |  |----+
   |App Layer|                |    routing   |  (            |  |VRFn|
   +---------+                |+--/---+---\-+|  (            +--+----+
   |PDU Layer|      relay     || PDU  |     ||  (            )
   +---------+ +---/--+--\---+|+------+IP+L2||  (           )
   |         | |      |GTP-U |||GTP-U |     ||   (         )
   | 5G-AN   | |5G-AN +------+||------+  or ||    +----+--+
   |         | |      |UDP+IP|||UDP+IP|     ||----+VRF1|  |
   | Proto   | |Proto +------+||------+Ether||    +----+  |
   |         | |      |  L2  |||  L2  |     ||    |VRFn|  |
   | Layers  | |Layers+------+||------+-----+|    +----+--+
   |         | |      |  L1  |||  L1  |  L1 ||      PE2
   +---------+ +------+------+|+------+-----+|
                              |              |

   The central PSA UPF is no longer needed in this case.  Distributed
   UPF1/UPF2 connect to VRF1 on PE1/PE2 and VRF1 is for the VPN of the
   DN that UE1/UE2 access.  There is also a PE3 for other sites of the
   VPN, which could be wireline sites including sites providing Internet
   access.

   UEs may keep their persistent IP addresses even when they re-anchor
   from one PSA UPF to another.  In that case, for downlink traffic to
   be sent to the right UPF, when a UE anchors at a UPF the UPF
   advertises a host route for the UE and when a UE de-achors from a UPF
   the UPF withdraws the host route.

   While this relies on host routes to direct to-UE traffic to the right
   UPF, it does not introduce additional scaling burden compared to
   centralized PSA UPF model, as the centralized UPFs need to maintain

Zhang, et al.            Expires 3 January 2025                 [Page 5]
Internet-Draft                MUP Evolution                    July 2024

   per-UE forwarding state (in the form of PDRs/FARs) and the number is
   the same as the number of host routes that a hub DN router (e.g. vrf1
   on PE3 for internet access) need to maintain in the distributed PSA
   UPFs model.  Since the host routes may be lighter-weighted than the
   PDRs/FARs, the total amount of state may be actually smaller in the
   distributed model.

   For UE-UE traffic, the distributed PSA UPFs may maintain host routes
   that they learn from each other.  With that the UE-UE traffic may
   take direct UPF-UPF path instead of going through a hub router in the
   DN (equivalent of central UPF).  That is important in LAN-type
   services that require low delay.  Alternatively, the distributed UPFs
   may maintain only a default route pointing to the hub router like PE3
   (besides the host routes for locally anchored UEs).  That way, they
   don't need to maintain many host routes though UPF-UPF traffic has to
   go through the hub router (and that is similar to all traffic going
   through a central PSA UPF).

   Optionally, even the host routes for locally anchored UEs can be
   omitted in the FIB of local UPF.  Traffic among local UEs can be
   simply routed to the hub router following the default route, who will
   then send back to local UPF using VPN tunnels (MPLS or SRv6) that are
   stitched to GTP tunnels for destination UEs.

2.1.1.  Advantages of Distributed PSA UPFs

   Distributed PSA UPFs have the following advantages:

   *  MEC becomes much simpler - no need for centralized PSA UPF plus
      ULCL UPFs, and no need for special procedures for location based
      edge server discovery.

   *  For LAN-type services, UE-UE traffic can be optimized (no need to
      go through centralized PSA UPFs) when UPFs maintain host routes.
      It also allows seamless integration of services across wireline/
      wireless-connected customer sites.

   *  N3/N9 tunneling is simplified

   In particular, there is now only short/simple N3 tunneling between
   AN/gNB and local UPFs in proximity.  Among the distributed UPFs and
   other DN sites, versatile IETF/wireline VPN technologies are used
   instead.  For example:

Zhang, et al.            Expires 3 January 2025                 [Page 6]
Internet-Draft                MUP Evolution                    July 2024

   *  Any tunneling technology - MPLS, SR-MPLS or SRV6 - with any
      traffic engineering/differentiation capabilities can be used.
      Removal of the GTP/UDP header (and IPv4/IPv6 header in case of
      MPLS data plane) brings additional bandwidth savings in the
      transport infrastructure.

   *  Any control plane model for VPN can be used - traditional
      distributed or newer controller based route advertisement.

   In short, the distributed PSA UPFs model achieves "N3/N9/N6 shortcut
   and central UPF bypass", which is desired by many operators.

   Notice that, since UPF has routing functions, depending on the
   capability of a UPF device, it may even be possible for a UPF device
   to act as a VPN PE.  That can be done in one of the two models:

   *  The UPF function and VPN PE function are separate but co-hosted on
      the same device with a logical/internal N6 connection between
      them.

   *  The UPF and VPN PE function are integrated and the PDU sessions
      become VPN PE-CE links.

   The second model is especially useful when a UE is multi-homed to
   different EVPN PEs in case of Ethernet PDU sessions - EVPN's all-
   active multihoming procedures can be utilized.

2.1.2.  Enablers of Distributed PSA UPFs

   To distribute PSA UPFs, if persistent addresses must be used for UEs,
   the SMF must be able to allocate persistent IP addresses from a
   central pool even when a UE re-anchors at different PSA UPFs (e.g.
   due to mobility).  If DHCPv4 is used, either the SMF acts as a
   central DHCP server or it relays DCHP requests to a central DHCP
   server on the DN.

   The distributed PSA UPFs must be able to advertise host routes in the
   DN.  This should not be a problem since a UPF is essentially a router
   in that it routes traffic between DN and UEs (that are connected via
   PDU sessions).

   Notice that, advertising host routes for persistent IP addresses is
   no different from advertising MAC addresses in case of Ethernet PDU
   sessions.

Zhang, et al.            Expires 3 January 2025                 [Page 7]
Internet-Draft                MUP Evolution                    July 2024

2.2.  Alternative Transport Options for 5G

2.2.1.  GTP vs. SRv6 vs. MPLS tunneling

   3GPP specifies that all tunneling (e.g.  N3/N9) use GTP, whose
   encapsulation includes IP header, UDP header and GTP header.  The
   tunnel is between 3GPP NFs (e.g. gNBs and UPFs) over an IP transport,
   and the IP transport may be a VPN over the multi-service transport
   infrastructure of an operator.

   There have been proposals to replace GTP with SRv6 tunnels for the
   following benefits:

   *  Traffic Engineering (TE) and Service Function Chaining (SFC)
      capability provided by SRv6

   *  Bandwidth savings because UDP and GTP headers are no longer needed

   While 3GPP has not adopted the proposal, and GTP can be transported
   over SRv6 (as overlay, instead of SRv6 replacing GTP), some operators
   still prefer to replace GTP with SRv6 "under the hood".  That is,
   while RAN/UPF still use N2/N4 signaling, the actual tunnel are no
   longer GTP but SRv6 based on GTP parameters signaled by N2/N4.  The
   SRv6 tunnel could be between two NFs, or a GW could be attached to an
   NF that still use traditional GTP and the GW will convert GTP to/from
   SRv6.  This is specified in [I-D.ietf-dmm-srv6-mobile-uplane].

   Similarly, if an operator prefers to use MPLS, a GTP tunnel can also
   be replaced with an MPLS PW instead of an SRv6 tunnel.  Compared with
   SRv6, it is even more bandwidth efficient (no need for a minimum
   40-byte IPv6 header) and SR-MPLS can also provide TE/SFC
   capabilities.  This is specified in
   [I-D.zzhang-pals-pw-for-ip-udp-payload].

   Note that, While only IPv6 can scale to the 5G requirements for the
   transport infrastructure, it does not mean MPLS can not be used as
   data plane in the IPv6 network.

2.2.2.  Routing Based UPF-Lite

   Traditionally, a UPF is implemented to follow 3GPP specifications.
   Specifically, N4 signaling is used for SMF to instruct a UPF to set
   up its session state in terms of PDRs/FARs.  On N6 side, a UPF
   receives downlink traffic with destination addresses that are covered
   by the UPF's address range for its anchored UEs.  The packet is
   matched against the installed PDRs and forwarded according to the
   associated FARs.  On N3 side, a UPF decapsulates GTP+UDP+IP header of
   uplink traffic and uses the TEID to identify the DN where inner IP

Zhang, et al.            Expires 3 January 2025                 [Page 8]
Internet-Draft                MUP Evolution                    July 2024

   routing or Ethernet switching is done.

   [I-D.mhkk-dmm-srv6mup-architecture] specifies a new SRv6 based MUP
   architecture.  When it is applied to a 3GPP based mobile
   architecture:

   *  BGP signaling from a MUP Controller replaces N4 signaling from
      SMF.  N4 signaling is still used between the MUP Controller and
      SMF - from SMF's point of view it is just interacting with a
      traditional UPF as usual.

   *  A MUP GW becomes a distributed UPF for uplink traffic.

   *  A MUP PE, which is different from a usually central PSA UPF,
      becomes a UPF for downlink traffic, in that traffic to each UE is
      placed into a different tunnel that is stitched to a GTP tunnel
      for that UE by a MUP GW (no route lookup is needed on the MUP GW
      for the downlink traffic).

   In this approach UE to UE traffic may still optionally go through the
   central PSA UPF.  This is similar to that a hub router may be used in
   Section 2.1.

   This approach can be viewed as a specific way of implementing/
   deploying a subset of functionalities of distributed UPFs discussed
   in Section 2.1, specifically the routing/switching functionalities,
   hence often referred to as UPF-Lite.  It does have the advantage that
   from SMF's point of view, nothing is different from before - both
   from N4 signaling and deployment model point of view.

   While the above is specific to SRv6, a similar MPLS based approach
   will be specified separately for operators who prefer MPLS data
   plane, and it can even be SR-agnostic.

3.  MUP Evolution for 6G

   This section discusses potential MUP evolution in 6G mobile networks.
   It does involve changes in 3GPP architecture and signaling, so the
   purpose is to share the ideas in IETF/wireline community first.  If
   it gains consensus within IETF/wireline community especially among
   mobile operators, then the proposal may be brought to 3GPP community
   for further discussions.

Zhang, et al.            Expires 3 January 2025                 [Page 9]
Internet-Draft                MUP Evolution                    July 2024

3.1.  UPF Distribution and RAN Decomposition

   As described earlier, with 5G, in the opposite direction of UPF
   distribution, some RAN components are becoming centralized as a
   result of the disaggregation and decomposition of baseband processing
   functions.  The AN functionality is now divided into the Radio Unit
   (RU, comprising the antenna and radiating elements), the Distributed
   Unit (DU, comprising the functions for the real time processing of
   the signal), and the Centralized Unit (CU, comprising the remaining
   signal processing functions).  CU is the AN function that handles N3
   GTP-U encapsulation for UpLink (UL) traffic and decapsulation for
   DownLink (DL) traffic.

   The placement of the decomposed CU component can converge with the
   distribution process of the UPF to some optimal and convenient
   location in the network - they become co-located in an edge or far
   edge data center (DC) either with direct/short local connections in
   between or both running as virtual functions on the same compute
   server.

3.2.  Integrated AN/UP Function (ANUP)

   While the AN (CU) and UPF can be co-located, in 5G they are still
   separate functions connected by N3 tunneling over a short/internal
   transport connection.  Routing happens on the UPF between the DN and
   UEs over the N3 tunnels, and relay happens on the AN between the N3
   tunnels and AN protocol stack.

   With AN and UPF functions more and more disaggregated and virtualized
   even in 5G, it is becoming more and more feasible and attractive to
   integrate the AN and UPF functions, eliminating the N3 tunneling and
   the relay on AN entirely.  The combined function is referred to as
   ANUP in this document, which does routing between DN and UEs over the
   AN protocol stack directly:

Zhang, et al.            Expires 3 January 2025                [Page 10]
Internet-Draft                MUP Evolution                    July 2024

                            N6
       UE1          ANUP     |
   +---------+               |
   |App Layer|     routing   |
   +---------+ +--/---+---\-+|
   |PDU Layer| | PDU  |     ||      PE1
   +---------+ +------+IP+L2||    +----+--+
   |         | |      |     ||----+VRF1|  |
   | xG-AN   | |xG-AN +  or ||    +----+  |
   |         | |      |     ||    |VRFn|  |
   | Proto   | |Proto +Ether||    +----+--+
   |         | |      |     ||   (         )
   | Layers  | |Layers+-----+|  (           )
   |         | |      |  L1 ||  ( Transport  )
   +---------+ +------+-----+|  (            )
                             |  ( Network    )  PE3
                             |  (            +--+----+
       UE2          ANUP     |  (            |  |VRF1|
   +---------+               |  (            |  |----+
   |App Layer|     routing   |  (            |  |VRFn|
   +---------+ +--/---+---\-+|  (            +--+----+
   |PDU Layer| | PDU  |     ||  (            )
   +---------+ +------+IP+L2||  (           )
   |         | |      |     ||   (         )
   | xG-AN   | |xG-AN +  or ||    +----+--+
   |         | |      |     ||----+VRF1|  |
   | Proto   | |Proto +Ether||    +----+  |
   |         | |      |     ||    |VRFn|  |
   | Layers  | |Layers+-----+|    +----+--+
   |         | |      |  L1 ||      PE2
   +---------+ +------+-----+|
                             |

   With this architecture, 3GPP and IETF technologies are applied where
   they are best applicable: 3GPP technologies responsible for radio
   access and IETF technologies for the rest.  As IETF technologies
   continue to evolve, they can be automatically applied in mobile
   networks without any changes in 3GPP architecture/specification.

   One way to view this is that the ANUP is a router/switch with
   wireless and wired interfaces and it routes/switches traffic among
   those interfaces.  The wireless interface is established by 3GPP
   technologies (just like an Ethernet interface is established by IEEE
   technologies) and the routing/switching function follows IETF/IEEE
   standards.

   Some advantages of this new architecture include:

Zhang, et al.            Expires 3 January 2025                [Page 11]
Internet-Draft                MUP Evolution                    July 2024

   *  5G-LAN and MEC become transparent applications that wireline
      networks have been supporting (PDU sessions terminate into the
      closest ANUP and routed/switched to various DNs).

   *  MBS becomes very simple – the ANUP gets the multicast traffic in
      the DN and then use either shared radio bearer or individual
      bearers to send to interested UEs.

   *  Simplified signaling - instead of seven-steps of separate N2/N4
      signaling from separate AMF/SMF to separate AN/UPF and N11
      signaling between AMF and SMF to set up the N3 tunneling for a PDU
      session, a two-step signaling between a new single control plane
      entity to the single integrated ANUP is enough - see Section 4.2
      for details.

   *  Simplified/Optimized data plane - AN-UPF connection and GTP-U
      encapsulation/decapsulation are not needed anymore.  This can
      significantly improve throughput, especially when compared to AN/
      UPF functions running on servers.

   *  Natural local break-out in traffic forwarding, by allowing the
      more efficient routing/switching of traffic according to its
      destination.

   *  Any kind of tunnels can be used for the DN VPN, whether it is MPLS
      or SRv6, w/o the overhead of UDP/GTP encapsulation compared to GTP
      tunneling.  Network slicing and QoS functions are still supported
      (even with current GTP tunneling the transport network need to
      instantiate slices and implement QoS for N3/N9 tunnels as well).

   Because the ANUP already implement the routing/switching functions,
   even the PE functions (for the DN VPN) could be optionally integrated
   into it, further streamlining end-to-end communication by reducing
   NFs and connections between them.  While integrating PE function is
   optional, it is desired and today's AN can be already considered as a
   PE (Section 4.6).

3.3.  ANUP Potential Use-case: 5G-A Satellite Services

   The 3GPP SA2 working group has several projects to study &
   standardize the 5G advanced services whose wireless connectivities
   are provided via satellite networks.  These projects cover various
   aspects of satellite services, e.g., one focusing on the support of
   wireless access considering the satellite-based discontinuous
   coverage, while the 2nd-one studying the service requirements via
   satellite backhaul taking into account 5G new capabilities.

Zhang, et al.            Expires 3 January 2025                [Page 12]
Internet-Draft                MUP Evolution                    July 2024

   Still, there is a 3rd project exploring the scenario that a gNB will
   be on board satellite while the corresponding anchor UPF may (i.e.,
   on-board a satellite) or may not (i.e., on the ground).  Evidently,
   this is a very challenging case that requires the seamless
   integration among AN (i.e., gNB), UPF & 5GS.

   An on-board UPF might not share the same underlaying satellite as the
   matching gNB.  For this case, thanks to the everlasting movement of
   (LEO-based) satellites, the highly mobile satellite constellation
   network will significantly impact the signaling performance between
   the gNB and the UPF.  Therefore, some measures must be adopted to
   reduce the signalling impact to the AN/RAT, to the UP (UPF) and to
   the CP (5GS).

   Further, a latest 5G-service, the satellite-based store & foward
   (S&F) feature for (on-ground) UEs via intermittent (satellite)
   service-link and/or feed-link connectivities [_3GPP-23.700-29], has
   embraced quite a few proposals in which the AN (i.e. gNB), the CP
   (i.e., 5GS/EPC) and the UP (i.e., UPF/S-GW,P-GW) could be either
   deployed together (being less challenging) or distributed (being much
   more complicated).  In some proposal(s), even an individual CP and/or
   UP NF (network function) might be decomposed into multiple (sub)-
   instances to accomodate the complexity of distributedness.  However,
   if we plug into the above S&F service requirements into the
   integrated ANUP architecture, there is no more implication of the
   distribution of gNB and UPF.  The complexity of both the CP signaling
   exchanges and the UP data transport will be greatly relieved.

   Given the ubiquitous discussion of the satellite communication for
   5G, beyond-5G and imminent 6G, we do believe our proposal ANUP will
   benefit materially both the IETF and the 3GPP communities.

3.4.  ANUP-like Feature in 4G: Local IP Access (LIPA)

   While Section 3.2 proposed the integrated AN and UPF, or ANUP, for
   the evolution of 6G MUP, the 3GPP specification 23.401 [_3GPP-23.401]
   has already standardized an ANUP-like function, i.e., the Local IP
   Access or LIPA, that fundamentally integrates together the 4G RAN
   entity 'HeNB or Home eNodeB' and the traffic switching gateway 'L-GW
   or Local Gateway'.

Zhang, et al.            Expires 3 January 2025                [Page 13]
Internet-Draft                MUP Evolution                    July 2024

        LIPA @ DN            DN: Data Network
       ^     |               UP: User Plane
       |     |SGi
       |  +--+---+    S5
       |  | L-GW |-----------\
       |  +------+   S1-U     \+-----+  S5  +------+ SGi  /----\
       |  | HeNB +-------------+ SGW +------+ P-GW +-----<  DN  >
       |  +--+---+\            +-----+      +------+      \----/
     UP|     |     \S1-MME    /S11
       |     |Uu    \        /
       |  +-----+    +------+
       |  |     |    |  MME |
       +--+ UE  |    +------+
          +-----+

   The above figure shows the LIPA architecture.  It enables a UE (on
   the bottom-left) that can connect via a HeNB to access the DN without
   the user plane traversing the mobile operator's network (e.g.,
   SGW->P-GW).  The LIPA feature is achieved using a L-GW (on the top-
   left) that is collocated with the HeNB.  The functionalities of HeNB
   and L-GW are integrated together to provide the direct User-Plane
   (UP) path between the HeNB and the L-GW.  There is NO reference
   interface between HeNB and L-GW.  That is, they are truly an
   integrated entity.

   As of now, while the LIPA feature has not yet been deployed
   extensively by MNO's, it does give somewhat promising indicator that
   the ANUP-like integration solution has been studied before by 3GPP
   and it is worthy of the continuous exploration.

4.  ANUP: Advanced Technical Considerations

   Various considerations/concerns were brought up during the
   discussions of the ANUP proposal.  They are documented in the
   following sections.

4.1.  Separate AN/UP Functions

   There are still cases where separate AN/UP functions are desired/
   required:

   *  An MNO may want to deploy one UPF for a cluster of ANs in
      proximity in some scenarios/locations

   *  An MNO may support MVNOs who have their own UP functions but make
      use of the hosting MNO's ANs

   *  Home Routed roaming requires separate HPLMN UPs and VPLMN ANs

Zhang, et al.            Expires 3 January 2025                [Page 14]
Internet-Draft                MUP Evolution                    July 2024

   Therefore, the integration does not have to be always used.  Rather,
   it is "integration when desired and feasible, separation when
   necessary".

   Note that, the same ANUP can handle both situations - some PDU
   sessions may be tunneled to a separate UPF while other sessions are
   terminated and then traffic is routed/switched to either local DN or
   remote/central DN.

   This is also the basis of interworking between 5G and xG:

   *  A 5G AN can have N3 tunneling to an xG UPF

   *  An xG ANUP can have N3 tunneling to a 5G/xG UPF

4.2.  Simplified/reduced Signaling and optimized data plane

   One may ask why bother with integration when it is still needed to
   support separate AN and UPF anyway.

   When AN and UPF are separate, to set up the N3 tunnel the following
   seven steps are needed, involving four NFs and three Nx interfaces:

   1.  SMF sends request to UPF (N4)

   2.  UPF responds with UPF-TEID (N4)

   3.  SMF passes <UPF, UPF-TEID> to AMF (N11)

   4.  AMF sends request to gNB, passing <UPF, UPF-TEID> (N2)

   5.  gNB responds with AN-TEID (N2)

   6.  AMF passes <AN, AN-TEID> to SMF (N11)

   7.  SMF sends <AN, AN-TEID> to UPF (N4)

   With integrated ANUP, there is no need for N3 tunnel anymore.  A new
   control plane NF only needs to tell the ANUP which DN that PDU
   session belongs to.

   Additionally, the N3 tunnel is maintained by periodical signaling
   refreshes - otherwise timeout will happen.  This causes significant
   control plane load on the NFs and interfaces, which no longer exists
   with ANUP since N3 tunneling is eliminated.

Zhang, et al.            Expires 3 January 2025                [Page 15]
Internet-Draft                MUP Evolution                    July 2024

   As mentioned before, with ANUP the AN-UPF connection and GTP-U
   encapsulation/decapsulation are not needed anymore.  This can
   significantly improve performance/throughput, especially when
   compared to AN/UPF functions running on servers.

4.3.  Microservice architecture

   One may argue that the integration of AN and UP functions are against
   the microservice trend.

   The following is a verbatim quote from https://microservices.io/:

     Microservices - also known as the microservice architecture -
     is an architectural style that structures an application as a
     collection of services that are:

     - Highly maintainable and testable
     - Loosely coupled
     - Independently deployable
     - Organized around business capabilities
     - Owned by a small team
     - The microservice architecture enables the rapid, frequent
       and reliable delivery of large, complex applications.
       It also enables an organization to evolve its technology stack.

   The counter argument is that microservice is about decomposing
   complex "applications".  ANUP is about integrating co-located and
   mature data plane entities to streamline and optimize forwarding.  It
   has real and significant benefits of simplified signaling and
   optimized data plane - it does not make sense to force microservice
   here for data plane.  Note that microservices can still be utilized
   in the control plane for ANUP.

4.4.  Increased burden on previously simple AN

   One may think that the AN only needed to do simple traffic stitching
   functions while now the ANUP has added UPF burden.  However, the main
   use case of ANUP is where the AN and UPF are co-located even if they
   are separate functions.  Therefore, the ANUP only absorbs the
   whatever functionalities that the separate UPF at the same site need
   to do anyway, with reduced signaling and data plane handling - the
   overall processing at the site actually decreases.  While a
   particular ANUP now has more processing to do, it can offload some
   sessions to additional ANUPs that are now made possible because of
   removal of separate UPFs at the same site.

Zhang, et al.            Expires 3 January 2025                [Page 16]
Internet-Draft                MUP Evolution                    July 2024

   This may also make it easier to allocate resources at the edge DC.
   Previously, an operator needs to consider how much resources to
   allocate for the separate UPFs and assign which sessions to which
   UPFs.  Now it simply is to decide which sessions are assigned to
   which ANUP (just like to decide which sessions are assigned to which
   AN).

   In addition, there are some similar or even overlapping
   functionalities in the current UPF and AN in 5GS; in integrated ANUP
   these functions can be re-designed.  For example for a rate control
   enforcement, UPF supports the enforcement of the aggregated MBR for
   the session (Session-AMBR) in UL/DL directions, while AN can enforce
   the aggregated MBR for the UE (UE-AMBR) in UL/DL directions.  Both
   UPF and AN support the enforcement of the QoS Flow MBR (MFBR) and GBR
   (GFBR) in both UL/DL directions (for the GBR flows), while AN can in
   additon to ensure the UE-Slice-MBR is not exceeded in UL/DL
   directions.  With ANUP, these previously separate functions may be
   optimized now that they are in the same entity.

4.5.  Use of ULCL I-UPF for MEC Purpose

   Notice that the ANUP is to integrate AN and distributed UPF that are
   co-located in edge DCs, and one use case of distributed UPF (in those
   edge DCs) is MEC.  UpLink CLassifier Intermediate UPF (ULCL I-UPF) is
   an existing way to achieve local breakout routing for MEC purpose,
   but it is not an optimized/elegant solution compared to ANUP.

   The ULCL I-UPF is placed between an AN and a central UPF as a
   filtering device.  While called an UPF it is different from a typical
   UPF - It inspects _all_ GTP-U UL traffic, and based on N4 signaling
   from SMF certain traffic is intercepted and forwarded to local DN.
   This places additional control plane burden on SMF in addition to the
   need of the special traffic-filtering UPF.  For example, the SMF will
   need to know which traffic (to some particular destination address)
   is to be intercepted.

   For comparison, with ANUP there is no need for the additional special
   UPF and corresponding N4 signaling at all.  Everything is standard
   routing/filtering w/o relying on SMF to determine which traffic is
   delivered locally:

   *  For some PDU sessions, all traffic may be tunneled to a separate
      UPF.

   *  For a particular PDU session, some traffic may be delivered
      locally while some other delivered to the central/remote DN all
      based on routing/filtering in the DN.

Zhang, et al.            Expires 3 January 2025                [Page 17]
Internet-Draft                MUP Evolution                    July 2024

4.6.  VPN PE Function in AN/ANUP

   As previously mentioned, the ANUP can optionally have the VPN PE
   function integrated, instead of being a standalone CE device for the
   VPN for the DN.

   While optional, it is a desired optimization.  Moreover, even the
   separate AN itself can be considered as a spoke PE for a hub-and-
   spoke VPN [RFC7024] for the DN.

   Consider a hub-and-spoke VPN outside the mobile network context:

   *  A spoke PE only imports a default route from a hub PE and
      therefore sends all traffic from its CEs to the hub PE

   *  A hub PE imports routes from all PEs and sends traffic to
      appropriate PEs or its CEs, whether the traffic is from a local CE
      or another PE

   Additionally, consider that a spoke PE advertise different per-prefix
   (vs. per VRF) VPN labels.  When it receives traffic with a per-prefix
   label, it can send traffic to a local CE purely based on the label
   without having to do a route lookup in the VRF.

   Now consider the AN and the central UPF in a mobile network.
   Effectively the AN is a spoke PE and the central UPF is a hub PE for
   the DN:

   *  The GTP-U tunnel corresponds to the MPLS label stack.

   *  For UL traffic, there is no need for route lookup on the AN
      because all is to be tunneled to the UPF.  The UPF TEID is used by
      the UPF to determine which DN the traffic belongs to, just like
      how a VPN label is used to determine VPN the traffic belongs to.

   *  For DL traffic, the UPF does a lookup based on the destination
      address (e.g., that of a UE) and a corresponding GTP-U tunnel is
      used to send traffic to an AN.  When traffic arrives on the AN,
      the per-UE TEID allows traffic to be relayed to the UE without a
      route lookup.

   In other words, the separate ANs and UPF form a hub-and-spoke VPN for
   the DN with per-prefix "labels", though no VRF is present on the ANs
   because there is no need for route lookup at all.

   For ANUP with VPN PE function integrated, the only difference is the
   addition of VRF in the AN.  That's so that some sessions will be
   locally terminated and traffic is locally routed.  For DL traffic,

Zhang, et al.            Expires 3 January 2025                [Page 18]
Internet-Draft                MUP Evolution                    July 2024

   the ANUP can either advertise per-VRF label (or SID in case of SR)
   and do a lookup for DL traffic, or advertises per-prefix/UE label (or
   SID in case of SR) - just like per-UE TEID - so that it does not to
   do a lookup before sending traffic to a UE.

4.7.  QoS Handling

   With separate AN and UPF, the QoS handling happens in the following
   segments:

   *  Between UE and AN over the air interface

   *  Between AN and UPF over the N3 tunnel, which can be:

      -  through a transport network, or

      -  through a local/internal link in co-location case

   The QoS over the air interface is the same for both AN and ANUP
   cases.

   For the trivial QoS previously over N3 tunnel through a local/
   internal link in co-location case, it is now completely eliminated
   with ANUP.

   The QoS over N3 tunnel through a transport network is realized
   through QoS mechanisms in the transport network.  With ANUP, it's
   likely that similar QoS is needed between the ANUP and a hub router
   in the DN, which is a VPN over the same transport network.
   Therefore, it is similar to the QoS over N3 tunnel - only that now it
   is QoS over VPN tunnel and realized through QoS mechanisms in the
   transport network.

   A central UPF may have rate limiting for N3 tunnels so that each PDU
   session's DL traffic is limited and the AN won't be overwhelmed by DL
   traffic.  With distributed UPF (whether integrated into AN or not),
   the routes advertised to the hub DN router may carry QoS information
   like rate limiting parameters, so that the hub DN router can do rate
   limiting.

4.8.  NAT

   Addresses assigned to UEs may be from a private address space and NAT
   is needed between the private space and public space.  In case of
   central UPFs, the NAT can be done on a central UPF (though NAT is
   still a logically separate function) or by a separate NAT Gateway
   (GW) connected to the central UPF.

Zhang, et al.            Expires 3 January 2025                [Page 19]
Internet-Draft                MUP Evolution                    July 2024

   With distributed UPFs (whether it is a separate UPF or an integrated
   ANUP), NAT can be done by a central NAT GW connected to the hub
   router, just like a NAT GW on or next to the previously central UPF.

   A large operator may have multiple central UPFs for different
   regions, and the regions may have overlapping private address spaces.
   Each UPF will have its own NAT GW, and UE to UE traffic across
   regions will go throw two NAT GWs.  With distributed UPFs, each
   region will have its own hub router with its own NAT GW, and UE to UE
   traffic across regions will go through two NAT GWs and two hub
   routers.

5.  ANUP Implications: IETF to 3GPP

5.1.  User-plane/UP vs. Control-plane/CP

   Stepping from the IETF perspective, this draft centers around the
   ANUP innovations along with its implications to 3GPP SDO.  Because
   IETF focuses more on the connectivity of transport network (TN), this
   draft addresses mainly the mobile user plane or UP, e.g., re-design
   of the hub-and-spoke VPN settings different from those over the
   current separate AN & UPF architecture, alternative UP protocol(s) to
   GTP-U tunnel between AN and UPF (in the TN domain), etc.  However,
   while this draft does not limit the discussions only to UP, but given
   the complexities of the 5G CP and the on-going discussions of the
   evolution of the 6G system architecture, the draft does not dive into
   the CP of the mobile wireless domain.  All those mobility related CP
   details, e.g., RM, MM, SM, paging, handover, QoS settings, etc., are
   left to the 3GPP's further exploration & refinement.  Certainly, the
   results from the UP investigation would benefit the CP design in 6G
   evolution.

5.2.  Impacts & Intentions to 5G/6G CP

   As set forth at the beginning, this draft does not intend to do the
   3GPP 5G/6G work in IETF.  In comparison, it actually acknowledges the
   principle that the complete studies should be done in the 3GPP SDO.
   The I.D. has argued that the innovative ANUP architecture does have
   certain advantageous impacts to the current 5GS CP (and likely to the
   future 6G evolution).  But, given the complexity of 5GS, any ANUP
   related achievement in the IETF domain shall only serve as a
   reference to the 3GPP, possibly via the liaison exchange between the
   two SDO's.

6.  Security Considerations

   To be provided.

Zhang, et al.            Expires 3 January 2025                [Page 20]
Internet-Draft                MUP Evolution                    July 2024

7.  Acknowledgements

   The authors thank Arda Akman, Constantine Polychronopoulos, Sandeep
   Patel and Shraman Adhikary for their review, comments and suggestions
   to make this document and solution more complete.

8.  Informative References

   [I-D.ietf-dmm-srv6-mobile-uplane]
              Matsushima, S., Filsfils, C., Kohno, M., Camarillo, P.,
              and D. Voyer, "Segment Routing IPv6 for Mobile User
              Plane", Work in Progress, Internet-Draft, draft-ietf-dmm-
              srv6-mobile-uplane-24, 17 January 2023,
              <https://datatracker.ietf.org/doc/html/draft-ietf-dmm-
              srv6-mobile-uplane-24>.

   [I-D.mhkk-dmm-srv6mup-architecture]
              Matsushima, S., Horiba, K., Khan, A., Kawakami, Y.,
              Murakami, T., Patel, K., Kohno, M., Kamata, T., Camarillo,
              P., Horn, J., Voyer, D., Zadok, S., Meilik, I., Agrawal,
              A., and K. Perumal, "Mobile User Plane Architecture using
              Segment Routing for Distributed Mobility Management", Work
              in Progress, Internet-Draft, draft-mhkk-dmm-srv6mup-
              architecture-06, 23 October 2023,
              <https://datatracker.ietf.org/doc/html/draft-mhkk-dmm-
              srv6mup-architecture-06>.

   [I-D.zzhang-dmm-5g-distributed-upf]
              Zhang, Z. J., Patel, K., Jiang, T., and L. M. Contreras,
              "5G Distributed UPFs", Work in Progress, Internet-Draft,
              draft-zzhang-dmm-5g-distributed-upf-01, 11 July 2022,
              <https://datatracker.ietf.org/doc/html/draft-zzhang-dmm-
              5g-distributed-upf-01>.

   [I-D.zzhang-pals-pw-for-ip-udp-payload]
              Zhang, Z. J. and K. Patel, "PW for IP/UDP Payload without
              IP/UDP Headers", Work in Progress, Internet-Draft, draft-
              zzhang-pals-pw-for-ip-udp-payload-01, 4 March 2022,
              <https://datatracker.ietf.org/doc/html/draft-zzhang-pals-
              pw-for-ip-udp-payload-01>.

   [ORAN-Arch]
              "O-RAN Architecture Description, V06.00", 2022.

   [RFC7024]  Jeng, H., Uttaro, J., Jalil, L., Decraene, B., Rekhter,
              Y., and R. Aggarwal, "Virtual Hub-and-Spoke in BGP/MPLS
              VPNs", RFC 7024, DOI 10.17487/RFC7024, October 2013,
              <https://www.rfc-editor.org/info/rfc7024>.

Zhang, et al.            Expires 3 January 2025                [Page 21]
Internet-Draft                MUP Evolution                    July 2024

   [_3GPP-23.401]
              "General Packet Radio Service (GPRS) enhancements for
              Evolved Universal Terrestrial Radio Access Network
              (E-UTRAN) access, V18.2.0", June 2023.

   [_3GPP-23.501]
              "System architecture for the 5G System (5GS), V18.4.0",
              December 2023.

   [_3GPP-23.700-29]
              "Study on integration of satellite components in the 5G
              architecture; Phase 3, V19.2.0", February 2024.

Authors' Addresses

   Zhaohui Zhang
   Juniper Networks
   Email: zzhang@juniper.net

   Keyur Patel
   Arrcus
   Email: keyur@arrcus.com

   Luis M. Contreras
   Telefonica
   Email: luismiguel.contrerasmurillo@telefonica.com

   Kashif Islam
   Redhat
   Email: kislam@redhat.com

   Jari Mutikainen
   NTT Docomo
   Email: mutikainen@docomolab-euro.com

   Tianji Jiang
   China Mobile
   Email: tianjijiang@chinamobile.com

   Luay Jalil
   Verizon
   Email: luay.jalil@verizon.com

Zhang, et al.            Expires 3 January 2025                [Page 22]
Internet-Draft                MUP Evolution                    July 2024

   Ori Prio Sejati
   XL Axiata
   Email: ORIP@xl.co.id

   Shay Zadok
   Broadcom
   Email: shay.zadok@broadcom.com

Zhang, et al.            Expires 3 January 2025                [Page 23]