dmm                                                             Z. Zhang
Internet-Draft                                          Juniper Networks
Intended status: Informational                                  K. Patel
Expires: 7 September 2022                                         Arrcus
                                                                T. Jiang
                                                            China Mobile
                                                            6 March 2022

                          5G Distributed UPFs


   This document describes evolution of mobile user plane in 5G,
   including distributed UPFs and alternative user plane implementations
   that some vendors/operators are pushing without changing 3GPP
   architecture/signaling.  This also sets the stage for discussions in
   a companion document about potentially integrating UPF and Acess Node
   (AN) in a future generation (xG) of mobile network.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 7 September 2022.

Copyright Notice

   Copyright (c) 2022 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components

Zhang, et al.           Expires 7 September 2022                [Page 1]

Internet-Draft             5G Distributed UPFs                March 2022

   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Current User Plane in 5G  . . . . . . . . . . . . . . . . . .   2
   2.  MUP Evolution in 5G: Distributed UPFs . . . . . . . . . . . .   3
     2.1.  Advantages of Distributed PSA UPFs  . . . . . . . . . . .   5
     2.2.  Enablers of Distributed PSA UPFs  . . . . . . . . . . . .   6
   3.  MUP Evolution in 5G: Alternative Implementation Options . . .   7
     3.1.  GTP vs. SRv6 vs. MPLS tunneling . . . . . . . . . . . . .   7
     3.2.  Routing Based UPF . . . . . . . . . . . . . . . . . . . .   7
   4.  Security Considerations . . . . . . . . . . . . . . . . . . .   8
   5.  Informative References  . . . . . . . . . . . . . . . . . . .   8
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   9

1.  Current User Plane in 5G

   Mobile User Plane (MUP) in 5G [_3GPP-23.501] has two distinct parts:
   the Access Network part between UE and AN/gNB, and the Core Network
   part between AN/gNB and UPF.

                              N3              N9             N6
       UE          AN(gNB)    |    I-UPF      |  PSA UPF     |
   +---------+                |               |              |
   |App Layer|                |               |    routing   |    __
   +---------+                |               |+--/---+---\-+|   (  )
   |PDU Layer|      relay     |    relay      || PDU  |     ||  (    )
   +---------+ +---/--+--\---+|+---/--+--\---+|+------+IP+L2|| (      )
   |         | |      |GTP-U |||GTP-U |GTP-U |||GTP-U |     || (  DN  )
   | 5G-AN   | |5G-AN +------+||------+------+||------+  or || (      )
   |         | |      |UDP+IP|||UDP+IP|UDP+IP|||UDP+IP|     ||  (    )
   | Proto   | |Proto +------+||------+------+||------+Ether||   (  )
   |         | |      |  L2  |||  L2  |  L2  |||  L2  |     ||    --
   | Layers  | |Layers+------+||------+------+||------+-----+|
   |         | |      |  L1  |||  L1  |  L1  |||  L1  |  L1 ||
   +---------+ +------+------+|+------+------+|+------+-----+|
                              |               |              |

Zhang, et al.           Expires 7 September 2022                [Page 2]

Internet-Draft             5G Distributed UPFs                March 2022

   For the core network (CN) part, N3 interface extends the PDU layer
   from AN/gNB towards the PSA UPF, optionally through I-UPFs and in
   that case N9 interface is used between I-UPF and PSA UPF.
   Traditionally, UPFs are deployed at central locations and the N3/N9
   tunnels extend the PDU layer to them.  The N3/N9 interface uses GTP-U
   tunnels that are typically over a VPN over a transport
   infrastructure.  While N6 is a 3GPP defined interface, it is for
   reference only and there is no tunneling or specification involved -
   it is simply a direct IP (in case of IP PDU session) or Ethernet (in
   case of Ethernet PDU session) connection to the DN.

   At the AN/gNB, relay is done between the radio layer and the GTP-U
   layer.  At the PSA UPF, routing/switching is done for IP/Ethernet
   before GTP-U encapsulation (for downlink traffic) or after GTP-U
   decapsulation (for uplink traffic).

2.  MUP Evolution in 5G: Distributed UPFs

   With MEC, ULCL UPFs are deployed closer to gNBs, while centralized
   PSA UPFs are still used to provide persistent IP addresses to UEs.

   In fact, even PSA UPFs could be distributed closer to gNBs and then
   the N3 interface becomes very simple - over a direct or short
   transport connection between gNB and UPF (or even an internal
   connection if the gNB and UPF are hosted on the same server).  On the
   other hand, since the UPF to DN connection is direct, the DN becomes
   a VPN (e.g., IP VPN in case of IP PDU sessions or EVPN in case of
   Ethernet PDU sessions) over a transport infrastructure, most likely
   the same transport infrastructure for the VPN supporting the N3/N9
   tunneling in centralized PSA UPF case, as shown in the following

Zhang, et al.           Expires 7 September 2022                [Page 3]

Internet-Draft             5G Distributed UPFs                March 2022

                             N3             N6
       UE1         AN1/gNB1   |  PSA UPF1    |
   +---------+                |              |
   |App Layer|                |    routing   |
   +---------+                |+--/---+---\-+|
   |PDU Layer|      relay     || PDU  |     ||      PE1
   +---------+ +---/--+--\---+|+------+IP+L2||    +----+--+
   |         | |      |GTP-U |||GTP-U |     ||----+VRF1|  |
   | 5G-AN   | |5G-AN +------+||------+  or ||    +----+  |
   |         | |      |UDP+IP|||UDP+IP|     ||    |VRFn|  |
   | Proto   | |Proto +------+||------+Ether||    +----+--+
   |         | |      |  L2  |||  L2  |     ||   (         )
   | Layers  | |Layers+------+||------+-----+|  (           )
   |         | |      |  L1  |||  L1  |  L1 ||  ( Transport  )
   +---------+ +------+------+|+------+-----+|  (            )
                              |              |  ( Network    )  PE3
                              |              |  (            +--+----+
       UE2         AN2/gNB2   |  PSA UPF2    |  (            |  |VRF1|
   +---------+                |              |  (            |  |----+
   |App Layer|                |    routing   |  (            |  |VRFn|
   +---------+                |+--/---+---\-+|  (            +--+----+
   |PDU Layer|      relay     || PDU  |     ||  (            )
   +---------+ +---/--+--\---+|+------+IP+L2||  (           )
   |         | |      |GTP-U |||GTP-U |     ||   (         )
   | 5G-AN   | |5G-AN +------+||------+  or ||    +----+--+
   |         | |      |UDP+IP|||UDP+IP|     ||----+VRF1|  |
   | Proto   | |Proto +------+||------+Ether||    +----+  |
   |         | |      |  L2  |||  L2  |     ||    |VRFn|  |
   | Layers  | |Layers+------+||------+-----+|    +----+--+
   |         | |      |  L1  |||  L1  |  L1 ||      PE2
   +---------+ +------+------+|+------+-----+|
                              |              |

   The central PSA UPF is no longer needed in this case.  Distributed
   UPF1/UPF2 connect to VRF1 on PE1/PE2 and VRF1 is for the VPN of the
   DN that UE1/UE2 access.  There is also a PE3 for other sites of the
   VPN, which could be wireline sites including sites providing Internet

   UEs may keep their persistent IP addresses even when they re-anchor
   from one PSA UPF to another.  In that case, for downlink traffic to
   be sent to the right UPF, when a UE anchors at a UPF the UPF
   advertises a host route for the UE and when a UE de-achors from a UPF
   the UPF withdraws the host route.

   While this relies on host routes to direct to-UE traffic to the right
   UPF, it does not introduce additional scaling burden compared to
   centralized PSA UPF model, as the centralized UPFs need to maintain

Zhang, et al.           Expires 7 September 2022                [Page 4]

Internet-Draft             5G Distributed UPFs                March 2022

   per-UE forwarding state (in the form of PDRs/FARs) and the number is
   the same as the number of host routes that a hub DN router (e.g. vrf1
   on PE3 for internet access) need to maintain in the distributed PSA
   UPFs model.  Since the host routes may be lighter-weighted than the
   PDRs/FARs, the total amount of state may be actually smaller in the
   distributed model.

   For UE-UE traffic, the distributed PSA UPFs may maintain host routes
   that they learn from each other.  With that the UE-UE traffic may
   take direct UPF-UPF path instead of going through a hub router in the
   DN (equivalent of central UPF).  That is important in LAN-type
   services that require low delay.  Alternatively, the distributed UPFs
   may maintain only a default route pointing to the hub router like PE3
   (besides the host routes for locally anchored UEs).  That way, they
   don't need to maintain many host routes though UPF-UPF traffic has to
   go through the hub router (and that is similar to all traffic going
   through a central PSA UPF).

   Optionally, even the host routes for locally anchored UEs can be
   omitted in the FIB of local UPF.  Traffic among local UEs can be
   simply routed to the hub router following the default route, who will
   then send back to local UPF using VPN tunnels (MPLS or SRv6) that are
   stitched to GTP tunnels for destination UEs.

2.1.  Advantages of Distributed PSA UPFs

   Distributed PSA UPFs have the following advantages:

   *  MEC becomes much simpler - no need for centralized PSA UPF plus
      ULCL UPFs, and no need for special procedures for location based
      edge server discovery.

   *  For LAN-type services, UE-UE traffic can be optimized (no need to
      go through centralized PSA UPFs) when UPFs maintain host routes.
      It also allows seamless integration of services across wireline/
      wireless-connected customer sites.

   *  N3/N9 tunneling is simplified

   In particular, there is now only short/simple N3 tunneling between
   AN/gNB and local UPFs in proximity.  Among the distributed UPFs and
   other DN sites, versatile IETF/wireline VPN technologies are used
   instead.  For example:

Zhang, et al.           Expires 7 September 2022                [Page 5]

Internet-Draft             5G Distributed UPFs                March 2022

   *  Any tunneling technology - MPLS, SR-MPLS or SRV6 - with any
      traffic engineering/differentiation capabilities can be used.
      Removal of the GTP/UDP header (and IPv4/IPv6 header in case of
      MPLS data plane) brings additional bandwidth savings in the
      transport infrastructure.

   *  Any control plane model for VPN can be used - traditional
      distributed or newer controller based route advertisement.

   In short, the distributed PSA UPFs model achieves "N3/N9/N6 shortcut
   and central UPF bypass", which is desired by many operators.

   Notice that, since UPF has routing functions, depending on the
   capability of a UPF device, it may even be possible for a UPF device
   to act as a VPN PE.  That can be done in one of the two models:

   *  The UPF function and VPN PE function are separate but co-hosted on
      the same device with a logical/internal N6 connection between

   *  The UPF and VPN PE function are integrated and the PDU sessions
      become VPN PE-CE links.

   The second model is especially useful when a UE is multi-homed to
   different EVPN PEs in case of Ethernet PDU sessions - EVPN's all-
   active multihoming procedures can be utilized.

2.2.  Enablers of Distributed PSA UPFs

   To distribute PSA UPFs, if persistent addresses must be used for UEs,
   the SMF must be able to allocate persistent IP addresses from a
   central pool even when a UE re-anchors at different PSA UPFs (e.g.
   due to mobility).  If DHCPv4 is used, either the SMF acts as a
   central DHCP server or it relays DCHP requests to a central DHCP
   server on the DN.

   The distributed PSA UPFs must be able to advertise host routes in the
   DN.  This should not be a problem since a UPF is essentially a router
   in that it routes traffic between DN and UEs (that are connected via
   PDU sessions).

   Notice that, advertising host routes for persistent IP addresses is
   no different from advertising MAC addresses in case of Ethernet PDU

Zhang, et al.           Expires 7 September 2022                [Page 6]

Internet-Draft             5G Distributed UPFs                March 2022

3.  MUP Evolution in 5G: Alternative Implementation Options

3.1.  GTP vs. SRv6 vs. MPLS tunneling

   3GPP specifies that all tunneling (e.g.  N3/N9) use GTP, whose
   encapsulation includes IP header, UDP header and GTP header.  The
   tunnel is between 3GPP NFs (e.g. gNBs and UPFs) over an IP transport,
   and the IP transport may be a VPN over the multi-service transport
   infrastructure of an operator.

   There have been proposals to replace GTP with SRv6 tunnels for the
   following benefits:

   *  Traffic Engineering (TE) and Service Function Chaining (SFC)
      capability provided by SRv6

   *  Bandwidth savings because UDP and GTP headers are no longer needed

   While 3GPP has not adopted the proposal, and GTP can be transported
   over SRv6 (as overlay, instead of SRv6 replacing GTP), some operators
   still prefer to replace GTP with SRv6 "under the hood".  That is,
   while RAN/UPF still use N2/N4 signaling, the actual tunnel are no
   longer GTP but SRv6 based on GTP parameters signaled by N2/N4.  The
   SRv6 tunnel could be between two NFs, or a GW could be attached to an
   NF that still use traditional GTP and the GW will convert GTP to/from
   SRv6.  This is specified in [I-D.ietf-dmm-srv6-mobile-uplane].

   Similarly, if an operator prefers to use MPLS, a GTP tunnel can also
   be replaced with an MPLS PW instead of an SRv6 tunnel.  Compared with
   SRv6, it is even more bandwidth efficient (no need for a minimum
   40-byte IPv6 header) and SR-MPLS can also provide TE/SFC
   capabilities.  This is specified in

   Note that, While only IPv6 can scale to the 5G requirements for the
   transport infrastructure, it does not mean MPLS can not be used as
   data plane in the IPv6 network.

3.2.  Routing Based UPF

   Traditionally, a UPF is implemented to follow 3GPP specifications.
   Specifically, N4 signaling is used for SMF to instruct a UPF to set
   up its session state in terms of PDRs/FARs.  On N6 side, a UPF
   receives downlink traffic with destination addresses that are covered
   by the UPF's address range for its anchored UEs.  The packet is
   matched against the installed PDRs and forwarded according to the
   associated FARs.  On N3 side, a UPF decapsulates GTP+UDP+IP header of
   uplink traffic and uses the TEID to identify the DN where inner IP

Zhang, et al.           Expires 7 September 2022                [Page 7]

Internet-Draft             5G Distributed UPFs                March 2022

   routing or Ethernet switching is done.

   [I-D.mhkk-dmm-srv6mup-architecture] specifies a new SRv6 based MUP
   architecture.  When it is applied to a 3GPP based mobile

   *  BGP signaling from a MUP Controller replaces N4 signaling from
      SMF.  N4 signaling is still used between the MUP Controller and
      SMF - from SMF's point of view it is just interacting with a
      traditional UPF as usual.

   *  A MUP GW becomes a distributed UPF for uplink traffic.

   *  A MUP PE, which is different from a usually central PSA UPF,
      becomes a UPF for downlink traffic, in that traffic to each UE is
      placed into a different tunnel that is stitched to a GTP tunnel
      for that UE by a MUP GW (no route lookup is needed on the MUP GW
      for the downlink traffic).

   In this approach UE to UE traffic may still optionally go through the
   central PSA UPF.  This is similar to that a hub router may be used in
   Section 2.

   This approach can be viewed as a specific way of implementing/
   deploying distributed UPFs discussed in Section 2.  It does have the
   advantage that from SMF's point of view, nothing is different from
   before - both from N4 signaling and deployment model point of view.

   While the above is specific to SRv6, a similar MPLS based approach
   will be specified separately for operators who prefer MPLS data
   plane, and it can even be SR-agnostic.

4.  Security Considerations

   To be provided.

5.  Informative References

              Matsushima, S., Filsfils, C., Kohno, M., Garvia, P. C.,
              Voyer, D., and C. E. Perkins, "Segment Routing IPv6 for
              Mobile User Plane", Work in Progress, Internet-Draft,
              draft-ietf-dmm-srv6-mobile-uplane-18, 18 February 2022,

Zhang, et al.           Expires 7 September 2022                [Page 8]

Internet-Draft             5G Distributed UPFs                March 2022

              Matsushima, S., Horiba, K., Khan, A., Kawakami, Y.,
              Murakami, T., Patel, K., Kohno, M., Kamata, T., Camarillo,
              P., Voyer, D., Zadok, S., Meilik, I., Agrawal, A., and K.
              Perumal, "Segment Routing IPv6 Mobile User Plane
              Architecture for Distributed Mobility Management", Work in
              Progress, Internet-Draft, draft-mhkk-dmm-srv6mup-
              architecture-01, 10 November 2021,

              Zhang, Z., "PW for IP/UDP Payload without IP/UDP Headers",
              Work in Progress, Internet-Draft, draft-zzhang-pals-pw-
              for-ip-udp-payload-00, 22 December 2021,

              "System architecture for the 5G System (5GS), V17.3.0",
              December 2021.

Authors' Addresses

   Zhaohui Zhang
   Juniper Networks


   Keyur Patel


   Tianji Jiang
   China Mobile


Zhang, et al.           Expires 7 September 2022                [Page 9]