Network Working Group                                        J. Korhonen
Internet-Draft                                      Broadcom Corporation
Intended status: Informational                              May 27, 2015
Expires: November 28, 2015

           Deterministic networking for radio access networks


   This document describes requirements for deterministic networking in
   cellular radio access and transport networks context.  The
   requirements include time synchronization, clock distribution and
   ways of establishing time-sensitive streams for both Layer-2 and
   Layer-3 user plane traffic using IETF protocol solutions.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on November 28, 2015.

Copyright Notice

   Copyright (c) 2015 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   ( in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Korhonen                Expires November 28, 2015               [Page 1]

Internet-Draft               DetNet for RAN                     May 2015

Table of Contents

   1.  Introduction and background . . . . . . . . . . . . . . . . .   2
   2.  Network architecture  . . . . . . . . . . . . . . . . . . . .   3
   3.  Time synchronization requirements . . . . . . . . . . . . . .   5
   4.  Time-sensitive stream requirements  . . . . . . . . . . . . .   6
   5.  Security considerations . . . . . . . . . . . . . . . . . . .   7
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   7
   7.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   8
   8.  Informative References  . . . . . . . . . . . . . . . . . . .   8
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  11

1.  Introduction and background

   The recent developments in telecommunication networks, especially in
   the cellular domain, are heading towards transport networks where
   precise time synchronization support has to be one of the basic
   building blocks.  While the transport networks themselves have
   practically transitioned to all-AP packet based networks to meet the
   bandwidth and cost requirements, a highly accurate clock distribution
   has become a challenge.  Earlier the transport networks in the
   cellular domain were typically time division and multiplexing (TDM)
   -based and provided frequency synchronization capabilities as a part
   of the transport media.  Alternatively other technologies such as
   Global Positioning System (GPS) or Synchronous Ethernet (SyncE)
   [SyncE] were used.  New radio access network deployment models and
   architectures may require time sensitive networking services with
   strict requirements on other parts of the network that previously
   were not considered to be packetized at all.  The time and
   synchronization support are already topical for backhaul and midhaul
   packet networks [MEF], and becoming a real issue for fronthaul
   networks.  Specifically in the fronthaul networks the timing and
   synchronization requirements can be extreme for packet based
   technologies, for example, in order of sub +-20 ns packet delay
   variation (PDV) and frequency accuracy of +0.002 PPM [Fronthaul].

   Both Ethernet and IP/MPLS [RFC3031] (and PseudoWires (PWE) [RFC3985]
   for legacy transport support) have become popular tools to build and
   manage new all-IP radio access networks (RAN)
   [].  Although various timing and
   synchronization optimizations have already been proposed and
   implemented including 1588 PTP enhancements
   [I-D.ietf-tictoc-1588overmpls][I-D.mirsky-mpls-residence-time], these
   solution are not necessarily sufficient for the forthcoming RAN
   architectures or guarantee the higher time-synchronization
   requirements [CPRI].  There are also existing solutions for the TDM
   over IP [RFC5087] [RFC4553] or Ethernet transports [RFC5086].  The
   really interesting and important existing work for time sensitive

Korhonen                Expires November 28, 2015               [Page 2]

Internet-Draft               DetNet for RAN                     May 2015

   networking has been done for Ethernet [TSNTG], which specifies the
   use of IEEE 1588 time precision protocol (PTP) [IEEE1588] in the
   context of IEEE 802.1D and IEEE 802.1Q.  While IEEE 802.1AS
   [IEEE8021AS] specifies a Layer-2 time synchronizing service other
   specification, such as IEEE 1722 [IEEE1722] specify Ethernet-based
   Layer-2 transport for time-sensitive streams.  New promising work
   seeks to enable the transport of time-sensitive fronthaul streams in
   Ethernet bridged networks [IEEE8021CM].  Similarly to IEEE 1722 there
   is an ongoing standardization effort to define Layer-2 transport
   encapsulation format for transporting radio over Ethernet (RoE) in
   IEEE 1904.3 Task Force [IEEE19043].

   As already mentioned all-IP RANs and various "haul" networks would
   benefit from time synchronization and time-sensitive transport
   services.  Although Ethernet appears to be the unifying technology
   for the transport there is still a disconnect providing Layer-3
   services.  The protocol stack typically has a number of layers below
   the Ethernet Layer-2 that shows up to the Layer-3 IP transport.  It
   is not uncommon that on top of the lowest layer (optical) transport
   there is the first layer of Ethernet followed one or more layers of
   MPLS, PseudoWires and/or other tunneling protocols finally carrying
   the Ethernet layer visible to the user plane IP traffic.  While there
   are existing technologies, especially in MPLS/PWE space, to establish
   circuits through the routed and switched networks, there is a lack of
   signaling the time synchronization and time-sensitive stream
   requirements/reservations for Layer-3 flows in a way that the entire
   transport stack is addressed and the Ethernet layers that needs to be
   configured are addressed.  Furthermore, not all "user plane" traffic
   will be IP.  Therefore, the same solution need also address the use
   cases where the user plane traffic is again another layer or Ethernet
   frames.  There is existing work describing the problem statement
   [I-D.finn-detnet-problem-statement] and the architecture
   [I-D.finn-detnet-architecture] for deterministic networking (DetNet)
   that eventually targets to provide solutions for time-sensitive (IP/
   transport) streams with deterministic properties over Ethernet-based
   switched networks.

   This document describes requirements for deterministic networking in
   a cellular telecom transport networks context.  The requirements
   include time synchronization, clock distribution and ways of
   establishing time-sensitive streams for both Layer-2 and Layer-3 user
   plane traffic using IETF protocol solutions.

2.  Network architecture

   Figure Figure 1 illustrates a typical, 3GPP defined, cellular network
   architecture, which also has fronthaul and midhaul network segments.
   The fronthaul refers to the network connecting base stations (base

Korhonen                Expires November 28, 2015               [Page 3]

Internet-Draft               DetNet for RAN                     May 2015

   band processing units) to the remote radio heads (antennas).  The
   midhaul network typically refers to the network inter-connecting base
   stations (or small/pico cells).

   Fronthaul networks build on the available excess time after the base
   band processing of the radio frame has completed.  Therefore, the
   available time for networking is actually very limited, which in
   practise determines how far the remote radio heads can be from the
   base band processing units (i.e.  base stations).  For example, in a
   case of LTE radio the Hybrid ARQ processing of a radio frame is
   allocated 3 ms.  Typically the processing completes way earlier (say
   up to 400 us, could be much less, though) thus allowing the remaining
   time to be used e.g. for fronthaul network. 200 us equals roughly 40
   km of optical fiber based transport (assuming round trip time would
   be total 2*200 us).  The base band processing time and the available
   "delay budget" for the fronthaul is a subject to change, possibly
   dramatically, in the forthcoming "5G" to meet, for example, the
   envisioned reduced radio round trip times, and other architecural and
   service requirements [NGMN].

   The maximum "delay budget" is then consumed by all nodes and required
   buffering between the remote radio head and the base band processing
   in addition to the distance incurred delay.  Packet delay variation
   (PDV) is problematic to fronthaul networks and must be minimized.  If
   the transport network cannot guarantee low enough PDV additional
   buffering has to be introduced at the edges of the network to buffer
   out the jitter.  Any buffering will eat up the total available delay
   budget, though.  Section Section 3 will discuss the PDV requirements
   in more detail.

Korhonen                Expires November 28, 2015               [Page 4]

Internet-Draft               DetNet for RAN                     May 2015

              Y (remote radios)
           Y__  \.--.                   .--.       +------+
              \_(    `.     +---+     _(Back`.     | 3GPP |
       Y------( Front  )----|eNB|----(  Haul  )----| core |
             ( `  .Haul )   +---+   ( `  .  )  )   | netw |
             /`--(___.-'      \      `--(___.-'    +------+
          Y_/     /            \.--.       \
               Y_/            _( Mid`.      \
                             (   Haul )      \
                            ( `  .  )  )      \
                             `--(___.-'\_____+---+    (small cells)
                                   \         |SCe|__Y
                                  +---+      +---+
                                Y_/   \_Y ("local" radios)

      Figure 1: Generic 3GPP-based cellular network architecture with
                        Front/Mid/Backhaul networks

3.  Time synchronization requirements

   Cellular networks starting from long term evolution (LTE) [TS36300]
   [TS23401] radio the phase synchronization is also needed in addition
   to the frequency synchronization.  The commonly referenced fronthaul
   network synchronization requirements are typically drawn from the
   common public radio interface (CPRI) [CPRI] specification that
   defines the transport protocol between the base band processing -
   radio equipment controller (REC) and the remote antenna - radio
   equipment (RE).  However, the fundamental requirements still
   originate from the respective cellular system and radio
   specifications such as the 3GPP ones [TS25104][TS36104][TS36211]

   The fronthaul time synchronization requirements for the current 3GPP
   LTE-based networks are listed below:

   Transport link contribution to radio frequency error:

      +-2 PPB.  The given value is considered to be "available" for the
      fronthaul link out of the total 50 PPB budget reserved for the
      radio interface.

   Delay accuracy:

      +-8.138 ns i.e. +-1/32 Tc (UMTS Chip time, Tc, 1/3.84 MHz) to
      downlink direction and excluding the (optical) cable length in one

Korhonen                Expires November 28, 2015               [Page 5]

Internet-Draft               DetNet for RAN                     May 2015

      direction.  Round trip accuracy is then +-16.276 ns.  The value is
      this low to meet the 3GPP timing alignment error (TAE) measurement

   Packet delay variation (PDV):

      *  For multiple input multiple output (MIMO) or TX diversity
         transmissions, at each carrier frequency, TAE shall not exceed
         65 ns (i.e. 1/4 Tc).

      *  For intra-band contiguous carrier aggregation, with or without
         MIMO or TX diversity, TAE shall not exceed 130 ns (i.e. 1/2

      *  For intra-band non-contiguous carrier aggregation, with or
         without MIMO or TX diversity, TAE shall not exceed 260 ns (i.e.
         one Tc).

      *  For inter-band carrier aggregation, with or without MIMO or TX
         diversity, TAE shall not exceed 260 ns.

   The above listed time synchronization requirements are hard to meet
   even with point to point connected networks, not to mention cases
   where the underlying transport network actually constitutes of
   multiple hops.  It is expected that network deployments have to deal
   with the jitter requirements buffering at the very ends of the
   connections, since trying to meet the jitter requirements in every
   intermediate node is likely to be too costly.  However, every measure
   to reduce jitter and delay on the path are valuable to make it easier
   to meet the end to end requirements.

   In order to meet the timing requirements both senders and receivers
   must is perfect sync.  This asks for a very accurate clock
   distribution solution.  Basically all means and hardware support for
   guaranteeing accurate time synchronization in the network is needed.
   As an example support for 1588 transparent clocks (TC) in every
   intermediate node would be helpful.

4.  Time-sensitive stream requirements

   In addition to the time synchronization requirements listed in
   Section Section 3 the fronthaul networks assume practically error
   free transport.  The maximum bit error rate (BER) has been defined to
   be 10^-12.  When packetized that would equal roughly to packet error
   rate (PER) of 2.4*10^-9 (assuming ~300 bytes packets).
   Retransmitting lost packets and/or using forward error coding (FEC)
   to circumvent bit errors are practically impossible due additional
   incurred delay.  Using redundant streams for better guarantees for

Korhonen                Expires November 28, 2015               [Page 6]

Internet-Draft               DetNet for RAN                     May 2015

   delivery is also practically impossible due to high bandwidth
   requirements fronthaul networks have.  For instance, current
   uncompressed CPRI bandwidth expansion ratio is roughly 20:1 compared
   to the IP layer user payload it carries in a "radio sample form".

   The other fundamental assumption is that fronthaul links are
   symmetric.  Last, all fronthaul streams (carrying radio data) have
   equal priority and cannot delay or pre-empt each other.  This implies
   the network has always be sufficiently under subscribed to guarantee
   each time-sensitive flow meets their schedule.

   Mapping the fronthaul requirements to [I-D.finn-detnet-architecture]
   Section 3 "Providing the DetNet Quality of Service" what is seemed
   usable are:

      (a) Zero congestion loss.

      (b) Pinned-down paths.

   The current time-sensitive networking features may still not be
   sufficient for fronthaul traffic.  Therefore, having specific
   profiles that take the requirements of fronthaul into account are
   deemed to be useful [IEEE8021CM].

   The actual transport protocols and/or solutions to establish required
   transport "circuits" (pinned-down paths) for fronthaul traffic are
   still undefined.  Those are likely to include but not limited to
   solutions directly over Ethernet, over IP, and MPLS/PseudoWire

5.  Security considerations

   Establishing time-sensitive streams in the network entails reserving
   networking resources sometimes for a considerable long time.  It is
   important that these reservation requests must be authenticated to
   prevent malicious reservation attempts from hostile nodes or even
   accidental misconfiguration.  This is specifically important in a
   case where the reservation requests span administrative domains.
   Furthermore, the reservation information itself should be digitally
   signed to reduce the risk where a legitimate node pushed a stale or
   hostile configuration into the networking node.

6.  IANA Considerations

   This document has no IANA considerations.

Korhonen                Expires November 28, 2015               [Page 7]

Internet-Draft               DetNet for RAN                     May 2015

7.  Acknowledgements

   The author(s) ACK and NACK.

8.  Informative References

   [CPRI]     CPRI Cooperation, "Common Public Radio Interface (CPRI);
              Interface Specification", CPRI Specification V6.1, July
              2014, <

              Chen, D. and T. Mustala, "Ethernet Fronthaul
              Considerations", IEEE 1904.3, February 2015,
              tf3_1502_che n_1a.pdf>.

              Finn, N., Thubert, P., and M. Teener, "Deterministic
              Networking Architecture", draft-finn-detnet-
              architecture-01 (work in progress), March 2015.

              Finn, N. and P. Thubert, "Deterministic Networking Problem
              Statement", draft-finn-detnet-problem-statement-01 (work
              in progress), October 2014.

              Davari, S., Oren, A., Bhatia, M., Roberts, P., and L.
              Montini, "Transporting Timing messages over MPLS
              Networks", draft-ietf-tictoc-1588overmpls-06 (work in
              progress), April 2014.

              Khasnabish, B., hu, f., and L. Contreras, "Segment Routing
              in IP RAN use case", draft-kh-spring-ip-ran-use-case-02
              (work in progress), November 2014.

              Mirsky, G., Ruffini, S., Gray, E., Drake, J., Bryant, S.,
              and S. Vainshtein, "Residence Time Measurement in MPLS
              network", draft-mirsky-mpls-residence-time-06 (work in
              progress), May 2015.

Korhonen                Expires November 28, 2015               [Page 8]

Internet-Draft               DetNet for RAN                     May 2015

              IEEE, "IEEE Standard for a Precision Clock Synchronization
              Protocol for Networked Measurement and Control Systems",
              IEEE Std 1588-2008, 2008,

              IEEE, "1722-2011 - IEEE Standard for Layer 2 Transport
              Protocol for Time Sensitive Applications in a Bridged
              Local Area Network", IEEE Std 1722-2011, 2011,

              IEEE Standards Association, "IEEE 1904.3 TF", IEEE 1904.3,
              2015, <>.

              IEEE, "Timing and Synchronizations (IEEE 802.1AS-2011)",
              IEEE 802.1AS-2001, 2011,

              Farkas, J., "Time-Sensitive Networking for Fronthaul",
              Unapproved PAR, PAR for a New IEEE Standard; IEEE
              P802.1CM, April 2015,
              new-P802-1CM-dr aft-PAR-0515-v02.pdf>.

   [MEF]      MEF, "Mobile Backhaul Phase 2 Amendment 1 -- Small Cells",
              MEF 22.1.1, July 2014,

   [NGMN]     NGMN Alliance, "5G White Paper", NGMN 5G White Paper v1.0,
              February 2015, <

   [RFC3031]  Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
              Label Switching Architecture", RFC 3031, January 2001.

   [RFC3985]  Bryant, S. and P. Pate, "Pseudo Wire Emulation Edge-to-
              Edge (PWE3) Architecture", RFC 3985, March 2005.

Korhonen                Expires November 28, 2015               [Page 9]

Internet-Draft               DetNet for RAN                     May 2015

   [RFC4553]  Vainshtein, A. and YJ. Stein, "Structure-Agnostic Time
              Division Multiplexing (TDM) over Packet (SAToP)", RFC
              4553, June 2006.

   [RFC5086]  Vainshtein, A., Sasson, I., Metz, E., Frost, T., and P.
              Pate, "Structure-Aware Time Division Multiplexed (TDM)
              Circuit Emulation Service over Packet Switched Network
              (CESoPSN)", RFC 5086, December 2007.

   [RFC5087]  Stein, Y(J)., Shashoua, R., Insler, R., and M. Anavi,
              "Time Division Multiplexing over IP (TDMoIP)", RFC 5087,
              December 2007.

   [SyncE]    ITU-T, "G.8261 : Timing and synchronization aspects in
              packet networks", Recommendation G.8261, August 2013,

   [TS23401]  3GPP, "General Packet Radio Service (GPRS) enhancements
              for Evolved Universal Terrestrial Radio Access Network
              (E-UTRAN) access", 3GPP TS 23.401 10.10.0, March 2013.

   [TS25104]  3GPP, "Base Station (BS) radio transmission and reception
              (FDD)", 3GPP TS 25.104 3.14.0, March 2007.

   [TS36104]  3GPP, "Evolved Universal Terrestrial Radio Access
              (E-UTRA); Base Station (BS) radio transmission and
              reception", 3GPP TS 36.104 10.11.0, July 2013.

   [TS36133]  3GPP, "Evolved Universal Terrestrial Radio Access
              (E-UTRA); Requirements for support of radio resource
              management", 3GPP TS 36.133 12.7.0, April 2015.

   [TS36211]  3GPP, "Evolved Universal Terrestrial Radio Access
              (E-UTRA); Physical channels and modulation", 3GPP TS
              36.211 10.7.0, March 2013.

   [TS36300]  3GPP, "Evolved Universal Terrestrial Radio Access (E-UTRA)
              and Evolved Universal Terrestrial Radio Access Network
              (E-UTRAN); Overall description; Stage 2", 3GPP TS 36.300
              10.11.0, September 2013.

   [TSNTG]    IEEE Standards Association, "IEEE 802.1 Time-Sensitive
              Networks Task Group", 2013,

Korhonen                Expires November 28, 2015              [Page 10]

Internet-Draft               DetNet for RAN                     May 2015

Author's Address

   Jouni Korhonen
   Broadcom Corporation
   3151 Zanker Road
   San Jose, CA  95134


Korhonen                Expires November 28, 2015              [Page 11]