Internet Engineering Task Force J. Arkko
Internet-Draft Ericsson
Intended status: Informational May 15, 2017
Expires: November 16, 2017
Low Latency Applications and the Internet Architecture
draft-arkko-arch-low-latency-00
Abstract
Some recent Internet technology developments relate to improvements
in communications latency. For instance, improvements in radio
communications or the recent work in IETF transport, security, and
web protocols. There are also potential applications where latency
is potentially in a more significant role than it has traditionally
been in Internet communications. Modern networking systems offer
many tools for building low-latency networks, from highly optimised
individual protocol components to software controlled, virtualised
and tailored network functions. This memo views the developments
from a system viewpoint, and considers the potential future stresses
that the strive for low-latency support for applications may bring.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on November 16, 2017.
Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
Arkko Expires November 16, 2017 [Page 1]
Internet-Draft Low Latency May 2017
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Applications with Special Focus on Low Latency . . . . . . . 3
3. Role of Low-Latency vs. Other Communications . . . . . . . . 4
4. Selected Improvements to Communications Latency . . . . . . . 4
5. Architectural Considerations . . . . . . . . . . . . . . . . 5
5.1. Background . . . . . . . . . . . . . . . . . . . . . . . 5
5.2. Implications . . . . . . . . . . . . . . . . . . . . . . 6
5.3. Recommendations for Further Work . . . . . . . . . . . . 8
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 8
7. Informative References . . . . . . . . . . . . . . . . . . . 9
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 11
1. Introduction
Some recent Internet technology developments relate to improvements
in communications latency. For instance, improvements in radio
communications or the recent work in IETF transport, security, and
web protocols.
There are also potential applications where latency is potentially in
a more significant role than it has traditionally been in Internet
communications.
New applications or technologies does not necessarily imply that
latency should be the main driving concern, or that any further
efforts beyond those already ongoing are needed. Indeed, modern
networking systems offer many tools for building low-latency
networks, across the stack. At the IETF, for instance, there has
been a recent increase in work related to transport, security, and
web application protocols, in part to make significant improvements
in latency and connection set-up times. Similar efforts in other
parts of the stack exist in 3GPP, IEEE, and other standards
organisations.
Despite a large number of specific developments, it may be
interesting to view the developments from a system viewpoint, and to
consider the potential future stresses that the strive for low-
latency support for applications may bring.
Arkko Expires November 16, 2017 [Page 2]
Internet-Draft Low Latency May 2017
The rest of this memo is organised as follows. Section 2 discusses
potential applications for low-latency communications. Section 4
reviews some of the recent work across the stack, relating to latency
improvements. Finally, Section 5 discusses some of the implications
(and non-implications) from an architectural perspective.
2. Applications with Special Focus on Low Latency
Most Internet applications enjoy significant benefits from low-
latency communications.
There are also potential applications where latency is potentially in
an even more significant role. For instance, embedding
communications technology in automation or traffic systems, or
consumer applications such as augmented or virtual reality where
advance buffering may not be feasible.
Many of the Internet-of-Things and critical services use cases in 5G,
for instance, have been listed as requiring low latency and high
reliability for communications [ER2015] [HU2015] [NGMN2015] [NO2015]
[QU2016] [IMT2020].
Some example use cases include optimisation of utility services such
as electricity networks, connected automation systems in factories,
remote control of machinery such as mining equipment, or embedded
technology in road or railway traffic.
The different applications vary in terms of their needs. Some may be
very focused on high-speed local area communication, others need to
connect at optimal speed over a wide-area network, and yet others
need to find the right ways to provide global services without
incurring unreasonable delays.
Note that when we say "low-latency capabilities", there is no intent
to imply any specific implementation of those capabilities. In
particular, we look at the low-latency requirements from a broader
perspective than Quality-of-Service guarantees or separating traffic
onto different classes. Indeed, while today's virtualisation and
software-driven technologies give us more tools to deal with those
kinds of arrangements as well, past experience on deploying Quality-
of-Service mechanisms in the Internet should give us pause [CC2015].
It is not the purpose of this memo to analyse the application
requirements for low-latency applications much further; for our
purposes it suffices to note that there are applications that are
enabled by low-latency capabilities of the underlying network
infrastructure.
Arkko Expires November 16, 2017 [Page 3]
Internet-Draft Low Latency May 2017
3. Role of Low-Latency vs. Other Communications
There are some limited applications that rely solely on local
communication. One example of such an application is vehicles
communicating braking status to nearby ones. However, many
applications will include also wide-area communication. If the
factory automation machines are not talking other than with
themselves, at least their control systems are doing so in order to
ensure parts orders, monitoring and maintenance by equipment
manufacturers, and so on. This does not imply that these perhaps
critical applications are openly accessible from the Internet, but
many of them are likely to communicate outside their immediate
surroundings.
Many applications also rely on wide-area connectivity for software
updates.
As a result, this document recommends that when building
architectures for low-latency applications it is important to take
into account that these applications can also benefit from
communications elsewhere.
4. Selected Improvements to Communications Latency
It should be noted that latency is a very broad topic in
communications protocol design, almost as broad as "security", or
even "correctness".
Implementation techniques to satisfy these requirements vary, some
applications can be built with sufficient fast local networking
capabilities, others may require, for instance, building world-wide,
distributed content delivery mechanisms.
Modern networking systems offer many tools for building low-latency
networks, across the stack. from highly optimised individual protocol
components [I-D.ietf-tls-tls13] [I-D.ietf-quic-transport] [RFC7540]
to software controlled, virtualised and tailored network functions
[NFV2012] [I-D.ietf-sfc-nsh] [OF2008]. Data- and software-driven
network managment and orchestration tools enable networks to be built
to serve particular needs.
Across the stack there are also many other tools, as well as tools
being in development, e.g., a new transport design [L4S] at the IETF.
On the lower layers, improvements in radio communications are being
made. For instance, the IEEE 802.1 Time-Sensitive Networking Task
Group [TSN8021] has worked to define precise time synchronization
mechanisms for a local area network, and scheduling mechanisms to
Arkko Expires November 16, 2017 [Page 4]
Internet-Draft Low Latency May 2017
enable different classes of traffic to use the same network while
minimising jitter and latency. At the IETF, the DETNET working group
is taking these capabilities and applying them for layer 3 networking
[DETNET].
The 3GPP 5G requirements for next-generation access technology are
stringent, and are leading to the optimization of the radio
interfaces. The requirements specify a one-way latency limit of
0.5ms for ultra-reliable low-latency communications [TS38913].
5. Architectural Considerations
Despite a large number of specific developments, it may be
interesting to view the developments from a system viewpoint, and to
consider the potential future stresses that the strive for low-
latency support for applications may bring.
5.1. Background
To begin with, it may be useful to observe that the requirements and
developments outlined above do not necessarily imply that any
specific new technology is needed or that the nature of
communications in the Internet would somehow fundamentally change.
And certainly not that latency should be the only or primary concern
in technology development.
With the drive for a new class of applications, there is often an
expectation that this means significant changes. However, all
changes need to stand on their own, be justifiable and deployable on
a global network. For instance, the discussion around the
introduction of the newest 4K or 8K high-definition video streaming
applications is reminiscent of the discussions about the introduction
of VoIP applications in the Internet. At the time, there was some
expectation that special arrangements and Quality-of-Service
mechanisms might be needed to support this new traffic class. This
turned out to be not true, at least not in general networks.
Experience tells us, for instance, that deploying Quality-of-Service
mechanisms in the Internet is hard, not so much because of the
technology itself, but due to lack of forces that would be able to
drive the necessary business changes in the ecosystem for the
technology to be feasibly deployable [CC2015]. As claffy and Clark
note:
"Although the Internet has a standards body (the IETF) to resolve
technical issues, it lacks any similar forum to discuss business
issues such as how to allocate revenues among competing ISPs
offering enhanced services. In the U.S., ISPs feared such
Arkko Expires November 16, 2017 [Page 5]
Internet-Draft Low Latency May 2017
discussions would risk anti-trust scrutiny. Thus, lacking a way
to negotiate the business implications of QoS, it was considered a
cost rather than a potential source of revenue. Yet, the
relentless growth of a diversity of applications with widely
varying performance requirements continued on the public Internet,
with ISPs using relatively primitive, and not always completely
benign, mechanisms for handling them."
These difficulties should not be read as prohibiting all changes. Of
course, change can also seem unlikely even in cases where it becomes
absolutely necessary or the forces necessary to make a change have
actually built up. As a result, statements regarding change in the
Internet should be carefully evaluated on their merits from both
technical and ecosystem perspective.
Secondly, we often consider characteristics from a too narrow
viewpoint. In the case of latency, it is easy to focus on a
particular protocol or link, whereas from the user perspective
latency is a property of the system, not a property of an individual
component.
For instance, improvements on the performance of one link on a
communications path can be insignificant, if the other parts make up
a significant fraction of the system-level latency. That may seem
obvious, but many applications are highly dependent on communications
between a number of different parties which may reside in different
places. For instance, a third party may perform authentication for a
cloud-based service that also interacts with user's devices and a
number of different sensors and actuators.
We cannot change the speed of light, and a single exchange with
another part of the world may result in a 100ms delay, or about 200
times longer than the expected 5G radio link delay for critical
applications. It is clear that designing applications from a system
perspective is very important.
5.2. Implications
As noted above, low-latency applications need to pay particular
attention to the placement of services in the global network.
Operations that are on the critical path for the low-latency aspects
of an application are unlikely to work well if those communications
need to traverse half of the Internet.
Many widely used services are already distributed and replicated
throughout the world, to minimise communications latency. But many
other services are not distributed in this manner. For low-latency
applications such distribution becomes necessary. Hosting a global
Arkko Expires November 16, 2017 [Page 6]
Internet-Draft Low Latency May 2017
service in one location is not feasible due to latency, even when
from a scale perspective a single server might otherwise suffice for
the service.
Content-Delivery Networks (CDNs) and similar arrangements are likely
to flourish because of this. In the most extreme cases, edge
computing services are needed.
How the communications are routed also matters. For instance,
architectures based on tunneling to a central point may incur extra
delay. One way to address this pressure is to use SDN- and
virtualisation-based networks that can be provisioned in the desired
manner, so that, for instance, instances of tunnel servers can be
placed in the topologically optimal place for a particular
application.
Recent developments in multipath transport protocols [RFC6824] also
provide application- and service-level control of some of the
networking behaviour. There is tension between application control
(often by content providers) and network control (often by network
operators).
One special case where that tension has appeared in the past is
whether there should be ways to provide information from applications
to networks on how packets should be treated. This was extensively
discussed during the discussion stemming from implications of
increased use of encryption in the Internet, and how that affects
operators [I-D.nrooney-marnew-report].
Another case where there is tension is between mechanisms designed
for a single link or network vs. end-to-end mechanisms. Many of the
stated requirements for low-latency applications are explicitly about
end-to-end characteristics and capabilities. Yet, the two mechanisms
are very different, and most of the deployment difficulties reported
in [CC2015] relate to end-to-end mechanisms.
Finally, in the search for even faster connection setup times one
obvious technique is cross-layer optimisation. We have seen some of
this in the IETF in the rethinking of the layers for transport,
transport layer security, and application framework protocols.
But while cross-layer optimisation can bring benefits, it also has
downsides. In particular, it connects different parts of the stack
in additional ways. This can lead to difficulties in further
evolution of the technology, if done wrong.
In the case of the IETF transport protocol evolution, significant
improvements were made to ensure better evolvability of the
Arkko Expires November 16, 2017 [Page 7]
Internet-Draft Low Latency May 2017
protocols than what we've experienced with TCP, starting from an
ability to implement the new protocols in applications rather than
in the kernel.
The effects of badly designed cross-layer optimisation are a
particular form of Internet ossification. The general networking
trend, however, is for greater flexibility and programmability.
Arguably, the ease at which networks can evolve is probably even more
important than their specific characteristics.
5.3. Recommendations for Further Work
Low-latency applications continue to be a hot topic in networking.
The following topics in particular deserve further work from an
architectural point of view:
o Application architectures for globally connected but low-latency
services.
o What are the issues with inter-domain Quality-of-Service
mechanisms? Are there approaches that would offer progress on
this field?
o Network architectures that employ tunneling, and mitigations
against the delay impacts of tunnels (such as tunnel server
placement or "local breakout" techniques).
o The emergence of cross-layer optimisations and how that affects
the Internet architecture and its future evolution.
o Inter-organisatorial matters, e.g., to what extent different
standards organisations need to talk about low latency effects and
ongoing work, to promote system-level understanding?
Overall, this memo stresses the importance of the system-level
understanding of Internet applications and their latency issues.
Efforts to address specific sub-issues are unlikely to be fruitful
without a holistic plan.
6. Acknowledgements
The author would like to thank Brian Trammell, Mirja Kuehlewind,
Linda Dunbar, Goran Rune, Ari Keranen, Jeff Tantsura, Stephen
Farrell, and many others for interesting discussions in this problem
space.
The author would also like to acknowledge the important contribution
that [I-D.dunbar-e2e-latency-arch-view-and-gaps] made in this topic.
Arkko Expires November 16, 2017 [Page 8]
Internet-Draft Low Latency May 2017
7. Informative References
[CC2015] claffy, kc. and D. Clark, "Adding Enhanced Services to the
Internet: Lessons from History", September 2015
(https://www.caida.org/publications/papers/2015/
adding_enhanced_services_internet/
adding_enhanced_services_internet.pdf).
[DETNET] "Deterministic Networking (DETNET) Working Group", March
2016 (https://tools.ietf.org/wg/detnet/charters).
[ER2015] Yilmaz, O., "5G Radio Access for Ultra-Reliable and Low-
Latency Communications", Ericsson Research Blog, May 2015
(https://www.ericsson.com/research-blog/5g/5g-radio-
access-for-ultra-reliable-and-low-latency-
communications/).
[HU2015] "5G Vision: 100 Billion connections, 1 ms Latency, and 10
Gbps Throughput", Huawei 2015
(http://www.huawei.com/minisite/5g/en/defining-5g.html).
[I-D.dunbar-e2e-latency-arch-view-and-gaps]
Dunbar, L., "Architectural View of E2E Latency and Gaps",
draft-dunbar-e2e-latency-arch-view-and-gaps-01 (work in
progress), March 2017.
[I-D.ietf-quic-transport]
Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed
and Secure Transport", draft-ietf-quic-transport-02 (work
in progress), March 2017.
[I-D.ietf-sfc-nsh]
Quinn, P. and U. Elzur, "Network Service Header", draft-
ietf-sfc-nsh-12 (work in progress), February 2017.
[I-D.ietf-tls-tls13]
Rescorla, E., "The Transport Layer Security (TLS) Protocol
Version 1.3", draft-ietf-tls-tls13-20 (work in progress),
April 2017.
[I-D.nrooney-marnew-report]
Rooney, N., "IAB Workshop on Managing Radio Networks in an
Encrypted World (MaRNEW) Report", draft-nrooney-marnew-
report-02 (work in progress), August 2016.
Arkko Expires November 16, 2017 [Page 9]
Internet-Draft Low Latency May 2017
[IMT2020] "Framework and overall objectives of the future
development of IMT for 2020 and beyond", ITU
Recommendation M.2083-0, September 2015
(http://www.itu.int/rec/R-REC-M.2083-0-201509-I/en).
[L4S] "Low Latency Low Loss Scalable throughput (L4S) Birds-of-
Feather Session", July 2016
(https://datatracker.ietf.org/wg/l4s/charter/).
[NFV2012] "Network Functions Virtualisation - Introductory White
Paper", ETSI,
http://portal.etsi.org/NFV/NFV_White_Paper.pdf, October
2012.
[NGMN2015]
"5G White Paper", NGMN Alliance, February 2015
(https://www.ngmn.org/fileadmin/ngmn/content/downloads/
Technical/2015/NGMN_5G_White_Paper_V1_0.pdf).
[NO2015] Doppler, K., "5G the next major wireless standard", DREAMS
Seminar, January 2015
(https://chess.eecs.berkeley.edu/pubs/1084/doppler-
DREAMS_5G_jan15.pdf).
[OF2008] McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G.,
Peterson, L., Rexford, J., Shenker, S., and J. Turner,
"OpenFlow: Enabling Innovation in Campus Networks", ACM
SIGCOMM Computer Communication Review, Volume 38, Issue 2,
pp. 69-74 2008.
[QU2016] "Leading the world to 5G", Qualcomm, February 2016
(https://www.qualcomm.com/media/documents/files/qualcomm-
5g-vision-presentation.pdf).
[RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure,
"TCP Extensions for Multipath Operation with Multiple
Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013,
<http://www.rfc-editor.org/info/rfc6824>.
[RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext
Transfer Protocol Version 2 (HTTP/2)", RFC 7540,
DOI 10.17487/RFC7540, May 2015,
<http://www.rfc-editor.org/info/rfc7540>.
Arkko Expires November 16, 2017 [Page 10]
Internet-Draft Low Latency May 2017
[TS38913] "3rd Generation Partnership Project; Technical
Specification Group Radio Access Network; Study on
Scenarios and Requirements for Next Generation Access
Technologies; (Release 14)", 3GPP Technical Report TR
38.913 V14.2.0, March 2017
(https://portal.3gpp.org/desktopmodules/Specifications/
SpecificationDetails.aspx?specificationId=2996).
[TSN8021] "Time-Sensitive Networking Task Group", IEEE
(http://www.ieee802.org/1/pages/tsn.html).
Author's Address
Jari Arkko
Ericsson
Kauniainen 02700
Finland
Email: jari.arkko@piuha.net
Arkko Expires November 16, 2017 [Page 11]