COINRG I. Kunze
Internet-Draft K. Wehrle
Intended status: Informational RWTH Aachen University
Expires: September 10, 2020 March 09, 2020
Industrial Use Cases for In-Network Computing
draft-kunze-coin-industrial-use-cases-02
Abstract
Cyber-physical systems and the Industrial Internet of Things are
characterized by diverse sets of requirements which can hardly be
satisfied using standard networking technology. One example are
latency-critical computations which become increasingly complex and
are consequently outsourced to more powerful cloud platforms for
feasibility reasons. The intrinsic physical propagation delay to
these remote sites can already be too high for given requirements.
The challenge is to develop techniques that bring together these
requirements. Utilizing available computational capabilities within
the network for in-network computing concepts can be a solution to
this challenge. This document discusses selected industrial use
cases to demonstrate how in-network computing concepts can be applied
to the industrial domain and to point out essential requirements of
industrial applications.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 10, 2020.
Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved.
Kunze & Wehrle Expires September 10, 2020 [Page 1]
Internet-Draft Industrial Use Cases March 2020
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. In-Network Control / Time-sensitive applications . . . . . . 4
2.1. Characterization and Requirements . . . . . . . . . . . . 4
2.1.1. Approaches . . . . . . . . . . . . . . . . . . . . . 5
3. Large Volume Applications/ Traffic Filtering . . . . . . . . 6
3.1. Characterization and Requirements . . . . . . . . . . . . 6
3.2. Approaches . . . . . . . . . . . . . . . . . . . . . . . 7
3.2.1. Traffic Filters . . . . . . . . . . . . . . . . . . . 7
3.2.2. In-Network (Pre-)Processing . . . . . . . . . . . . . 8
4. Industrial Safety (Dead Man's Switch) . . . . . . . . . . . . 9
4.1. Characterization and Requirements . . . . . . . . . . . . 9
4.1.1. Approaches . . . . . . . . . . . . . . . . . . . . . 9
5. Security Considerations . . . . . . . . . . . . . . . . . . . 10
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10
7. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 10
8. Informative References . . . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11
1. Introduction
The Internet is based on a best-effort network that provides limited
guarantees regarding the timely and successful transmission of
packets. This design-choice is suitable for general Internet-based
applications, but specialized industrial applications demand a number
of strict performance guarantees, e.g., regarding real-time
capabilities, which cannot be provided over regular best-effort
networks.
Enhancements to the standard Ethernet such as Time-Sensitive-
Networking [TSN] try to achieve the requirements on the link layer by
statically reserving shares of the bandwidth. These concepts are
well-suited for industrial settings with well understood
communication patterns where the communication paths are encapsulated
at the factory sites. In the Industrial Internet of Things (IIoT),
however, more and more parts of the industrial production domain are
interconnected. This increases the complexity of the industrial
Kunze & Wehrle Expires September 10, 2020 [Page 2]
Internet-Draft Industrial Use Cases March 2020
networks, makes them more dynamic, and creates more diverse sets of
requirements. Furthermore, process control is imagined to be
exercised from remote clouds for feasibility reasons which is why
solutions on the link layer alone are not sufficient in these
scenarios.
Common components of the IIoT can be divided into three categories as
illustrated in Figure 1. Following
[I-D.mcbride-edge-data-discovery-overview], EDGE DEVICES, such as
sensors and actuators, constitute the boundary between the physical
and digital world. They communicate the current state of the
physical world to the digital world by transmitting sensor data or
let the digital world interact with the physical world by executing
actions after receiving (simple) control information. The processing
of the sensor data and the creation of the control information is
done on COMPUTING DEVICES. They range from small-powered controllers
close to the EDGE DEVICES, to more powerful edge or remote clouds in
larger distances. The connection between the EDGE and COMPUTING
DEVICES is established by NETWORKING DEVICES. In the industrial
domain, they range from standard devices, e.g., typical Ethernet
switches, which can interconnect all Ethernet-capable hosts, to
proprietary equipment with proprietary protocols only supporting
hosts of specific vendors.
The challenge is to develop concepts that can include off-premise
entities (such as distant cloud platforms) as well as proprietary
--------
|Sensor| ------------| ~~~~~~~~~~~~ ------------
-------- ------------- { Internet } --- |Remote Cloud|
. |Access Point|--- ~~~~~~~~~~~~ ------------
-------- ------------- | |
|Sensor| ----| | | |
-------- | | -------- |
. | | |Switch| ----------------------
. | | -------- |
. | | ------------ |
---------- | |----------------- | Controller | |
|Actuator| ------------ ------------ |
---------- | -------- ------------
. |----|Switch|---------------------------| Edge Cloud |
---------- -------- ------------
|Actuator| ---------|
----------
|-----------| |------------------| |-------------------|
EDGE DEVICES NETWORKING DEVICES COMPUTING DEVICES
Figure 1: Industrial networks show a high level of heterogeneity.
Kunze & Wehrle Expires September 10, 2020 [Page 3]
Internet-Draft Industrial Use Cases March 2020
hosts into the communication and still satisfy the performance
requirements of modern industrial networks. The in-network computing
paradigm presents a promising starting point because (pre-)processing
data within the network can speed up the communication, e.g., by
reducing the amount of transmitted data and thus congestion.
Flexibly distributing the computation tasks across the network helps
to manage dynamic changes. Specifying general requirements for the
different application scenarios is difficult due to the mentioned
diversity. This draft characterizes and analyzes three distinct
scenarios to showcase potential requirements for the industrial
production domain and to illustrate how in-network computations can
be helpful.
2. In-Network Control / Time-sensitive applications
The control of physical processes and components of a production line
is essential for the growing automation of production and ideally
allows for a consistent quality level. Traditionally, the control
has been exercised by control software running on programmable logic
controllers (PLCs) located directly next to the controlled process or
component. This approach is best-suited for settings with a simple
model that is focussed on a single or few controlled components.
Modern production lines and shop floors are characterized by an
increasing amount of involved devices and sensors, a growing level of
dependency between the different components, and more complex control
models. A centralized control is desirable to manage the large
amount of available information which often has to be pre-processed
or aggregated with other information before it can be used. PLCs are
not designed for this array of tasks and computations could
theoretically be moved to more powerful devices. These devices are
no longer close to the controlled objects and induce additional
latency.
It is worthwhile to investigate whether the outsourcing of control
functionality to distant computation platforms is viable because
these platforms have a high level of flexibility and scalability. In
the following, we describe the requirements and characteristics of
the control setting in more detail.
2.1. Characterization and Requirements
A control process consists of two main components as illustrated in
Figure 2: a system under control and a controller. In feedback
control, the current state of the system is monitored, e.g., using
sensors and the controller influences the system based on the
difference between the current and the reference state to keep it
close to this reference state.
Kunze & Wehrle Expires September 10, 2020 [Page 4]
Internet-Draft Industrial Use Cases March 2020
reference
state ------------ -------- Output
----------> | Controller | ---> | System | ---------->
^ ------------ -------- |
| |
| observed state |
| --------- |
-------------------| Sensors | <-----
---------
Figure 2: Simple feedback control model.
Apart from the control model, the quality of the control primarily
depends on the timely reception of the sensor feedback, because the
controller can only react if it is notified of changes in the system
state. Depending on the dynamics of the controlled system, the
control can be subject to tight latency constraints, often in the
single-digit millisecond range. While low latencies are essential,
there is an even greater need for stable and deterministic levels of
latency, because controllers can generally cope with different levels
of latency, if they are designed for them, but they are significantly
challenged by dynamically changing or unstable latencies. The
unpredictable latency of the Internet exemplifies this problem if
off-premise cloud platforms are included.
The main requirements for the industrial control scenario are low and
stable latencies to ensure that processes can work continuously and
that no machines are damaged.
2.1.1. Approaches
Control models, in general, can become involved but there is a
variety of control algorithms that are composed of simple
computations such as matrix multiplication. As these are supported
by programmable network devices, it is possibile to compose
simplified approximations of the more complex algorithms and deploy
them in the network. While the simplified versions induce a more
inaccurate control, they allow for a quicker response and might be
sufficient to operate a basic tight control loop while the overall
control can still be exercised from the cloud. The problem, however,
is that networking devices typically only allow for integer precision
computation while floating-point precision is needed by most control
algorithms. Early approaches like [RUETH] have already shown the
general applicability of such ideas, but there are still a lot of
open research questions not limited to the following:
o How can one derive the simplified versions of the overall
controller?
Kunze & Wehrle Expires September 10, 2020 [Page 5]
Internet-Draft Industrial Use Cases March 2020
* How complex can they become?
* How can one take the limited computational precision of
networking devices into account when making them?
o How does one distribute the simplified versions in the network?
o How does the overall controller interact with the simplified
versions?
3. Large Volume Applications/ Traffic Filtering
In the IIoT, processes and machines can be monitored more effectively
resulting in more available information. This data can be used to
deploy machine learning (ML) techniques and consequently help to find
previously unknown correlations between different components of the
production which in turn helps to improve the overall production
system. Newly gained knowledge can be shared between different sites
of the same company or even between different companies.
Traditional company infrastructure is neither equipped for the
management and storage of such large amounts of data nor for the
computationally expensive training of ML approaches. Similar to the
considerations in Section 2, off-premise cloud platforms offer cost-
effective solutions with a high degree of flexibility and
scalability. While the unpredictable latency of the Internet is only
a subordinate problem for this use case, moving all data to off-
premise locations primarily poses infrastructural challenges which
are presented in more detail in the following.
3.1. Characterization and Requirements
Processes in the industrial domain are monitored by distributed
sensors which range from simple binary (e.g., light barriers) to
sophisticated sensors measuring the system with varying degrees of
resolution. Sensors can further serve different purposes, as some
might be used for time-critical process control while others are only
used as redundant fallback platforms. Overall, there is a high level
of heterogeneity which makes managing the sensor output a challenging
task.
Depending on the deployed sensors and the complexity of the observed
system, the resulting overall data volume can easily be in the range
of several Gbit/s [GLEBKE]. Using off-premise clouds for managing
the data requires uploading or streaming the growing volume of sensor
data using the companies' Internet access which is typically limited
to a few hundred of Mbit/s. While large networking companies can
simply upgrade their infrastructure, most industrial companies rely
Kunze & Wehrle Expires September 10, 2020 [Page 6]
Internet-Draft Industrial Use Cases March 2020
on traditional ISPs for their Internet access. Higher access speeds
are hence tied to higher costs and, above all, subject to the supply
of the ISPs and consequently not always available. A major challenge
is thus to devise a methodology that is able to handle such amounts
of data over limited access links.
Another aspect is that business data leaving the premise and control
of the company further comes with security concerns, as sensitive
information or valuable business secrets might be contained in it.
Typical security measures such as encrypting the data make in-network
computing techniques hardly applicable as they typically work on
unencrypted data. Adding security to in-network computing
approaches, either by adding functionality for handling encrypted
data or devising general security measures, is thus an auspicious
field for research which we describe in more detail in Section 5.
3.2. Approaches
There are at least two concepts which might be suitable for reducing
the amount of transmitted data in a meaningful way:
1. filtering out redundant or unnecessary data
2. aggregating data by applying pre-processing steps within the
network
Both concepts require detailed knowledge about the monitoring
infrastructure at the factories and the purpose of the transmitted
data.
3.2.1. Traffic Filters
Sensors are often set up redundantly, i.e., part of the collected
data might also be redundant. Moreover, they are often hard to
configure or not configurable at all which is why their resolution or
sampling frequency is often larger than required. Consequently, it
is likely that more data is transmitted than is needed or desired. A
trivial idea for reducing the amount of data is thus to filter out
redundant or undesired data before it leaves the premise using simple
traffic filters that are deployed in the on-premise network. There
are different approaches to how this topic can be tackled. A first
step would be to scale down the available sensor data to the data
rate that is needed. For example, if a sensor transmits with a
frequency of 5 kHz, but the control entity only needs 1 kHz, only
every fifth packet containing sensor data is let through.
Alternatively, sensor data could be filtered down to a lower
frequency while the sensor value is in an uninteresting range, but
let through with higher resolution once the sensor value range
Kunze & Wehrle Expires September 10, 2020 [Page 7]
Internet-Draft Industrial Use Cases March 2020
becomes interesting. It is important that end-hosts are informed
about the filtering so that they can distinguish between data loss
and data filtered out on purpose.
In this context, the following research questions can be of interest:
o How can traffic filters be designed?
o How can traffic filters be coordinated and deployed?
o How can traffic filters be changed dynamically?
o How can traffic filtering be signaled to the end-hosts?
3.2.2. In-Network (Pre-)Processing
There are manifold computations that can be performed on the sensor
data in the cloud. Some of them are very complex or need the
complete sensor data during the computation, but there are also
simpler operations which can be done on subsets of the overall
dataset or earlier on the communication path as soon as all data is
available. One example is finding the maximum of all sensor values
which can either be done iteratively on each intermediate hop or at
the first hop, where all data is available.
Using expert knowledge about the exact computation steps and the
concrete transmission path of the sensor data, simple computation
steps can be deployed in the on-premise network to reduce the overall
data volume and potentially speed up the processing time in the
cloud.
Related work has already shown that in-network aggregation can help
to improve the performance of distributed ML applications [SAPIO].
Investigating the applicability of stream data processing techniques
to programmable networking devices is also interesting, because
sensor data is usually streamed. In this context, the following
research questions can be of interest:
o Which (pre-)processing steps can be deployed in the network?
* How complex can they become?
o How can applications incorporate the (pre-)processing steps?
o How can the programming of the techniques be streamlined?
Kunze & Wehrle Expires September 10, 2020 [Page 8]
Internet-Draft Industrial Use Cases March 2020
4. Industrial Safety (Dead Man's Switch)
Despite increasing automation in production processes, human workers
are still often necessary. Consequently, safety measures have a high
priority to ensure that no human life is endangered. In traditional
factories, the regions of contact between humans and machines are
well-defined and interactions are simple. Simple safety measures
like emergency switches at the working positions are enough to
provide a decent level of safety.
Modern factories are characterized by increasingly dynamic and
complex environments with new interaction scenarios between humans
and robots. Robots can either directly assist humans or perform
tasks autonomously. The intersect between the human working area and
the robots grows and it is harder for human workers to fully observe
the complete environment.
Additional safety measures are essential to prevent accidents and
support humans in observing the environment. The increased
availability of sensor data and the detailed monitoring of the
factories can help to build additional safety measures if the
corresponding data is collected early at the correct position.
4.1. Characterization and Requirements
Industrial safety measures are typically hardware solutions because
they have to pass rigorous testing before they are certified and
deployment-ready. Standard measures include safety switches, which
need to be triggered manually, and light barriers. Additionally, the
working area can be explicitly divided into 'contact' and 'safe'
areas, indicating when workers have to watch out for interactions
with machinery.
These measures are static solutions, potentially relying on
specialized hardware, and are challenged by the increased dynamics of
modern factories where the factory configuration can be changed on
demand. Software solutions offer higher flexibility as they can
dynamically respect new information gathered by the sensor systems.
Depending on the corresponding occupational safety laws, the software
has to satisfy stringent requirements which cannot be satisfied by
regular best-effort networks.
4.1.1. Approaches
Software-based solutions can take advantage of the large amount of
available sensor data. Different safety indicators within the
production hall can be combined within the network so that
programmable networking devices can give early responses if a
Kunze & Wehrle Expires September 10, 2020 [Page 9]
Internet-Draft Industrial Use Cases March 2020
potential safety breach is detected. A rather simple possibility
could be to track the positions of human workers and robots.
Whenever a robot gets too close to a human in a non-working area or
if a human enters a defined safety zone, robots are stopped to
prevent injuries. More advanced concepts could also include image
data or combine arbitrary sensor data.
In this context, the following research questions can be of interest:
o How can the software give guaranteed safety over best-effort
networks?
o Which sensor information can be combined and how?
5. Security Considerations
Current in-network computing approaches typically work on unencrypted
plain text data because today's networking devices usually do not
have crypto capabilities. As is already mentioned in Section 3.1,
this above all poses problems when business data, potentially
containing business secrets, is streamed into remote computing
facilities and consequently leaves the control of the company.
Insecure on-premise communication within the company and on the shop-
floor is also a problem as machines could be intruded from the
outside. It is thus crucial to deploy security and authentication
functionality on on-premise and outgoing communication although this
might interfere with in-network computing approaches. Ways to
implement and combine security measures with in-network computing are
described in more detail in [I-D.fink-coin-sec-priv].
6. IANA Considerations
N/A
7. Conclusion
In-network computing concepts have the potential to improve
industrial applications. There are at least three scenarios for
which in-network processing can be beneficial, each having a unique
set of requirements.
In the control scenario, tight latency constraints in the single
digit millisecond range have to be satisfied despite the use of cloud
platforms and the corresponding unstable latency of the Internet.
In a second scenario, large amounts of data have to be transmitted to
cloud platforms for further evaluation. One important task here is
to reduce the amount of data that needs to be transmitted as the
Kunze & Wehrle Expires September 10, 2020 [Page 10]
Internet-Draft Industrial Use Cases March 2020
available Internet access speed is most likely non-sufficient. Apart
from that, security measures have to be implemented as business data
is transmitted to the Internet.
Regarding safety, software-based measures often lack the required
guarantees and do not withstand the testing for certification. In-
network processing with its potential for early responses can be a
solution by combining different sensor outputs early and acting
quickly.
8. Informative References
[GLEBKE] Glebke, R., "A Case for Integrated Data Processing in
Large-Scale Cyber-Physical Systems", DOI: 10125/60162, in
HICSS, January 2019.
[I-D.fink-coin-sec-priv]
Fink, I. and K. Wehrle, "Enhancing Security and Privacy
with In-Network Computing", draft-fink-coin-sec-priv-00
(work in progress), March 2020.
[I-D.mcbride-edge-data-discovery-overview]
McBride, M., Kutscher, D., Schooler, E., and C. Bernardos,
"Edge Data Discovery for COIN", draft-mcbride-edge-data-
discovery-overview-03 (work in progress), January 2020.
[RUETH] Rueth, J., "Towards In-Network Industrial Feedback
Control", DOI: 10.1145/3229591.3229592, in ACM SIGCOMM
NetCompute, August 2018.
[SAPIO] Sapio, A., "Scaling Distributed Machine Learning with In-
Network Aggregation", 2019,
<https://arxiv.org/abs/1903.06701>.
[TSN] "Time-Sensitive Networking (TSN) Task Group", 2019,
<https://1.ieee802.org/tsn/>.
Authors' Addresses
Ike Kunze
RWTH Aachen University
Ahornstr. 55
Aachen D-50274
Germany
Phone: +49-241-80-21422
Email: kunze@comsys.rwth-aachen.de
Kunze & Wehrle Expires September 10, 2020 [Page 11]
Internet-Draft Industrial Use Cases March 2020
Klaus Wehrle
RWTH Aachen University
Ahornstr. 55
Aachen D-50274
Germany
Phone: +49-241-80-21401
Email: wehrle@comsys.rwth-aachen.de
Kunze & Wehrle Expires September 10, 2020 [Page 12]