ICN Research Group                                          J.K. Choi
    Internet Draft                                               J.S. Han
    Intended status: Informational                               G.H. Lee
    Expires: June 28, 2023                                      H.S. Park
                                                                    KAIST
                                                              Dec 27, 2022
    
    
    
    
         Requirements and Challenges for User-level Service Managements of
                  IoT Network by utilizing Artificial Intelligence
                             draft-choi-icnrg-aiot-10
    
    
    Abstract
    
       This document describes the requirements and challenges to employ
       artificial intelligence (AI) into the constraint Internet of Things
       (IoT) service environment for embedding intelligence and increasing
       efficiency.
    
       The IoT service environment includes heterogeneous and multiple IoT
       devices and systems that work together in a cooperative and
       intelligent way to manage homes, buildings, and complex autonomous
       systems. Therefore, it is becoming very essential to integrate IoT
       and AI technologies to increase the synergy between them. However,
       there are several limitations to achieve AI enabled IoT as the
       availability of IoT devices is not always high, and IoT networks
       cannot guarantee a certain level of performance in real-time
       applications due to resource constraints.
    
       This document intends to present a right direction to empower AI in
       IoT for learning and analyzing the usage behaviors of IoT
       devices/systems and human behaviors based on previous records and
       experiences. With AI enabled IoT, the IoT service environment can be
       intelligently managed in order to compensate for the unexpected
       performance degradation often caused by abnormal situations.
    
    Status of this Memo
    
       This Internet-Draft is submitted in full conformance with the
       provisions of BCP 78 and BCP 79.
    
       Internet-Drafts are working documents of the Internet Engineering
       Task Force (IETF), its areas, and its working groups.  Note that
       other groups may also distribute working documents as Internet-Drafts.
    
    
    Choi, et.al.            Expires June 28, 2023                 [Page 1]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       Internet-Drafts are draft documents valid for a maximum of six months
       and may be updated, replaced, or obsoleted by other documents at any
       time.  It is inappropriate to use Internet-Drafts as reference
       material or to cite them other than as "work in progress."
    
       The list of current Internet-Drafts can be accessed at
       http://www.ietf.org/ietf/1id-abstracts.txt
    
       The list of Internet-Draft Shadow Directories can be accessed at
       http://www.ietf.org/shadow.html
    
       This Internet-Draft will expire on June 28, 2023.
    
    Copyright Notice
    
       Copyright (c) 2022 IETF Trust and the persons identified as the
       document authors. All rights reserved.
    
       This document is subject to BCP 78 and the IETF Trust's Legal
       Provisions Relating to IETF Documents
       (http://trustee.ietf.org/license-info) in effect on the date of
       publication of this document. Please review these documents carefully,
       as they describe your rights and restrictions with respect to this
       document. Code Components extracted from this document must include
       Simplified BSD License text as described in Section 4.e of the Trust
       Legal Provisions and are provided without warranty as described in
       the Simplified BSD License.
    
    Table of Contents
    
    
       1. Introduction ................................................ 3
       2. Challenging Issues of IoT network
                                           ............................ 6
          2.1. Untrusted and incorrect IoT devices ..................... 6
          2.2. Traffic burstiness of IoT network ....................... 7
          2.3. Management overheads of heterogeneous IoT sensors
                                                               ........ 8
          2.4. Power management of IoT devices ......................... 9
          2.5. Computational resource management for IoT services
                                                                ...... 10
          2.6. Network management of heterogeneous IoT services
                                                              ........ 11
       3. Overview of AI/ML-based IoT services ........................ 12
       4. Requirements for AI/ML-based IoT services ................... 14
          4.1. Requirements for AI/ML-based IoT data collection and delivery
    
           ........................................................... 15
          4.2. Requirements for intelligent and context-aware IoT services16
          4.3. Requirements for applying AI/ML to IoT data ............ 18
             4.3.1. Training AI/ML algorithm
                                            .......................... 18
    
    
    Choi, et.al.            Expires June 28, 2023                 [Page 2]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
             4.3.2. AI/ML inference in IoT application ................ 19
             4.3.3. AI/ML models update in IoT application ............ 19
          4.4. Requirements for distributed computing for AI/ML-based IoT
          services ................................................... 20
       5. State of arts of the artificial intelligence/machine learning
       technologies for IoT services
                                    .................................. 22
          5.1. Machine learning and artificial intelligence technologies
          review ..................................................... 22
             5.1.1. Supervised learning for IoT ....................... 22
             5.1.2. Unsupervised learning for IoT ..................... 24
             5.1.3. Reinforcement learning for IoT .................... 25
             5.1.4. Neural Network based algorithms for IoT ........... 27
          5.2. Technologies for lightweight and real-time intelligence
                                                                     . 28
          5.3. Federated learning
                                 ..................................... 30
       6. Use cases of AI/ML into IoT service ......................... 31
          6.1. Surveillance and Security in Smart Home ................ 31
             6.1.1. Characteristics of Smart Home for AI/ML processing
                                                                     . 31
             6.1.2. Use case ......................................... 32
          6.2. Smart home for elderly welfare ......................... 32
             6.2.1. Characteristics of Smart Home for elderly welfare
                                                                    .. 32
          6.3. Smart Building Management
                                        .............................. 33
             6.3.1. Characteristics of Smart Building for AI/ML processing33
             6.3.2. Use case ......................................... 33
          6.4. Manufacturing optimization in Smart factory ............ 33
             6.4.1. Characteristics of Smart Building for AI/ML processing34
             6.4.2. Use case ......................................... 34
          6.5. Smart parking ......................................... 34
             6.5.1. Characteristics of Smart Parking for AI/ML processing34
          6.6. Augmented Reality (AR) in IoT
                                            .......................... 35
             6.6.1. Characteristics of Augmented Reality for AI/ML
             processing .............................................. 35
          6.7. Smart Farm ............................................ 35
             6.7.1. Characteristics of Smart Farm for AI/ML processing
                                                                     . 36
       7. IANA Considerations ........................................ 36
       8. Acknowledgements ........................................... 36
       9. Contributors ............................................... 36
       10. Informative References
                                 ..................................... 36
    
    1. Introduction
    
       The document explains the effects of applying artificial intelligence
       /machine learning (AI/ML) algorithms in the Internet of Thing (IoT) s
       ervice environments.
    
       IoT applications will be deployed in heterogeneous and different area
    
    
    Choi, et.al.            Expires June 28, 2023                 [Page 3]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       s such as the energy, transportation, automation and manufacturing in
       dustries as well as the information and communication technology (IC
       T) industry. Many IoT sensors and devices can connect to an IoT servi
       ce environment where IoT objects cannot interoperate with each other
       and can interact with different applications. The IoT service may not
        run in a single administrative domain. If market demand exists, the
       cross-domain service scenarios for IoT applications could be widely d
       eployed. Future IoT applications occur at multiple domains of heterog
       eneity with various time scales.
    
       The IoT service requirements for common architectures and public APIs
        poses some challenges to the underlying service environment and netw
       orking technologies. Some IoT applications require significant securi
       ty and privacy as well as significant resource and time constraints.
       These mission-critical applications can be separated from many common
        IoT applications that current technology may not provide. It means t
       hat IoT service requirements are difficult to classify common require
       ments and functional requirements depending on IoT service scenario.
    
       Recently, artificial intelligence technologies can help the context-a
       ware IoT service scenarios apply rule-based knowledge accumulation. T
       he IoT service assumes that many sensing devices are connected to sin
       gle or multiple IoT network domains. Each sensor sends small packets
       to the IoT servers periodically or non-periodically. Detection data c
       ontains periodic status information that monitors whether the system
       is in a normal state or not. In some cases, alert information is incl
       uded for quick processing. Most IoT applications can operate in two m
       odes. One is a simple monitoring mode and the other is an abnormal mo
       de for rapid processing. In a simple monitoring phase, the IoT device
        periodically sends sensing data to the server. If the measured data
       is outside the normal range, the IoT service can change the operating
        mode to an abnormal phase and activate future probes. Alarm conditio
       ns should be promptly notified to responsible persons. For mission-cr
       itical applications, reliable communication with robust QoS requireme
       nts in terms of error and latency is required.
    
       Periodic data accumulation from IoT devices is cumbersome. Under norm
       al conditions, the IoT data is simply accumulated without further act
       ion. In an unusual situation, incoming IoT data can cause an urgent a
    
    
    Choi, et.al.            Expires June 28, 2023                 [Page 4]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       ction to notify the administrator of the problem. Streaming data traf
       fic from thousands of IoT devices is annoying to store in the databas
       e because it is not easy to extract unidentified or future incidents.
        Only a significant portion of the incoming data stream can be stored
        in a real-time database that is time-sensitive and capable of rapid
       query processing. A combination of different IoT detection data, incl
       uding location, time, and status, allows you to sort and categorize a
        portion of streaming data when an additional inspection is required,
        and perform real-time processing. One of the missions of the IoT dat
       abase is to be able to extract preliminary symptoms of unexpected acc
       idents from a large amount of streaming data.
    
       If some transmitted data is important to invoke the corresponding act
       ion, there are some questions about whether the incoming data is corr
       ect. If the incoming data contains accurate and time-critical events,
        appropriate real-time control and management can be performed. Howev
       er, if the incoming data is inaccurate or intentionally corrupted, ad
       ditional accidents may occur. In these cases, incoming data can trigg
       er to initiate additional inspections to protect against future unacc
       eptable situations. But, if time-critical data is missed due to error
       s in the sensing devices and the delivery protocol, there is no reaso
       n to configure IoT networks and devices at a high cost.
    
       It is not easy to analyze data collected through IoT devices installe
       d to monitor complex IoT service environments. If the sensor malfunct
       ions, the data of the sensor cannot be trusted. Additional investigat
       ion should be done if abnormal status from specific sensors is collec
       ted. The data of the redundant sensor installed in the same area shou
       ld be received or combined with other sensor information adjacent to
       the sensor to determine the abnormal state.
    
       For sensors installed in a specific area, sensing records will remain
        for a certain period of time. IoT service operators can look at the
       operational history of the sensor for a period of time to determine w
       hat problems were encountered when data was collected. When an abnorm
       al situation occurs, IoT sensor should investigate whether it noticed
        normal operations and notified the IoT service operator. If the abno
       rmal situation is not properly detected, the operator should analyze
       whether it was caused by malfunction of the IoT sensor/other reasons.
    
    
    Choi, et.al.            Expires June 28, 2023                 [Page 5]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       In the IoT service environment, it is possible to analyze the situati
       on accurately by applying recent artificial intelligence and machine
       learning technologies. If there is an operational record of the past,
        it is possible to determine when an abnormal situation arises. Most
       problems are likely to be repeated, so if the past learning experienc
       e is accumulated, the anomaly of IoT services can be easily and immed
       iately identified. In addition, when information gathered from variou
       s sensors is synthesized, it is possible to accurately determine whet
       her abnormal situations have occurred.
    
       Various types of IoT sensors are installed with certain purposes. It
       expects that all the IoT sensors intend to monitor the occurrence of
       special abnormal situations in advance. Therefore, it should be set i
       n advance what actions are required when a specific anomaly occurs. T
       he appropriate work is performed on the abnormal situation according
       to the procedure, predefined by the human. By using artificial intell
       igence and machine learning algorithms, the appropriate actions are t
       aken when an abnormal situation is detected from various IoT sensors.
    
    2. Challenging Issues of IoT network
    
       This section describes the challenging issues of data sensing,
       collection, transfer, and intelligent decision from untrusted data
       quality and unexpected situations of IoT service environments.
    
    2.1. Untrusted and incorrect IoT devices
    
       IoT traffic is similar to traditional Internet traffic with small
       packet sizes. Mobile IoT traffic can cause some errors and delays
       because wireless links are unstable and signal strength may be
       degraded with device mobility. If the signal strength of the IoT
       device with a power limit is not so strong, the reception quality of
       the IoT server may not be sufficient to obtain the measurement data.
    
       For mission-critical applications, such as smart-grid and factory-
       automation, expensive IoT sensors with self-rechargeable batteries
       and redundant hardware logic may be required. However, unexpected
       abnormal situations may occur due to sensor malfunctions. There are
       trade-offs between implementation cost and efficiency for cost-
       effective IoT services. When smart-grid and factory-automation
    
    
    Choi, et.al.            Expires June 28, 2023                 [Page 6]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       applications are equipped with IoT devices, the acceptable quality
       from IoT solutions can be required. Sometimes, expensive and
       duplicated IoT solutions may be needed.
    
    2.2. Traffic burstiness of IoT network
    
       IoT traffic includes two types of traffic characteristic: periodic
       with small packet sizes and bursty with high bandwidth. Under normal
       conditions, the IoT traffic periodically transmits status information
       with a small bandwidth, several kilobits/sec. However, in an abnormal
       state, IoT devices need a high bandwidth, up to several tens of
       megabits/sec, in order to identify actual events and investigate
       accurate status information. In addition, traffic volume can
       explosively increase in response to emergencies. For example, in the
       case of smart-grid application, the bandwidth of several kilobits/sec
       is usually used, and when an urgent situation occurs, a broadband
       channel is required up to several tens of megabits/sec.
    
       The other traffic can be integrated at an IoT network to increase
       bandwidth efficiency. If an emergency situation occurs in the IoT
       service, IoT traffic volumes suddenly increase, in which case network
       processing capacity may be not sufficient. If the IoT service is
       integrated with voice and video applications, the problem can become
       more complex. As time goes by, traffic congestion and bottlenecks are
       frequent in some areas. In addition, if an existing service policy
       changes (for example, prioritizing certain traffic or suddenly
       changing the route), other unexpected problems may be encountered.
       Various congestion control and load balancing algorithms with the
       help of artificial intelligence can be applied to handle time-varying
       traffic on a network.
    
       Until now, much research has been done on traffic variability in an
       integrated network service environment. All networks have their own
       traffic characteristics, depending on geographical area, number of
       subscribers, subscribers' preferences, and types of applications used.
       In the case of IoT traffic, the normal bandwidth is very small. If
       the IoT traffic volume increases abruptly in an abnormal situation,
       the network may suffer unacceptable delay and loss. If emergency
       situations detected by IoT networks occur in a smart grid or
    
    
    Choi, et.al.            Expires June 28, 2023                 [Page 7]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       intelligent transportation system, the processing power of the IoT
       network alone cannot solve the problem and the help of existing
       network resources is inevitable.
    
    2.3. Management overheads of heterogeneous IoT sensors
    
       Traffic management in an integrated network environment is not easy.
       In order to operate the network steadily, a network operator has its
       own know-hows and experiences. If there are plenty of network
       resources, it is easy to set up a bypass route even if network
       failure or congestion occurs in a specific area. For operating
       network steadily, network resources may be designed to be over-
       provisioned in order to cope with various possible outages. A network
       operator predicts the amount of traffic generated by the
       corresponding equipment and grasps to what extent a transmission
       bandwidth is required. If traffic fluctuation is very severe, the
       network operator can allocate network resources in advance. In case
       of frequent failures or severe traffic fluctuation, some network
       resources are separated in order not to affect normal traffic.
    
       More than a billion IoT devices are expected to connect to
       smartphones, tablets, wearables, and vehicles. Therefore, IoT
       services are targeted at mobile applications. In particular,
       intelligent transportation systems need the help of IoT technology to
       provide traffic monitoring and prevent public or private traffic
       accidents. IoT technology can play an important role in reducing
       traffic congestion, saving people's travel time and costs, and
       providing a pleasant journey.
    
       The IoT service has troublesome administrative problems to configure
       an IoT network which consists of IoT servers, gateways, and many
       sensing devices. The small-sized but large-numbered IoT devices may
       incur administrative overhead since all the IoT devices should be
       initialized and the bootstrapping information of IoT resources should
       be loaded into the IoT service environments. Whenever some IoT
       devices are newly added and some devices have to be removed, the
       dynamic reconfiguration of IoT resources is essential. In addition,
       the IoT device's preinstalled software should be regularly inspected
       and upgraded according to its version. Frequent upgrades and changes
    
    
    Choi, et.al.            Expires June 28, 2023                 [Page 8]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       to some IoT devices may require autonomic management and
       bootstrapping techniques.
    
       Network management generally assumes that all network resources
       operate reliably with acceptable quality. In most failure situations,
       the network operator decides to switch to a redundant backup device
       or bypass the failed communication path. If some IoT devices are not
       stable, duplicate IoT devices can be installed for the same purpose.
       If IoT resources are not duplicated, various mechanisms are needed to
       reduce the damage. Therefore, it is necessary to prioritize the
       management tasks to be performed first when an abnormality occurs in
       the IoT service environment. However, managing duplicate networks can
       cause another problem. If two IoT devices are running at the same
       time, the recipient can get redundant information. If two or more
       unusual situations occur at the same time, it is difficult to solve
       the problem since tasks for urgent processing should be distinguished
       from tasks that can be performed over time.
    
       In addition, the operations manager's mistakes or misunderstanding of
       problem situations can lead to other unexpected complications.
       Therefore, artificial intelligence technologies can help what kind of
       network management work is required when an unexpected complicated
       situation occurs even though a procedure for an abnormal situation is
       already prepared.
    
    2.4. Power management of IoT devices
    
       RF-enabled wireless energy transfer (WET) technology has the
       advantage of enhancing poor user experience quality, which is a
       critical issue that can occur in the existing energy harvesting
       equipment of IoT equipment or conventional battery-powered
       communication, and can have sustainable throughput performance. WET
       technology is expected to become the core of next-generation
       communication because it can transmit stable and continuous power to
       equipment through electromagnetic waves that propagate far-field.
       This technology is expected to have advantages in communication
       stability and battery replacement cycle, which are limitations of
       existing IoT battery equipment.
    
    
    
    Choi, et.al.            Expires June 28, 2023                 [Page 9]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       The WET service simultaneously charges energy and transmits data at
       the hybrid access point, and in this process, network operation
       management technology that helps each user manage battery life is
       essential. To this end, the network must be able to manage the power
       and traffic budget of multiple APs and enhance the battery life of
       each IoT device. By efficiently managing energy and power
       transmission between multiple APs operating at the same time, it can
       be helpful for battery-life and reliable communication of a large
       number of IoT devices used at the same time. For example, for users
       who are constantly far away from the AP and take a long time to
       charge, the nearest AP can continuously transmit power. In addition,
       transmissions can proceed to more distant APs to avoid congestion on
       specific APs.
    
       Until now, only traffic in the network service environment is
       considered. However, in an environment where WET becomes more common,
       it seems difficult to manage network resources only with the existing
       traffic-oriented network environment. For efficient network operation
       in an environment that considers energy and traffic at the same time,
       period control or network slicing of a new concept will be required.
    
    2.5. Computational resource management for IoT services
    
       IoT networks for various industries require intelligent IoT services
       designed to make decisions and take actions in various situations by
       applying data analysis techniques based on data collected from IoT
       devices. For example, an IoT system for an autonomous vehicle should
       analyze vehicle information (speed, acceleration, steering angle,
       etc.) and camera images collected from the vehicle to control vehicle
       speed, direction, and vehicle convenience functions, etc. An IoT
       system for augmented/virtual reality needs to analyze lots of
       information such as the user's visual information, motion information,
       and object location information to perform display control suitable
       for the user's movement.
    
       Since these IoT applications need to collect data and make context-
       aware decisions in real-time, they generally require low latency.
       However, as the complexity and computation requirements of IoT
       applications increase, it is hard to satisfy the required latency
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 10]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       time by utilizing only IoT devices due to the limited computing
       capability.
    
       Therefore, in order to overcome the limited computational capability
       of IoT devices and satisfy the service latency requirements of users,
       many studies have focused on offloading computational workload to
       computing nodes in the IoT network such as gateways and cloudlets, or
       to cloud by extending the IoT network to the cloud. While offloading
       computational workload, the optimal computing nodes are selected in
       consideration of various factors such as status of the computing
       nodes and communication bandwidth. After the optimal node selection,
       the computational workloads are processed in the selected nodes and
       data transmissions are performed.
    
       However, various problems exist in such computational workload
       offloading solutions. Communication overhead occurs because data must
       be exchanged between computing nodes in order to distribute and
       process computation. In addition, in the process of data movement,
       users' privacy issues are also emerging. In conclusion, as IoT
       applications become computational-intensive and users'perceptions of
       data are changing, it is necessary to overcome the limited
       computational capability of IoT devices by applying various
       computation distribution technologies, and in this process, various
       problems mentioned above need to be considered.
    
    2.6. Network management of heterogeneous IoT services
    
       The number of applications targeting not only mobile devices but also
       IoT devices is increasing as many more IoT devices are connected
       through a shared network. Correspondingly, the application services
       targeting different types of IoT devices have become more diversified.
       For example, mobile broadband, massive IoT, and mission-critical IoT
       have different requirements concerning mobility, charging, security,
       police control, latency, and reliability. In terms of IoT devices for
       massive IoT, their locations tend to be fixed which results in less
       mobility, but high throughput while maintaining their massive
       connection. On the other hand, autonomous driving and unmanned aerial
       vehicle containing mission-critical IoT require low latency within
       milliseconds and highly reliable transmission while considering their
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 11]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       mobility. As a result, it is difficult to guarantee such different
       services with the existing mobile communication network.
    
       While constructing new networks for each service has many limitations
       in terms of spatial and financial aspects, network slicing allows
       operators to manage different services with more flexibility and
       agility within the existing network at low cost. The concept of
       network slicing is to run multiple logical customized networks,
       referred to as slices, on a shared infrastructure. Devices using the
       same application services are associated with the same slice to
       separately manage their requirements from that of devices in other
       slices. However, to efficiently manage the network slicing,
       coordination of physical resources between slices is necessary. These
       physical resources include frequency bandwidth, cache memory, and
       backhaul link capacity. For efficient management, such resources
       should be dynamically coordinated between slices considering traffic
       characteristics, amount of data requests, and transmission protocol.
    
    3. Overview of AI/ML-based IoT services
    
       In this section, successful applications of artificial intelligence i
       n IoT domains are provided. The common property of IoT applications a
       nd services is that they require fast analytics rather than later ana
       lytics with piled data. Recently, neural-network-based artificial int
       elligence technologies are widely used across many IoT applications.
    
       Simple IoT applications include dynamic contexts that share common fe
       atures among social relations at the same administration domain. IoT
       devices in the same domain can provide their service contexts to the
       IoT server. When a dynamic change occurs in an IoT service context, t
       he IoT device needs real-time processing to activate urgent events, a
       lert notifications, update, and reconnect contexts. The IoT service m
       ust support real-time interactions between the IoT device and the sys
       tem in the same domain. The IoT service contexts must be shared betwe
       en physical objects and social members in the same domain as well.
    
       Artificial intelligence technologies have been shown promising in man
       y areas, including IoT. For example, contextual information for a car
       -sharing business must interact with customers, car owners, and car s
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 12]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       haring providers. All entities in the value chain of a car sharing bu
       siness must share the corresponding situation to pick up, board, and
       return shared cars. Communication networks and interactive informatio
       n, including registration and payment, can be shared tightly among th
       e entities. Home IoT service environment can be equipped with sensors
        for theft detection, door lock, temperature, fire detection, gas det
       ection, short circuit, air condition to name a few. Office IoT servic
       e environments, including buildings such as shopping centers and bus/
       airport terminals, have their own sensors, including alarm sensors. W
       hen an alarm signal is detected by the sensor, the physical position
       and occurrence time of the sensor is determined in advance. All signa
       ls from various sensors are analyzed comprehensively to make the righ
       t decision. If some sensors frequently malfunction, the situation can
        be grasped more accurately by analyzing the information of the adjac
       ent sensor. In particular, when installing multiple sensors in a part
       icular building (e.g., surveillance camera, location monitoring, temp
       erature, etc.), a much wider range of sensors can be used when utiliz
       ing artificial intelligence and machine learning technologies.
    
       (Smart home) Smart home concept span over multiple IoT applications,
       health, energy, entertainment, education, etc. It involves voice reco
       gnition, natural language processing, image-based object recognition,
        appliance management, and many more artificial intelligence technolo
       gies integrated with IoT. Smart connected-devices monitor the house t
       o provide better control over home supplies and expenses. The energy
       consumption and efficiency of home appliances are monitored and analy
       zed with deep learning based technologies, such as artificial neural
       network, long-short-term-memory, etc.
    
       (Smart city) Smart city, as well, contains multiple IoT domains, tran
       sportation, infrastructure, energy, agriculture, etc. Since heterogen
       eous data from different domains are gathered in smart cities, variou
       s artificial intelligence approaches are studied in smart-city applic
       ation. Public transportation behaviors and crowd movements patterns a
       re important issues, and they are often dealt with neural network bas
       ed methods, long-short-term-memory and convolutional neural network.
    
       (Smart energy) As two-way communication energy infrastructure is depl
       oyed, smart grid has become a big IoT application, which requires int
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 13]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       elligent data processing. The traditional energy providers are highly
        interested in recognizing local energy consumption patterns and fore
       casting the needs in order to make appropriate decisions on real-time.
        Moreover, the energy consumers, as well, want analyzed information o
       n their own energy consumption behaviors. Recently, many works on ene
       rgy consumption prediction, energy flexibility analysis, etc. are act
       ively ongoing. Most works are based on the latest deep learning techn
       ologies, such as multi-layered-perceptron, recurrent neural network,
       long-short-term-memory, autoencoder, etc.
    
       (Smart transportation) The intelligent transportation system is anoth
       er source of big data in IoT domains. Many use cases, such as traffic
        flow and congestion prediction, traffic sign recognition, vehicle in
       trusion detection, etc., have been studied. Moreover, a lot of advanc
       ed artificial intelligence technologies are required in autonomous an
       d smart vehicles, which require many intelligent sub-tasks, such as p
       edestrian's detection, obstacle avoidance, etc.
    
       (Smart healthcare) IoT and artificial intelligence are integrated int
       o the healthcare and wellbeing domain as well. By analyzing food imag
       es with convolutional neural network on mobile devices, dietary intak
       es can be measured. With voice signal captured from sensor devices, v
       oice pathologies can be detected. Moreover, recurrent neural network
       and long-short-term-memory technologies are actively being studied fo
       r early diagnosis and prediction of diseases with time series medical
        data.
    
       (Smart agriculture) To manage a vast area of land, IoT and artificial
        intelligence technologies are recently used in agriculture domains.
       Deep neural network and convolutional neural network are utilized for
        crop detection or classification and disease recognition in the plan
       ts. Moreover, for automatic farming with autonomous machine operation,
        obstacle avoidance, fruit location, and many more sub-tasks are hand
       led with advanced artificial intelligence technologies.
    
    4. Requirements for AI/ML-based IoT services
    
       In this section, requirements for AI/ML-based IoT data collection and
        delivery, intelligent and context-aware IoT services, and applying A
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 14]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       I/ML to IoT data will be described.
    
    4.1. Requirements for AI/ML-based IoT data collection and delivery
    
       IoT services store a vast amount of data that IoT devices
       periodically generate, and the refining and analyzing are costly.
       Effective analysis of IoT data has been considered to be the most
       important factor in data processing, but the analysis of efficient
       data collection and delivery methods are becoming other significant
       factors as the amount of the data collected is explosively increasing.
    
       In particular, as a number of IoT devices have been deployed within
       the IoT network, controlling data collection and delivery for each of
       them has become impossible. The introduction of AI/ML techniques for
       simultaneous and efficient management of the IoT devices should be
       considered as a countermeasure. For IoT data collection and delivery,
       the following two factors will need to be considered, IoT devices
       energy and data quality.
    
       (IoT Device Energy) As many IoT devices have begun to be deployed
       within the IoT network, it is impossible to deliver energy to many
       IoT devices simultaneously. Consequently, the efficient battery use
       has become an important issue.
    
       If IoT data collection and delivery periods are too short, a lifetime
       of the IoT device will be shortened through the reckless use of IoT
       device energy. Thereby, it increases the cost required to provide IoT
       service. On the other hand, if IoT data collection and delivery
       period are too long, the quality of the IoT services provided will be
       reduced due to the lack of details in the data for situation
       recognition and real-time processing. Therefore, taking into account
       the energy consumption of the IoT devices, research on proper IoT
       data collection and delivery period is necessary.
    
       (IoT device using WET) When many IoT devices use WET to charge energy,
       transmit data, and perform calculations, it is difficult to deliver
       desired amount of power to all users. Therefore, efficient energy
       transfer for communication is essential.
    
    
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 15]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       When the UE acquires and transmits energy using WET, it is necessary
       to consider the amount of energy to transmit to each AP or user to
       satisfy QoS or user experience. We also need information about the
       optimal amount of energy transfer for battery life. Therefore,
       research should be conducted considering the power consumption and
       traffic of each AP and UE.
    
        (Data Quality) Since the data collected from the majority of IoT
       devices usually contain redundant information, it causes additional
       costs for the data collection and refinement processes. Therefore, it
       will be necessary to select and deliver meaningful information from
       redundant IoT data to reduce unnecessary cost on the IoT network. To
       do so, it will need the research to identify the relationships among
       the data collected various devices and interpret the information that
       the data contains.
    
       (Optimal IoT Device Operation) Each IoT service may have its own
       defined information requirements and it is imperative that the data
       with certain quality level should be collected to extract the core
       information for each situation. As scheme of data collection
       considerably affects the data quality, the operation of IoT devices
       should be effectively adjusted to ensure the data quality. In general,
       high resolution of data collected from IoT devices as well as data
       preprocessing are required for high quality data. However, these
       methods leading tremendous operation cost of IoT devices causes a
       lifetime of IoT networks, degrading the QoS of IoT services.
       Eventually, an optimal IoT device operation technique for each IoT
       service must be considered.
    
    4.2. Requirements for intelligent and context-aware IoT services
    
       In a context-aware IoT service environment, it is important to
       establish a context to be aware of in advance since IoT devices will
       be deployed according to a pre-designed architecture and to check how
       characteristics of IoT data and data-to-data characteristics are
       expressed under these circumstances. For the data produced by IoT
       devices, since it contains the device's relative location information,
       sensing value over time, event information, it should be reviewed to
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 16]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       provide the target context-aware service using this information. Some
       of the necessary technologies will be described in the following.
    
       (Physical Clustering) To increase the accuracy of context-awareness,
       the provision of context-aware services should be considered in a
       situation where the relationship between IoT devices with respect to
       physical layout or physical environment is taken into account.
       Setting a rule using the service provider's domain knowledge may be
       possible, but introducing the physical clustering into a diverse IoT
       environment (e.g., in bedroom, kitchen, balcony, or a space connected
       through an open door) will require identifying the physical
       relationship between the devices using data generated from IoT
       devices.
    
       (Extra Data Processing) In order to prevent degradation of service
       quality from errors in data values or device malfunctions, extra
       sensors should be placed in the majority of IoT environments. In a
       context-aware service, they contain the same information, so the
       technologies filtering the data that contains only essential part
       among the same information while preventing data errors would be
       required.
    
       (Unreported data handling) If an event is detected on a particular
       IoT device, it will transmit data regardless of the device's sensing
       and delivery interval. At this time, the data of IoT devices which
       are physically clustered are needed to accurately detect events that
       occurred, and it is difficult to expect that these devices will
       provide data at the same time. A gateway can request data from
       clustered devices, but it has a problem for real-time processing for
       emergency situations. Therefore, handling unreported data will be
       required based on previously collected data.
    
       (Abnormal data in AI/ML) In the case of context-aware services that
       operates based on the predetermined rule, the flexibility to cope
       with emergency situations that have not been considered is low, and
       thus AI/ML algorithms are required to intelligently cope with a
       myriad of situations. However, many abnormal data are generated
       depending on environmental conditions such as device status, so AI/ML
    
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 17]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       algorithms that can operate in that imperfect environment should be
       considered.
    
       (Edge computing in IoT) There are two necessary prerequisites
       required in context-aware IoT services: IoT devices real-time
       management and IoT network architecture supporting the high-volume
       data transmission. When an abnormal situation is discovered, high-
       volume data should be utilized to adequately to monitor the situation
       through the IoT device's real-time management. As contrasted with
       conventional cloud computing structures, an edge computing structure,
       where IoT data processing servers are located in closer proximity to
       IoT devices, provides higher energy efficiency and lesser
       communication delay.
    
    4.3. Requirements for applying AI/ML to IoT data
    
       In this subsection, the requirements for applying AI/ML to IoT data
       are described.
    
    4.3.1. Training AI/ML algorithm
    
       To use AI/ML algorithm, two elements are required, AI/ML model and
       training data. The presence of training dataset in good quality is an
       important factor of the AI/ML model performance since the model is
       iteratively trained with the training data. However, for anomaly
       detection, there is not enough training data since not only the
       probability of anomaly occurrence is very low but also it is almost
       impossible to retrieve the ground truth value even when the situation
       has occurred. Therefore, using domain knowledge, AI/ML learning based
       on abnormal situation data generation or simulation should be
       considered. For example, for an external intrusion detection
       application within a smart home, when a camera and a motion sensor
       detect an intruder, a light sensor checks the measuring value. If the
       light does not turn on, then the IoT application recognizes it as an
       abnormal situation. In this way, by using the domain knowledge, the
       rule regarding the operational scenario of the IoT application is
       generated as the training data, and the generated training data can
       be used for model learning. This will not only enable learning the
       anomaly detection algorithm in IoT application but also improving the
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 18]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       accuracy. Therefore, IoT application, in which it is difficult to
       acquire dataset in good quality, will require data generation based
       on domain knowledge for AI/ML.
    
    4.3.2. AI/ML inference in IoT application
    
       In order for AI ML technology to be applied to IoT applications, the
       training data and the input data for model testing and inferencing
       must have the same characteristics such as dimension, time interval,
       types of features, etc. However, due to the volatile IoT data
       characteristics that vary from situations in many IoT applications,
       it is difficult to directly apply the AI/ML algorithms. For example,
       in a simple monitoring phase, the IoT devices periodically send
       sensing data, and AI/ML have no difficulty in operating. However, in
       an abnormal mode, the IoT applications require a fast response, and
       IoT devices transmit data at shorter intervals than normal, which
       changes the characteristics of the data being input to the AI/ML
       algorithm. Therefore, data preprocessing technology handling the
       abnormal data will be required in advance, such as data imputation,
       correcting data anomalies, and Interpolation of unreported data.
    
    4.3.3. AI/ML models update in IoT application
    
       While IoT devices deployed in an IoT environment continuously collect
       and transmit the necessary data to an IoT server, the IoT server
       delivers intelligent IoT services using AI/ML models, based on these
       collected data. As IoT devices monitor a wide variety of IoT
       environmental information such as time-relevant and place-relevant
       information, the data gathered from IoT devices changes significantly
       depending on the events occurred in an IoT environment. For this
       reason, complete information is no longer reflected during AI/ML
       training and this, sooner or later, affects the quality of IoT
       services provided. Therefore, AI/ML models must be updated
       periodically upon the data subsequently collected. However, if the
       models are updated in a too short period, not only the high
       management (or labor) cost for updating is required, but also it may
       detrimentally affect the overall performance of the models. Therefore,
       models should be updated by fairly considering between the cost
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 19]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       required for updating AI/ML models and the merits gained from the
       usage of updated AI/ML models.
    
    4.4. Requirements for distributed computing for AI/ML-based IoT services
    
       The AI/ML-based IoT services consist of large-scale computations. IoT
       devices themselves may not have enough computing power to handle
       these large-scale computations. Therefore, it is necessary to
       overcome the limitations of computing power of IoT devices by
       distributing computational workload across gateways, cloudlets, and
       clouds. Since there are many challenges in distributing computational
       workloads, the following factors must be considered:
    
       (Energy Consumption) In most AI/ML-based IoT services, data collected
       from IoT devices is transmitted to the edge or cloud. However, since
       IoT devices have limited battery life and it is impractical to charge
       them frequently, it is necessary to consider the energy consumption
       of IoT devices in distributed computing. The energy consumption of
       IoT devices is caused by many factors such as uplink data
       transmission and the amount of computation. In order to minimize the
       energy consumption of IoT devices, those factors should be considered
       in distributed computing.
    
       (Latency) In distributed computing, overall latency consists of many
       factors such as communication delay, queuing delay, and processing
       time. If all factors are not considered properly, overall latency may
       increase. Therefore, it is necessary to reduce the overall latency by
       considering various delay factors.
    
       (Communication Overhead) In order to distribute the computational
       workload, it is necessary to exchange input/output data and
       intermediate results between nodes. In other words, additional data
       should be moved through the network, which leads to communication
       overhead. How to reduce this overhead should also be considered.
    
       (Data Privacy) Data from IoT devices must be transmitted to other
       nodes for distributed processing. At this time, if the data to be
       sent is related to the user's personal information or sensitive data,
       the user will be reluctant to send the data. In this regard, the
       consideration of data privacy is required for distributed computing.
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 20]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       Also, from a technical point of view, distributing computational
       workloads requires the following technologies:
    
       (Computing Node Selection) In order to distribute computational
       workloads, the appropriate target nodes must be selected. In general,
       it is necessary to select a node suitable for the purpose of
       distribution in consideration of node state information and
       connection information between nodes. In this case, queue status and
       communication bandwidth will be important factors for purpose of
       minimizing delay, and power consumption for transmission between
       nodes and battery status of nodes will be important factors for the
       purpose of minimizing energy consumption. If the purpose is to
       improve service availability, nodes?malfunction and failure rates
       will be critical considerations. In this way, it is necessary to
       determine the importance of factors to be considered according to the
       purpose of distribution and to select appropriate nodes based on this.
    
       (Cooperation between Computing Nodes) In order to minimize the
       latency of IoT applications, cooperation between computing nodes is
       essential. After the selection of possible computing nodes based on
       the IoT network status, sub-task partitioning and distributed task
       allocation among nodes are performed. In general, most AI/ML
       algorithms can be divided into several sub-tasks and the sub-tasks
       can be allocated to the appropriate computing nodes based on several
       factors such as computational capacity, and network bandwidth.
       However, while allocating several sub-tasks among nodes, not only
       latency but also data privacy and energy consumption should be
       considered. For example, in order to protect user data privacy, it is
       necessary to allocate an initial part of AI/ML algorithms to the end
       device. Also, for reducing energy consumption, it is necessary to
       minimize the amount of computation in the end device as much as
       possible and reduce the amount of uplink data transmission. While
       considering the above several issues, the optimally distributed task
       allocation techniques which aim to minimize latency are required.
    
       (Network Slicing) Slice coordination is necessary for efficient
       management of network slicing. Because different slices have
       different service requirements, a balanced allocation of physical
       resources such as frequency bandwidth, cache memory, and backhaul
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 21]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       link capacity between slices is important to guarantee requirements
       associated with IoT devices. For a balanced allocation of physical
       resources, not only the amount of feasible physical resources but
       also the requirements and traffic characteristics of IoT devices
       should be considered. For example, massive IoT devices, in which the
       size of each packet tends to be short, require a low outage
       probability for massive connection and this can be achieved with
       HARQ-based grant-free transmission with a relatively small amount of
       frequency bandwidth. On the other hand, mission-critical IoT devices
       require both low latency and high reliability. Although grant-free
       transmission can reduce the latency caused by grant scheduling
       overhead, such transmission is vulnerable to preemption. To lower the
       probability of preemption, securing a sufficient amount of frequency
       bandwidth is essential compared to the previous case. The task of
       allocating the physical resources considering the requirements and
       traffic characteristics of numerous IoT devices becomes more critical
       in an environment where traffic characteristics and requirements vary
       over time. Because of these considerations, the numerical solution
       cannot be computed. Therefore, learning from historical data becomes
       important and AI/ML algorithms can be applied to cope with the
       overall problems to intelligently manage network slicing.
    
    5. State of arts of the artificial intelligence/machine learning
       technologies for IoT services
    
       In this section, well-known machine learning and artificial
       intelligence technologies applicable to IoT applications are reviewed.
    
    5.1. Machine learning and artificial intelligence technologies review
    
       The classical machine learning models can be divided into three types,
       supervised, unsupervised, and reinforcement learnings. Therefore, in
       this subsection, machine learning and artificial intelligence
       technology reviews are done in four different categories: supervised,
       unsupervised, reinforcement, and neural-network-based.
    
    5.1.1. Supervised learning for IoT
    
       Supervised learning is a task-based type of machine learning, which
       approximates function describing the relationship and causality
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 22]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
       between input and output data. Therefore, the input data needs to be
       clearly defined with proper output data since supervised learning
       models learn explicitly from direct feedback.
    
       (K-Nearest Neighbor) Given a new data point in K-Nearest Neighbor
       (KNN) classifier, it is classified according to its K number of the
       closest data points in the training set. To find the K nearest
       neighbors of the new data point, it needs to use a distance metric
       which can affect classifier performance, such as Euclidean,
       Mahalanobis or Hamming. One limitation of KNN in applying for IoT
       network is that it is unscalable to large datasets because it
       requires the entire training dataset to classify a newly incoming
       data. However, KNN required less processing power capability compared
       to other complex learning methods.
    
       (Naive Bayes) Given a new data point in Naive Bayes classifiers,
       it is classified based on Bayes' theorem with the "naive" assumption
       of independence between the features. Since Naive Bayes classifiers
       don't need a large number of data points to be trained, they can deal
       with high-dimensional data points. Therefore, they are fast and
       highly scalable. However, since its "naive" assumptions are somewhat
       strong, a certain level of prior knowledge on the dataset is required.
    
       (Support Vector Machine) Support Vector Machine (SVM) is a binary and
       non-probabilistic classifier which finds the hyperplane maximizing
       the margin between the classes of the training dataset. SVM has been
       the most pervasive machine learning technology until the study on
       neural network technologies are advanced recently. However, SVM still
       has advantages over neural network based and probabilistic approaches
       in terms of memory usage and capability to deal with high-dimensional
       data. In this manner, SVM can be used for IoT applications with
       severe data storage constraint.
    
       (Regression) Regression is a method for approximating the
       relationships of the dependent variable, which is being estimated,
       with the independent variables, which are used for the estimation.
       Therefore, this method is widely used for forecasting and inferring
       causal relationships between input data and output data in time-
       sensitive IoT application.
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 23]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       (Random Forests) In random forests, instead of training a single
       decision tree, a group of trees is trained. Each tree is trained on a
       subset of the training set using a randomly chosen subset of M input
       variables. Random forests considering various tree structures have
       very high accuracy, so it can be utilized in the accuracy-critical
       IoT applications.
    
    5.1.2. Unsupervised learning for IoT
    
       Unsupervised learning is a data-driven type of machine learning which
       finds hidden structure in unlabeled dataset without feedback during
       the learning process. Unlike supervised learning, unsupervised
       learning focuses on discovering patterns in the data distributions
       and gaining insights from them.
    
       (K-means clustering) K-means clustering aims to assign observations
       into K number of clusters in which each observation belongs to the
       cluster having the most similarities. The measure of similarity is
       the distance between K cluster centers and each observation. K-means
       is a very fast and highly scalable clustering algorithm, so it can be
       used for IoT applications with real-time processing requirements such
       as smart transportation.
    
       (Density-based spatial clustering of applications with noise)
       Density-Based approach to Spatial Clustering of Applications with
       Noise (DBSCAN) is a method that clusters dataset based on the density
       of its data samples. In this model, dense regions which include data
       samples with many close neighbors are considered as clusters, and
       data samples in low-density regions are classified as outliers
       [Kriegal]. Since this method is robust to outliers, DBSCAN is
       efficient data clustering method for IoT network environments with
       untrusted big datasets in practice.
    
       (Dimensionality Reduction) Dimensionality reduction aims to transform
       high-dimensional data into low-dimensional data while minimizing
       information loss. In other words, it serves to extract some features
       that can best represent data among the features. By using
       dimensionality reduction, we can reduce the complexity of the model
       and shorten the computation time. Examples of dimensionality
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 24]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       reduction include principal component analysis (PCA), T-stochastic
       neighbor embedding (T-SNE).
    
    5.1.3. Reinforcement learning for IoT
    
       Reinforcement learning is a reactive type of machine learning that
       learn a series of actions in a given set of possible states, actions,
       and rewards or penalties. It can be seen as the exploring decision-
       making process and choosing the action series with the most reward or
       the least penalty which can be cost, priority, time to name a few.
       Reinforcement learning can be helpful for selecting action of IoT
       device by providing a guideline.
    
       (Q-learning) Q-Learning is a model-free, off-policy reinforcement
       learning algorithm based on the well-known Bellman Equation. The goal
       is to learn an action-selection policy maximizing the Q-value, which
       tells an agent what action to take. It can be used for IoT device to
       determine which action it should take according to conditions.
    
       (State-Action-Reward-State-Action) Though State-Action-Reward-State-
       Action (SARSA) is a much similar algorithm to Q-learning, the main
       difference is that it is an on-policy algorithm in which agent
       interacts with the environment and updates the policy based on
       actions taken. It means that the Q-value is updated by an action
       performed by the current policy instead of the greed policy that
       maximizes Q-value. In this perspective, it is relevant when an action
       of one IoT device will greatly influence the condition of the
       environment.
    
       (Deep Q Network) Deep Q network (DQN) is developed to solve the
       exploration problem for unseen states. In the case of Q-learning, the
       agent is not capable of estimating value for unseen states. To handle
       this generality problem, DQN leverages neural network technology. As
       a variation of the classic Q-Learning algorithm, DQN utilizes a deep
       convolutional neural net architecture for Q-function approximation.
       In real environments not all possible states and conditions are not
       able to be observed. Therefore, DQN is more relevant than Q-learning
       or SARSA in real applications such as IoT. Since DQN could be used
    
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 25]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       within only discrete action space, it can be utilized for traffic
       routing in the IoT network.
    
       (Deep Deterministic Policy Gradient) DQN has solved generality and
       exploration problem of the unseen or rare states. Deep Deterministic
       Policy Gradient (DDPG) takes DQN into the continuous action domain.
       DDPG is a deterministic policy gradient based actor-critic, model-
       free algorithm. The actor decides the best action for each state and
       critic is used to evaluate the policy, the chosen action set. In IoT
       applications, DDPG can be utilized for the tasks that require
       controlled in continuous action spaces, such as energy-efficient
       temperature control, computation offloading, network traffic
       scheduling, etc.
    
       (Trust Region Policy Optimization) The concept of Trust Region Policy
       Optimization (TRPO) is to update parameters within Trust Region. In
       order to update the parameters of the new policy within the defined
       region, the parameters need to be updated with samples obtained from
       local points by solving conservative policy iteration. Moreover, the
       value of step size is constrained to update parameters within the
       defined Trust Region In IoT communication networks, considering
       that actions of TRPO are confidently decided within its boundary,
       TRPO can be implemented for load balancing, power allocation, service
       provisioning, etc.
    
       (Proximal Policy Optimization) Unlike TRPO, Proximal Policy
       Optimization (PPO) uses the first derivative to update the parameters
       of its policy. The computation complexity of PPO is much alleviated
       compared to that of TRPO. While the step size of TRPO is constantly
       adjusted to satisfy its constraint, the step size of PPO simply uses
       clipping to fix the maximum value of its update. For exploration and
       exploitation, PPO computes action entropy to determine whether an
       agent requires more exploration to maximize its objective function.
       PPO is suitable for training agents in an environment where action
       space is too large for DQN algorithm. Therefore, tasks such as user
       association, power control, and resource allocation in wireless
       communications can benefit from PPO.
    
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 26]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
    5.1.4. Neural Network based algorithms for IoT
    
       (Recurrent Neural Network) Recurrent Neural Network (RNN) is a
       discriminative type of supervised learning model that takes serial or
       time-series input data. RNN is specifically developed to address
       issue of time dependency of sequential time-series input data. It
       processes sequences of data through internal memory, and it is useful
       in IoT applications with time-dependent data, such as identifying
       time-dependent patterns of sensor data, estimating consumption
       behavior over time, etc.
    
       (Long Short Term Memory) As an extension of RNN, Long Short Term
       Memory (LSTM) is a discriminative type of supervised learning model
       that is specialized for serial or time-series input data as well
       [Hochreiter]. The main difference of LSTM from RNN is that it
       utilizes the concept of gates. It actively controls forget gates to
       prevent the long term time dependency from waning. Therefore,
       compared to RNN, it is more suitable for data with long time
       relationship and IoT applications requiring analysis on the long lag
       of dependency, such as activity recognition, disaster prediction, to
       name a few [Chung].
    
       (Convolutional Neural Network) Convolutional neural network (CNN) is
       a discriminative type of supervised learning model. It is developed
       specifically for processing 2-dimensional image data by considering
       local connectivity, but now generally used for multidimensional data
       such as multi-channel sound signals, IoT sensor values, etc. As in
       CNN neurons are connected only to a small subset of the input and
       share weight parameters, CNN is much more sparse compared to fully
       connected network. However, it needs a large training dataset,
       especially for visual tasks. In CNN, a new activation function for
       neural network, Rectified Linear Unit (ReLU), was proposed, which
       accelerates training time without affecting the generalization of the
       network [Krizhevsky]. In IoT domains, it is often used for detection
       tasks that require some visual analysis.
    
       (Variational Autoencoder) Autoencoder (AE) is a generative type
       of  unsupervised learning model. AE is trained to generate output to
       reconstruct input data, thus it has the same number of input and
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 27]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       output units. It is suitable for feature extraction and
       dimensionality reduction. Because of its behavior to reconstructing
       the input data at the output layer, it is often used for machinery
       fault diagnosis in IoT applications. The most popular type of AE,
       Variational Autoencoder (VAE) is a generative type of semi-supervised
       learning model. Its assumptions on the structure of the data are weak
       enough for real applications and its training process through
       backpropagation is fast [Doersch]. Therefore, VAE is suitable in IoT
       applications where data tends to be diverse and scarce.
    
       (Generative Adversarial Network) Generative Adversarial Network (GAN)
       is a hybrid type of semi-supervised learning model which contain two
       neural networks, namely the generative and discriminative networks
       [Goodfellow]. The generator is trained to learn the data distribution
       from a training dataset in order to generate new data which can
       deceive the latter network, so-called the discriminator. Then, the
       discriminator learns to discriminate the generated data from the real
       data. In IoT applications, GAN can be used in situations when
       something needs to be generated from the available data, such as
       localization, way-finding, and data type conversion.
    
    5.2. Technologies for lightweight and real-time intelligence
    
       As the era of IoT has come, some sort of light-weight intelligence is
       needed to support smart objects. Prior to the era of IoT, most of the
       works on learning did not consider resource-constrained environments.
       Especially, deep learning models require many resources such as
       processing power, memory, stable power source, etc. However, it has
       been recently shown that the parameters of the deep learning models
       contain redundant information, so that some parts of them can be
       delicately removed to reduce complexity without much degradation of
       performance [Ba], [Denil]. In this section, the technologies to
       achieve real-time and serverless learning in IoT environments are
       introduced.
    
       (network compression) Network compression is a method to convert a
       dense network into a sparse one. With this technology the network can
       be reduced in its size and complexity. By pruning irrelevant parts or
       sharing redundant parameters, the storage and computational
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 28]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       requirements can be decreased [Han]. After pruning, the performance
       of the network is examined and the pruning process is repeated until
       the performance reaches the minimum requirements for the specific
       applications and use cases. As many parameters are removed or shared,
       the memory required is reduced, as well as computational burden and
       energy. Especially as most energy in neural network is used to access
       memory, the consumed energy dramatically drops. Although its main
       limitation is that there is not a general solution to compress all
       kinds of network, but it rather depends on the characteristics of
       each network. However, network compression is still the most
       widespread method to make deep learning technologies to be
       lightweight and IoT-friendly.
    
       (approximate computing) Approximate computing is an approach to
       support deep learning in smart devices [Venkataramani], [Moons]. It
       is based on the facts that the results of deep learning do not need
       to be exact in many IoT applications but still valid if the results
       are in an acceptable range. By integrating approximate computing into
       deep learning, not only the execution time but also the energy
       consumption is reduced [Mohammadi]. Based on the optimal trade-off
       between accuracy and run-time or energy consumption, the network can
       be adjustably approximated. The network approximate technology can be
       well-used in such situations when the response time is more important
       than sophisticatedly analyzed results. Although it is a technology to
       facilitate real-time and lightweight intelligence, the process of
       training models and converting it to approximate network require some
       amount of resource. Therefore, the approximated model can be deployed
       on smart devices but the learning and approximation processes still
       need to take places on resource rich platforms.
    
       (early exit inference) Deep learning models take advantages of
       nonlinear properties, from the forwarding process through a large
       number of layers, but often allow inferences with a reasonable level
       of accuracy without going through all the layers. Starting with this
       idea, early exit points in the middle of a layer has been studied to
       determine the reliability of the calculation results at that point,
       and if they have a reasonable level of reliability, they are no
       longer calculated. These technologies, which can achieve quick
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 29]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       inference at the expense of some accuracy, can be useful in making
       decisions which are not fatal but require quick responses in
       resource-constrained IoT environments.
    
    5.3. Federated learning
    
       Federated learning is a method by which multiple IoT devices work
       together to train a single predictive model. Based on this, lower
       latency, less power, and smarter models can be obtained, and there
       are also advantages in terms of privacy. Federated learning allows
       mobile phones to jointly train predictive models to keep all data on
       the device, allowing machine learning to be performed without the
       need to store data in the cloud, making it well-suited for applying
       artificial intelligence in IoT devices.
    
       (Horizontal Federated Learning) Horizontal federated learning is
       mainly used when all training models are the same and each user has
       the same data type. The clients update individually, and the server
       collects all local updates, calculates global updates based on the
       aggregated information and sends them to all users. It is mainly used
       in technologies such as voice assistants in smart homes.
    
       (Federated Transfer Learning) Federated transfer learning, unlike the
       horizontal federated learning mentioned above, aims to train in
       situations with different data sets and different sample spaces. Send
       multiple data from different spaces, transfer them in the same space,
       and train the model. This is likely to be used in situations such as
       collaborations in hospitals with other patients.
    
       (Centralized Federated Learning) Centralized federated learning
       trains on a central server and trains all clients in parallel in each
       round of communication. The central server employs a weighted average
       algorithm such as federated averaging (FedAvg) and then distributes
       the weights to each client. Servers are critical to managing clients
       by aggregating and distributing updates and protecting privacy.
    
       (Decentralized federated learning) Decentralized federated learning
       is a network topology without a central server, and all learning
       subjects are connected through P2P. In every round of communication,
       clients are trained based on their own data and consensus on global
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 30]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       updates is reached based on models received from neighboring clients.
       Based on this, it can be used even when there is no communication
       with the server and can be applied to technologies such as blockchain.
    
    6. Use cases of AI/ML into IoT service
    
       Many IoT service environments are equipped with camera, door lock,
       temperature sensor, fire detector, gas detector, alarm, and so on.
       Each sensor is deployed with particular purposes of each own to
       provide a specific service. However, there is a problem that the
       sensor utilization is not high enough due to the provision of the
       service using only a single sensor rather than multiple sensors and
       their mutual relations. Therefore, the quality of the service
       provided is not high as well. To enhance the sensor utilization and
       the service quality, all signals from various sensors should be
       analyzed comprehensively to make the right decision. This section
       describes the use cases for introducing AI / ML techniques in actual
       IoT service, utilizing multiple sensors. In advance of each use case
       description of various IoT service domains, characteristics of each
       domain to adopt AI/ML techniques are investigated.
    
    6.1. Surveillance and Security in Smart Home
    
       To minimize users' inconvenience and ensure their safety,
       surveillance and safety IoT applications provided within smart homes
       require fast notification with good level of precision IoT service
       quality for abnormal conditions detection. To do this, both data
       preprocessing techniques and AI/ML technologies for analysis of
       anomalies with high accuracy will be required.
    
    6.1.1. Characteristics of Smart Home for AI/ML processing
    
       (Training Data Generation) For Surveillance and Security, the
       processed data is necessary because there is little data for
       anomalies and the ground truth values are hardly available. Therefore,
       first, the steps to detect and calibrate the abnormal data are
       essential before the anomaly data should be generated using domain
       knowledge. First, constructing simulators about targeted smart home
       and generating events against external intrusions and then collecting
       the anomaly data can be considered. Furthermore, based on the data
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 31]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       collected in the actual environment, anomaly data generation can
       proceed by breaking the relationship between sensors considering
       possible links between them within any intrusive environment.
    
       (AI/ML Algorithm) One of the characteristics of the IoT environment
       for surveillance and safety is that a massive amount of data is
       collected and real-time responses are required. For the kNN algorithm,
       since the more data sets, the stronger against the noise and the
       higher the accuracy. If the appropriate dataset is used, the fast
       response can be expected. It makes suitable for the service
       environment to be considered. In addition, considering real-time data
       forecasting and analysis via LSTM, it is believed that improved
       accuracy for real-time anomalies detection can be expected.
    
    6.1.2. Use case
    
       (To be continued)
    
    6.2. Smart home for elderly welfare
    
       In the currently existing smart home IoT system for elders, it is
       difficult to detect exact physical and mental health status from IoT
       data because it is a very complicated problem mixed with various
       factors such as depression levels, heart rates, surroundings, etc. If
       health status can be detected correctly, it could be used for various
       single living-elderly care services. To do this, various techniques
       will be required such as protecting user's privacy and recognizing
       complex situation.
    
    6.2.1. Characteristics of Smart Home for elderly welfare
    
       IoT devices for in-home healthcare operations need to be small and
       unobtrusive. This form factor limitation means that these devices are
       also energy constrained. At the same time, the devices need to be
       ultra-reliable and provide timely and accurate data, coupled with
       processing capabilities to extract vital information. However, it is
       not an easy task for single-living elders to manage the status of all
       IoT devices in the house. To support reliable smart healthcare
       services for elders, the trade-offs between the energy-efficiency of
       the devices and the reliability of the collected data should be
       properly considered. Additionally, to manage and handle various
       emergency situations, AI/ML-based real-time data monitoring and
       automatic behavior recognition, sophisticated situation aware
       technology should be required.
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 32]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
    6.3. Smart Building Management
    
       Smart buildings often consist of heterogeneous IoT devices. These
       devices cooperate and their data is integrated for efficient
       autonomous building management. Many of the events in a large
       building may not require deep, complicated learning or processing.
       Some of them may require a fast response than an accurate analysis.
       Above all, a lot of events simultaneously occur and can arise heavy
       loads on the main server. The edge-computing techniques can be used
       to offload the main server's tasks.
    
    6.3.1. Characteristics of Smart Building for AI/ML processing
    
       (Training Data Generation) In smart buildings, heterogeneous IoT
       devices are deployed. They are diverse in their types, functions,
       performances, etc. To utilize the data from diverse devices, data
       needs to be able to well-integrated. Therefore, it is better for data
       to be in a common data format, or it needs to be able to transform
       into one another. The other characteristic is that the IoT devices
       may interact in local and global environments of the building.
       Therefore, the scope of the dataset used in training can play a
       critical role in developing AL/ML model for building management.
    
       (AI/ML Algorithm) To offload and reduce the burden of the main server
       and to provide fast, efficient decision makings, the IoT and the
       other network-related devices can use their computing resources.
       Various edge-computing techniques can be applied to do so, such as
       developing light-weighted AI/ML models that can be easily deployed in
       the edge devices or balancing the learning and processing computation
       load from the server to the edge devices.
    
    6.3.2. Use case
    
       (To be continued)
    
    6.4. Manufacturing optimization in Smart factory
    
       When IoT meets smart factory Industry, Industrial Internet of Things
       (IIoT) technology integrating various technologies such as AI/ML, big
       data analytics, massive sensor connection, Machine to Machine
       communication (M2M) and automation are generally utilized. Its IIoT
       applications has initiated a profound change within companies making
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 33]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       them increasingly connected and allowing them to exploit data to
       optimize their production processes, since IIoT is able to support
       real-time monitoring of each process line, thanks to different types
       of sensors positioned in critical points.
    
       However, to anticipate and prevent possible failures, the health
       state of assets is always monitored, which generating tremendous
       volume of data, and instantons and intelligent responses to
       unexpected behavior are required.
    
    6.4.1. Characteristics of Smart Factory for AI/ML processing
    
       (To be continued)
    
    6.4.2. Use case
    
       (Real-time predictive maintenance) It is in needs to monitor the
       autonomous machines deployed in the manufacturing plants and forecast
       any maintenance requirements in prior to the machine failure.
    
       (Demand Forecasting) With forecasting, future demand can be predicted
       to meet the targeted delivery deadlines in advance. This is one of
       the widespread strategies adopted across the industry where a number
       of manufacturing plants are presently transforming from reactive to
       predictive inventory management.
    
    6.5. Smart parking
    
       Parking operators are nowadays in the process of transforming their
       business model to satisfy a more demanding end-user, by looking for
       new technologies that allow the enhancement of drivers?user-
       experience. These drivers are currently facing mobility issues in
       urban areas, where the correlation of traffic congestion in cities is
       highlighted with drivers looking for parking spaces. As a consequence,
       current drivers are spending more time searching for a free place,
       contributing to higher fuel consumption and increased environmental
       pollution. Therefore, intelligent smart parking system is required to
       collect and process real-time parking occupancy information.
    
    6.5.1. Characteristics of Smart Parking for AI/ML processing
    
       In smart parking system, any end-user has access to real-time parking
       occupancy information both through urban furniture (information
       displays) and mobile/automobile apps, reducing the time each driver
       spends looking for a parking space. Moreover, parking operators can
       enhance their ongoing operations by knowing in every instant what is
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 34]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       happening and by analyzing reports on the behavioural patterns
       detected by the platform, improving their operational and enforcement
       efficiency. To achieve this, the usage of a video camera will be
       required to monitor in real time the available slots in a parking
       park. Additionally, the media acquired from the deployed cameras will
       be processed using image recognition AI/ML model allowing to perceive
       in real time the status of each parking spot. A lightweight AI/ML
       model will be initially used to support real-time processing, however,
       if any potential change is detected (a parking slot was potentially
       filled up for instance) more powerful AI/ML models can be utilized to
       confirm/dismiss the potential change. It can be also confirmed which
       entity is occupying the slot (car, bike, truck person, etc.)
    
    6.6. Augmented Reality (AR) in IoT
    
       Augmented reality technologies enable visualizing digital information
       and combining it with the physical world. As massive data are
       generated from IoT, visualization of digital information helps
       monitoring the status of circumstance and understanding the IoT
       system. AR techniques can be applied to several IoT applications such
       as autonomous driving, smart manufacturing industries. In such
       applications,  massive IoT data are collected from IoT sensors
       including cameras, temperature & humidity sensors, and should be
       processed and analyzed in real-time.
    
    6.6.1. Characteristics of Augmented Reality for AI/ML processing
    
       As augmented reality techniques usually detect surrounding objects
       and interact with the physical world in real-time, it is
       computational-intensive and delay-sensitive. Also, the augmented
       reality devices which visualize digital information are usually
       mobile devices which have limited battery life. Therefore, augmented
       reality techniques should consider both latency minimization and
       energy consumption minimization. To achieve these goals, a
       lightweight AI/ML model would be used and edge computing techniques
       can be used to offload tasks to the edge server. However, as lots of
       data collected by IoT sensors, the data transmission time can be a
       critical issue in satisfying the latency restriction. Therefore, to
       support augmented reality in IoT systems, distributed and cooperative
       edge computing techniques which can distribute tasks at the edge
       server and mobile devices would be needed.
    
    6.7. Smart Farm
    
       Smart farm is a convenient and efficient agricultural form that
       manages the growing environment in an optimal state without time and
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 35]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       space restrictions by using automated facilities and information and
       communication technology. The most common example of smart farm
       implementation is a method in which sensors installed throughout the
       farm collect various information to help managers make decisions. An
       intelligent smart farm is required to process incoming information in
       real time and make decisions.
    
    6.7.1. Characteristics of Smart Farm for AI/ML processing
    
       Smart Farm collects meteorological information such as temperature,
       humidity, CO2, and solar radiation from IoT devices. In Smart Farm,
       various types of time series data are received sporadically, and
       compared to other IoT services, real time processing is not necessary.
       Data obtained from multiple sensors generally come in different time
       slots, and these data are combined into a single input. This combined
       input can be designed to control air conditioners and heaters, open
       and close windows, and feed nutrients using the AI/ML model.
    
    7. IANA Considerations
    
       This document requests no action by IANA.
    
    8. Acknowledgements
    
    9. Contributors
    
    10. Informative References
    
       [Hochreiter]S. Hochreiter and J. Schmidhuber, "Long short-term
                 memory," Neural Comput., vol. 9, no. 8, pp. 1735-1780, Nov.
                 1997.
    
       [Chung] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, "Empirical
                 evaluation of gated recurrent neural networks on sequence
                 modeling," arXiv preprint arXiv:1412.3555v1 [cs.NE], 2014.
    
       [Krizhevsky]A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet
                 classification with deep convolutional neural networks," in
                 Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1097-1105..
                 Hochreiter and J. Schmidhuber, "Long short-term memory,"
                 Neural Comput., vol. 9, no. 8, pp. 1735-1780, Nov. 1997.
    
       [Doersch] C. Doersch, "Tutorial on variational autoencoders," arXiv
                 preprint arXiv:1606.05908v2 [stat.ML], 2016.
    
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 36]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       [Goodfellow]I. Goodfellow et al., "Generative adversarial nets," in
                 Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 2672-2680.
    
       [Ba]    J. Ba and R. Caruana, "Do deep nets really need to be deep?"
                 in Proc. Adv. Neural Inf. Process. Syst., Montreal, QC,
                 Canada, 2014, pp. 2654-2662.
    
       [Denil] M. Denil, B. Shakibi, L. Dinh, N. de Freitas, and M. Ranzato,
       "Predicting parameters in deep learning," in Proc. Adv. Neural Inf.
       Process. Syst., 2013, pp. 2148-2156.
    
       [Han]   S. Han, J. Pool, J. Tran, and W. Dally, "Learning both
                 weights and connections for efficient neural network," in
                 Proc. Adv. Neural Inf. Process. Syst., Montreal, QC, Canada,
                 2015, pp. 1135-1143.
    
       [Venkataramani]S. Venkataramani, A. Ranjan, K. Roy, and A.
                 Raghunathan, "AxNN: Energy-efficient neuromorphic systems
                 using approximate computing," in Proc. Int. Symp. Low Power
                 Electron. Design, ACM, 2014, pp. 27-32. [Moons]S.
                 Hochreiter and J. Schmidhuber, "Long short-term memory,"
                 Neural Comput., vol. 9, no. 8, pp. 1735-1780, Nov. 1997.
    
       [Moons] B. Moons, B. De Brabandere, L. Van Gool, and M. Verhelst,
                 "Energy- efficient ConvNets through approximate computing,"
                 in Proc. IEEE Winter Conf. Appl. Comput. Vis. (WACV), Lake
                 Placid, NY, USA: IEEE, 2016, pp. 1-8.
    
       [Mohammadi] Mohammadi, Mehdi, et al. "Deep learning for IoT big data
                 and streaming analytics: A survey," IEEE Communications
                 Surveys & Tutorials, 2018, pp. 2923-2960.
    
       [Kriegel]  Kriegel, HansPeter, et al. "Densitybased clustering,"
       Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery,
       2011, pp. 231-240.
    
    
    
    
    
    
    
    
    
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 37]


    Internet-Draft    IoT with Artificial Intelligence            Dec 2022
    
    
    
       Authors' Addresses
    
       Jun Kyun Choi (editor)
       Korea Advanced Institute of Science and Technology (KAIST)
       193 Munji Ro, Yuseong-gu, Daejeon, Korea
    
       Email: jkchoi59@kaist.ac.kr
    
    
       Jae Seob Han
       Korea Advanced Institute of Science and Technology (KAIST)
       193 Munji Ro, Yuseong-gu, Daejeon, Korea
    
       Email: j89449@kaist.ac.kr
    
    
       Gyeong Ho Lee
       Korea Advanced Institute of Science and Technology (KAIST)
       193 Munji Ro, Yuseong-gu, Daejeon, Korea
    
       Email: gyeongho@kaist.ac.kr
    
    
       Hyun Seo Park
       Korea Advanced Institute of Science and Technology (KAIST)
       193 Munji Ro, Yuseong-gu, Daejeon, Korea
    
       Email: tkf92001@kaist.ac.kr
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    Choi, et.al.            Expires June 28, 2023                [Page 38]