Internet Engineering Task Force                                   J. Hui
Internet-Draft                                       Cisco Systems, Inc.
Intended status: Informational                                J. Vasseur
Expires: September 29, 2011                           Cisco Systems, Inc
                                                          March 28, 2011


      Routing Architecture in Low-Power and Lossy Networks (LLNs)
                   draft-routing-architecture-iot-00

Abstract

   The IETF ROLL Working Group has specified a routing protocol designed
   for the Low power and Lossy Networks (LLN) also referred to as IP
   smart objects networks or the "Internet of things".  Still, the
   debate of where routing functions should occur within the network
   stack tend to get revived on a regular basis.  A mesh-under approach
   places routing functions in the link layer to emulate a single
   broadcast domain where all devices appear as immediate neighbors to
   the network layer.  In contrast, a route-over approach places all
   routing functions at the network layer, following the IP
   architecture.

   For LLNs, a mesh-under approach may seem simple and attractive
   because it seeks to hide characteristics of multi-hop communication
   through the LLN from the network layer.  However, resource
   constraints and dynamic link characteristics limit to what extent
   link-layer routing can hide those characteristics.  This document
   presents architectural issues of using a mesh-under approach in LLNs
   and how a route-over approach does not suffer from these issues.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 29, 2011.




Hui & Vasseur          Expires September 29, 2011               [Page 1]


Internet-Draft        Routing Architecture in LLNs            March 2011


Copyright Notice

   Copyright (c) 2011 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  The Ethernet Abstraction . . . . . . . . . . . . . . . . . . .  4
   3.  The Mesh-Under Fantasy . . . . . . . . . . . . . . . . . . . .  5
   4.  Multi-Layer Routing Issues . . . . . . . . . . . . . . . . . .  6
     4.1.  Multi-Layer Path Computation . . . . . . . . . . . . . . .  7
     4.2.  Multi-Layer Recovery . . . . . . . . . . . . . . . . . . .  7
     4.3.  Incompatible Features  . . . . . . . . . . . . . . . . . .  8
     4.4.  Implementation Overhead  . . . . . . . . . . . . . . . . .  8
     4.5.  Management Overhead  . . . . . . . . . . . . . . . . . . .  8
   5.  Additional Mesh-Under Issues . . . . . . . . . . . . . . . . .  8
     5.1.  Insufficient Address Scoping . . . . . . . . . . . . . . .  9
     5.2.  Nondeterministic Link Characteristics  . . . . . . . . . . 10
     5.3.  Lack of Visibility into Link Topology  . . . . . . . . . . 10
   6.  Routing Only at the IP Layer . . . . . . . . . . . . . . . . . 11
   7.  Route-Over and Link-Layer Forwarding . . . . . . . . . . . . . 12
   8.  Lessons from the Past  . . . . . . . . . . . . . . . . . . . . 12
   9.  Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 13
   10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 14
   11. Security Considerations  . . . . . . . . . . . . . . . . . . . 14
   12. Informative References . . . . . . . . . . . . . . . . . . . . 14
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 16












Hui & Vasseur          Expires September 29, 2011               [Page 2]


Internet-Draft        Routing Architecture in LLNs            March 2011


1.  Introduction

   A decade ago, both academia and industry viewed the use of Internet
   Protocol (IP) as ill-suited for use in Low-Power and Lossy Networks
   (LLNs).  Unlike traditional IP networks, LLNs must operate under
   severe resource constraints (e.g. limited memory, computation, and
   communication) and handle the lossy and dynamic nature of low-power
   link technologies (e.g.  IEEE 802.15.4).  For this reason, the LLN
   community felt it was necessary to free themselves of the IP
   architecture and revisit assumptions and develop new networking
   solutions.

   Since those beginnings, the LLN field has matured substantially.  A
   variety of protocols have been invented and evaluated.  Significant
   networks have been deployed for a variety of real-world applications.
   But because much of those efforts were designed outside of the IP
   architecture, connecting those LLNs back to an existing IP-based
   infrastructure (both private IP networks and the public Internet)
   involved the use of application gateways that translate between the
   IP and non-IP worlds.  Unfortunately, such gateways are known to be
   difficult to design, deploy, and maintain - mapping between protocols
   often required significant functional and semantic translation (e.g.
   routing, quality of service, and security); state management made the
   gateway a single point of failure; and upgrades to application
   protocols required careful synchronization between the gateway and
   end devices.  In effect, gateways limit the innovation and growth
   that the IP architecture was designed to foster.

   To address these concerns, the IETF began to focus work on the use of
   IP in LLNs by forming the IPv6 over Low-Power Wireless Personal Area
   Networks (6LoWPAN) working group to standardize the use of IPv6 in
   IEEE 802.15.4 networks.  The IETF then formed the Routing over Low
   Power and Lossy Links (ROLL) to specify the Routing Protocol for LLNs
   (RPL) [I-D.ietf-roll-rpl]), an IP-layer routing protocol designed
   specifically for LLNs.  The IETF also formed the Constrained RESTful
   Environments working group to specify the Constrained Application
   Protocol (CoAP) [I-D.ietf-core-coap], an application protocols in
   LLNs.  All of these IETF working groups focused their efforts
   exclusively on IPv6, rather than IPv4, in anticipation of the IPv4
   address space depletion and to take advantage of new features in
   IPv6, such as autoconfiguration.

   A number of standards organizations have placed significant effort
   into standardizing networking protocols for LLNs, including
   ZigBee/IP, Wavenis, and IEEE.  All of these efforts began outside of
   the IP architecture when IP was viewed as ill-suited for use in LLNs.
   However, most of these organizations, including ZigBee/IP, Wavenis,
   and IEEE P1901.2, are now focusing their efforts to standardize IP-



Hui & Vasseur          Expires September 29, 2011               [Page 3]


Internet-Draft        Routing Architecture in LLNs            March 2011


   based architectures for LLNs.

   Today, it is safe to say that IPv6 has won the battle over non-IP-
   based protocols in LLNs.  However, from time to time, the debate on
   the role that routing plays within an LLN gets revived.  In one view,
   LLNs should implement link-specific networking protocols and add IP
   as a minimal shim below the application layer.  This so-called mesh-
   under approach performs routing below the IP layer and, in effect,
   all devices appear as immediate neighbors to the network layer.  In
   another view, LLNs should place all routing functions at the IP layer
   and none in the link layer, following the IP architecture.  This so-
   called route-over approach equates each physical hop as a single IP
   hop.  RPL supports the route-over approach because it is an IPv6
   network layer routing protocol designed for LLNs.

   The purpose of this document is to preset the architectural issues
   that arise with a mesh-under approach.  These issues have caused
   leading standards organizations to specify a route-over approach,
   including the IETF, ZigBee, and Wavenis.  In fact, no industry
   standard today places routing functions at the link layer.


2.  The Ethernet Abstraction

   Originally based on a single shared media, Ethernet provides a simple
   communication abstraction.  In particular, Ethernet provides the
   following properties:

   o  Single broadcast domain: All devices appear directly attached to
      the same broadcast medium and the IP layer can transmit a datagram
      to any number of devices attached to the same link.  More
      formally, all communication is transitive within a single
      broadcast domain (if A can send to B and B can send to C, then A
      can send to C).

   o  Deterministic link characteristics: any differences in link
      characteristics (e.g. communication latency, throughput, and loss
      rates) between different node pairs attached to the same link are
      insignificant.  From a routing perspective, the link cost rarely
      varies.

   o  High reliability: the link delivers messages to their intended
      destination with relatively high reliability.

   A link that provides these Ethernet properties is attractive because
   it simplifies the IP layer.  At the network layer, any device can
   send IP datagrams to any number of devices attached to the same link
   simply by indicating the unicast or multicast address of the



Hui & Vasseur          Expires September 29, 2011               [Page 4]


Internet-Draft        Routing Architecture in LLNs            March 2011


   destination(s).  The network layer never needs to perform routing
   functions when sending IP datagrams to other devices attached to the
   same link.  The network layer also does not need to consider
   differences in link characteristics.

   The most common WiFi deployments are infrastructure-based, where WiFi
   access points provide network connectivity for WiFi clients.  Due to
   the limited range of wireless communication and security policies,
   WiFi clients are often restricted to communicating only with an
   access point.  However, WiFi maintains the Ethernet abstraction by
   implementing link-layer routing functions on the access point.  Even
   in enterprise deployments that involve multiple access points
   connected by an Ethernet backbone, the entire WiFi network appears as
   a single broadcast domain.

   The ubiquity of the Ethernet abstraction has led critical IP-layer
   protocols, such as IPv6 Neighbor Discovery (ND) [RFC4861], to assume
   it.  For example, ND uses Duplicate Address Detection (DAD) to verify
   the uniqueness of an address, it sends a single multicast query to
   all devices in the network.  When ND checks that a device is
   reachable, it assumes that error rates are low and round-trip
   communication latency is deterministic.  Furthermore, when ND
   performs default router selection, it does not take into account the
   link cost of reaching the default router and assumes that link
   characteristics are uniform.


3.  The Mesh-Under Fantasy

   The mesh-under approach places mesh routing functions in the link
   layer in attempt to support the Ethernet abstraction.  Some argue
   that mesh-under provides the following advantages:

   o  Maintain existing non-IP networking solutions: a huge amount of
      time, money, and effort has been invested in existing solutions
      that do not adhere to the IP architecture.  As a result, one
      possible way to quickly obtain an IP-based solution is to simply
      transport IP datagrams over an existing non-IP networking solution
      for LLNs.

   o  Hide LLN complexities within the link layer: compared to more
      traditional link technologies, LLNs can exhibit complex topologies
      over a large physical extent combined with the lossy and dynamic
      nature of LLN link technologies.  Successfully hiding such
      complexities within the link layer allows the IP layer to ignore
      them.





Hui & Vasseur          Expires September 29, 2011               [Page 5]


Internet-Draft        Routing Architecture in LLNs            March 2011


   o  Routing outside the IP architecture: by operating at the link
      layer, the routing protocol does not need to conform to the IP
      architecture.

   As we will see later in this document, these properties of a mesh-
   under solution actually cause more technical challenges and risks
   than they solve, in addition to weakening the overall layered
   architecture of IP.  Even so, initial efforts to bring IP to LLNs
   focused on mesh-under solutions.  In 2007, the IETF 6LoWPAN working
   group set the foundation for a mesh-under solution by publishing RFC
   4944 that specifies the encoding IPv6 datagrams in IEEE 802.15.4
   frames [RFC4944].  The focus on mesh-under led to the following
   mechanisms in RFC 4944:

   o  Mesh Addressing Header: this header allows a packet to identify
      the end-points of a path using IEEE 802.15.4 addresses and enables
      link-layer routing and forwarding mechanisms to deliver IPv6
      datagrams over multiple radio hops.

   o  Broadcast Header: to support IPv6 multicast, the broadcast header
      carries a sequence number to support a simple PAN-wide flooding
      mechanism to emulate a single broadcast domain.

   o  Link-Local Address Compression: the IPv6 header compression
      mechanism is only capable of reducing the size of link-local IPv6
      addresses, making the assumption that devices will communicate
      primarily with link-local addresses.  At the same time, much of
      the forward looking efforts within the 6LoWPAN working group
      focused on filling out a mesh-under approach.  Proposals for link-
      layer routing [I-D.daniel-6lowpan-load-adhoc-routing], IPv6 ND
      optimizations [I-D.chakrabarti-6lowpan-ipv6-nd], and network
      bootstrap [I-D.daniel-6lowpan-commissioning] protocols all assumed
      a mesh-under approach.

   Initial efforts outside the IETF also focused on mesh-under
   solutions.  However, in light of the architectural issues with mesh-
   under presented in this document, most of those efforts have now
   adopted a route-over approach using RPL, the standardized IPv6
   routing protocol for LLNs by the IETF ROLL Working Group.


4.  Multi-Layer Routing Issues

   Much of the IP architecture's success is due to its layered approach,
   allowing each layer to evolve independently and build networks that
   are composed of a variety of different link technologies.  Although
   IEEE 802.15.4 was recently seen as the link technology for LLNs,
   other link technologies are beginning to proliferate (e.g.  Low Power



Hui & Vasseur          Expires September 29, 2011               [Page 6]


Internet-Draft        Routing Architecture in LLNs            March 2011


   WiFi, Wavenis, and Power Line Communication).  Each link technology
   targets different deployment environments, but also brings their own
   sets of communication characteristics.  The most effective
   deployments combine multiple link technologies into a single IP
   network.  Of course, this is a primary reason for the shift in
   consensus towards utilizing IP in LLNs.

   To interconnect multiple networks made of a variety of link layers
   (including LLNs), an IP routing protocol is needed.  As a result, a
   mesh-under approach would require routing functions operating at both
   the link and network layers.  While the IP-layer routing protocol
   computes end-to-end paths, the link-layer routing protocol computes
   paths within a link network.  As a result, the IP layer sees each
   other device in the same link network as neighbors and has limited,
   if any, visibility into the path computation to reach those devices.

   In this section, we discuss additional architectural issues that
   arise with multi-layer routing.

4.1.  Multi-Layer Path Computation

   A mesh-under approach makes it challenging for an IP routing protocol
   to compute an effective end-to-end path.  Strict adherence to the
   layered architecture would imply that the link-layer routing protocol
   does not have visibility in the cost of paths that extend beyond the
   link network.  Similarly, the IP-layer routing protocol does not have
   visibility into the path costs within a link network because all
   paths through the link network appear as direct IP neighbors.

   One approach is for the link and IP routing protocols to share
   dynamic path cost information through cross-layer APIs.  However, for
   the two protocols to make effective use of the information, they must
   implement the same objective functions, metrics, and constraints when
   optimizing routes.  It is not sufficient for them to optimize paths
   with similar goals, such as minimum latency or transmission cost.
   Even time-constant differences in low-pass filters used to compute
   metrics could cause inefficient operation and end-to-end path
   convergence issues due to unintended interactions with multi-layer
   recovery.

4.2.  Multi-Layer Recovery

   In the presence of physical connectivity changes due to link or node
   failures, it is challenging to coordinate recovery operations across
   routing protocols operating at different layers.  Complex
   interactions can occur when the different routing protocols
   independently notice and react to a link failure simultaneously.  The
   link-layer may detect a link failure and perform recovery operations



Hui & Vasseur          Expires September 29, 2011               [Page 7]


Internet-Draft        Routing Architecture in LLNs            March 2011


   to repair the path to a destination.  However, if the path is not
   restored quickly enough, the IP routing protocol may determine that a
   failure has occurred and it should begin searching for a new route.

   A commonly proposed solution is to use a bottom-up timer-based
   approach, where the higher layer must delay its rerouting operations
   for some time T. However, choosing an appropriate value of T can be
   non-trivial, as it effectively sets an upper bound on the link-
   layer's convergence time and a lower bound on the overall system's
   convergence time.  In other words, a value too large will increase
   the system's convergence time, and a value too small may lead to
   concurrent rerouting operations.

4.3.  Incompatible Features

   Significant issues arise when the link- and IP-layer routing
   protocols implement different functionalities.  In such cases, they
   must drop to the least common denominator.  For example, if the link
   layer routing protocol does not implement Multi-Topology Routing or
   constraint-based routing, the IP routing protocol will not be able to
   utilize similar functions effectively.  Differences in the selection
   of metrics, their computation, or their usage can make it difficult
   to effectively optimize a path end-to-end.  As mentioned before, such
   differences can also exacerbate issues with multi-layer recovery.

4.4.  Implementation Overhead

   The complex interactions of multi-layer path computation and recovery
   and feature incompatibilities lead to complex implementations that
   require significant computing and memory resources (e.g.  IP over ATM
   or DWDM).  In contrast, LLNs must operate with strict resource
   constraints (e.g. limited memory, computation, and communication).
   Furthermore, due to their embedded nature, LLNs must operate
   unattended for many years.

4.5.  Management Overhead

   Dealing with multiple routing protocols is operationally challenging.
   Network administrators must be educated and prepared to configure,
   operate, and debug different link-layer routing protocols for each
   link technology in addition to the IP-layer routing protocol that
   binds all the link technologies together.


5.  Additional Mesh-Under Issues

   There is a common mindset within the LLN community that a given LLN
   will operate as a stub network and utilize only a single link



Hui & Vasseur          Expires September 29, 2011               [Page 8]


Internet-Draft        Routing Architecture in LLNs            March 2011


   technology.  This is understandable, especially considering that many
   existing LLN networking standards began outside of the IP
   architecture and were not designed to inter-network with other LLNs
   or traditional networks.  Although there is momentum in adopting a
   route-over approach, some believe that the simplest path towards
   incorporating an IP-based architecture is with a mesh-under approach
   that transports IP datagrams over an existing mesh networking
   technology.  However, even with this simplistic view of LLNs, a
   number of architectural issues arise.

5.1.  Insufficient Address Scoping

   The IPv6 addressing architecture supports the notion of address
   scopes.  Each IPv6 address encodes a scope that defines the
   topological span within which the address is a unique identifier for
   a set of interfaces.  Unicast addresses have either link-local or
   global scope.  The former uniquely identifies interfaces attached to
   the same link and the latter uniquely identifies interfaces anywhere
   on the Internet.  In either case, the smallest address scope that can
   identify interfaces attached to the same link is link-local.  This is
   understandable since the IP layer assumes that any device attached to
   the same link is an immediate neighbor.

   Because all devices in the same network appear attached to the same
   link with a mesh-under approach, a link-local IP address is
   effectively limited to an identifier for any arbitrary device or set
   of devices in the network.  A mesh-under approach expands link-local
   scope to include the entire LLN.  In contrast, a route-over approach
   limits link-local scope to those devices reachable by the physical
   layer.  As a result, the link-layer in a mesh-under approach must be
   prepared to discover routes to arbitrary devices in the network when
   receiving any datagram from the IP layer.  In other words, all IP
   datagrams that the IP layer sends to the link layer may trigger non-
   trivial routing mechanisms at the link layer.

   The problem with mesh-under is that IP-based protocols have no
   mechanism to pass any additional topological knowledge they may have
   about the destination.  The best they can do is say whether or not
   the destination(s) are restricted to devices in the network.  In LLN
   applications, there are a number of practical scenarios where the
   application has better knowledge on the location of the device.
   Furthermore, LLN applications tend to take advantage of their spatial
   correlation and are more likely to communicate with devices in close
   physical proximity.

   IP-based protocols typically use link-local multicast communication
   to discover devices attached to the same link.  IPv6 Neighbor
   Discovery uses multicast for discovering devices attached to the same



Hui & Vasseur          Expires September 29, 2011               [Page 9]


Internet-Draft        Routing Architecture in LLNs            March 2011


   link.  But because the link spans the entire LLN with mesh-under, the
   link layer must be prepared to deliver a single multicast message to
   all devices in the network.  Whether multicast is implemented at the
   link layer using simple flooding or more sophisticated multicast
   trees, delivering a message to many devices in a network is a costly
   operation.

5.2.  Nondeterministic Link Characteristics

   IP-based protocols often assume that link characteristics are
   deterministic when communicating with any device attached to the same
   link.  However, using link-layer routing techniques to maintain
   deterministic link characteristics with a mesh-under routing approach
   is nearly impossible.  Communication latency, throughput, and
   reliability can vary greatly between different devices depending on
   their location in the network and variations in traffic load or link
   qualities.  Limited throughput, channel capacity, and memory make
   communication properties highly sensitive to the network topology and
   variations in network conditions.  At first glance, it may seem
   simple enough to provide hints to the IP layer on current
   communication properties to another device in the network.

   The challenge comes in managing the interactions between link and IP
   layer protocols.  The responsibility of a link-layer routing protocol
   is to discover and maintain paths for destinations within the
   network.  Changes in physical connectivity may trigger the link-layer
   routing protocol to repair the route.  If the link layer is unable to
   repair the route quickly enough, the IP layer may prematurely
   determine that a destination is unreachable.  This can reduce the
   reliability of IPv6 ND Neighbor Unreachability Detection or Address
   Resolution.

   A more significant interaction arises in deployments that involve
   multiple default routers.  IPv6 ND is also responsible for
   dynamically selecting a default route.  Using Neighbor Unreachability
   Detection, IPv6 ND can detect reachability issues and change its
   default route if necessary.  If the link layer is unable to repair a
   route to the default router in time, IPv6 ND may prematurely
   determine the router is unreachable and select a different default
   router.  However, doing so will cause the link layer to configure
   routes to the new default router, causing additional control traffic
   and wasting any effort in repairing the route to the old default
   router, a typical multi-layer routing architecture challenge.

5.3.  Lack of Visibility into Link Topology

   Due to the resource constraints of LLNs, there has been significant
   work in the use of overlays to support in-network processing within



Hui & Vasseur          Expires September 29, 2011              [Page 10]


Internet-Draft        Routing Architecture in LLNs            March 2011


   LLNs.  Overlays are often proposed to reduce communication latency,
   channel utilization, and energy consumption.  An inherent requirement
   for building an effective overlay in LLNs is that devices must be
   able to discover and utilize the link connectivity to maximize the
   benefits of using such mechanisms.

   A mesh-under routing approach makes it impossible to support
   effective overlays using IP-based protocols.  In performing data
   aggregation, for example, devices must be able to process message
   payloads as they forward them along to their destination.  But by
   routing and forwarding datagrams at the link layer, forwarding
   devices never pass IP datagrams up to the IP layer.  While it may be
   possible to form an IP datagram destined for the next aggregation
   point at each hop, it is not possible to do so in a way that mirrors
   the physical topology.


6.  Routing Only at the IP Layer

   A route-over approach places routing functions only at the IP-layer,
   following the IP architecture.  In doing so, a route-over approach
   fully exposes the link connectivity to the IP layer.  Device's IP
   neighbors are those devices within physical transmission range of its
   transceiver.  As a result, the LLN is composed of multiple
   overlapping broadcast domains (one per device) and communication is
   non-transitive within the LLN.  If an IP datagram must traverse one
   or more devices to reach a destination, then routing occurs at the IP
   layer.

   With the route-over approach, all routing functions are implemented
   by a single routing protocol at the IP layer (e.g.  RPL for LLN),
   even when the LLN is composed of multiple different link
   technologies.  As a result, the route-over approach does not suffer
   from issues of multi-layer path computation and recovery,
   incompatible functionality, and increased implementation and
   management costs associated with supporting multiple layers of
   independent routing protocols.

   Futhermore, route-over does not suffer from architectural issues due
   to insufficient address scope, nondeterministic link statistics, and
   lack of visibility into physical topology.  IPv6-based application
   protocols can use link-local addresses to limit the scope of
   addressable devices to immediate neighbors or the IPv6 Hop Limit
   field to control the topological extent of a message.  Furthermore,
   with the ability to discover the physical link topology, IP-based
   applications can implement in-network processing.  Finally, because
   the IP link is constrained to those devices within physical range,
   any differences in link characteristics are independent of network



Hui & Vasseur          Expires September 29, 2011              [Page 11]


Internet-Draft        Routing Architecture in LLNs            March 2011


   diameter or a link-layer routing protocol.

   For many of the reasons mentioned above, focus shifted towards
   supporting a route-over approach.  Recently, the 6LoWPAN standards
   were updated to support a route-over approach.  In particular, IPv6
   header compression for 6LoWPANs was updated to support for IPv6
   extension headers, such as the Hop-by-Hop Options
   [I-D.ietf-6man-rpl-option] and Routing headers
   [I-D.ietf-6man-rpl-routing-header].  Specifications for the use of
   IPv6 ND in 6LoWPANs were also updated with both the route-over and
   mesh-under approaches in mind.

   To support IP routing within LLNs, the IETF Routing Over Low Power
   and Lossy Networks (ROLL) working group specified Routing Protocol
   for LLNs (RPL), an IP routing protocol designed specifically for
   LLNs.  RPL is a distance vector routing protocol designed to meet the
   requirements of scale (e.g. thousands of resource-constrained
   devices), low-power and lossy links that provide limited throughput
   and have highly variable link characteristics, and limited memory and
   computation resources.  Today, standards organizations, including
   ZigBee/IP and Wavenis, have adopted a route-over approach using RPL.


7.  Route-Over and Link-Layer Forwarding

   Note that while a route-over approach places all routing functions at
   the IP layer, it does not require forwarding functions to occur only
   at the IP layer.  In particular, a route-over approach allows
   forwarding IP datagrams at the link layer.  This is done today in
   more traditional IP-based networks with the combination of IP-layer
   routing protocols (e.g.  OSPF [RFC5340] or IS-IS [RFC1142] and Multi-
   Protocol Label Switching (MPLS) [RFC3031].  An advantage of
   forwarding datagrams at the link layer is that each forwarding device
   does not need to process IP header to forward packet.  This may be
   especially interesting in LLNs to reduce the overhead of encoding a
   path when source routing.  Furthermore, the use of Label Switch Paths
   allow networks to implement traffic engineering or VPN (Virtual
   Private Networks) functions.


8.  Lessons from the Past

   Emulating a single broadcast domain was largely successful with
   Ethernet and WiFi due to their abundant resources and simple network
   topologies.  There is little issue in providing subnet-wide multicast
   and both Ethernet and WiFi can easily and reliably deliver any
   subnet-wide multicast traffic generated by the network layer.
   Furthermore, there is little variation in link-layer communication



Hui & Vasseur          Expires September 29, 2011              [Page 12]


Internet-Draft        Routing Architecture in LLNs            March 2011


   properties between different pairs off nodes within the subnet.  In
   particular, a host need not consider the link topology in selecting
   among multiple default routers.  A host simply assumes that the link
   layer can provide the same level of service regardless of which
   default router it chooses.  In other words, unlike resource-
   constrained LLNs, Ethernet and WiFi take advantage of their abundant
   resources and simple network topologies to make any differences in
   communication properties insignificant to IP.  As a result, it was
   acceptable to hide those differences from the IP layer.

   The mesh-under approach in IP networks have not always been
   successful, especially with more complex link technologies.  Applying
   a mesh-under approach to asynchronous transfer mode (ATM) using the
   PNNI routing protocol did not take hold because they could neither
   adequately hide nor give necessary visibility into the complex
   dynamics of connection-based communication.  In particular, efforts
   to support a mesh-under approach in ATM turned out to be extremely
   complex, requiring heavy processing on the nodes with a limited
   success due to a lack of end-to-end path cost visibility, the co-
   existence of routing protocols (e.g.  OSPF or IS-IS) with different
   objective functions and routing metrics, issues related to multi-
   layer recovery, not to mention the complexity to operate and manage
   two routing protocols.

   As with mesh-under in ATMs, the resource constraints and complex link
   dynamics typical to LLNs make it very difficult and inefficient to
   effectively apply a mesh-under approach.  Variability in
   communication latency, throughput, and reliability grows
   significantly with the network size in both hops and diameter.
   Indeed, this would imply that a mesh-under approach may be successful
   in a LLN with a handful of devices communicating through a single LLN
   Border Router.  However, LLNs are becoming more complex, utilizing
   multiple LLN Border Routers, larger network topologies in nodes and
   physical extent, as well as different link technologies.  For this
   reason, adopting a route-over approach is the only viable option that
   does not suffer from the architectural issues presented in this
   document.


9.  Conclusion

   The IP architecture has proven to be flexible and robust even while
   sustaining remarkable grown and innovation of the past three decades.
   IP's layered architecture has allowed each layer to evolve
   independently of other layers while still allowing cross-layer
   optimizations through standard APIs.  The application of the IP
   architecture to LLNs is just another stage in the evolution of IP
   networks to support the "Internet of Things."



Hui & Vasseur          Expires September 29, 2011              [Page 13]


Internet-Draft        Routing Architecture in LLNs            March 2011


   LLNs are evolving to incorporate a number of low-power and lossy link
   technologies (e.g.  IEEE 802.15.4, Low Power WiFi, and Power-Line
   Communication) in addition to more capable ones (e.g.  Ethernet and
   WiFi) and grow in size (both density and diameter).  The use of
   multiple link-layer technologies requires IP-layer routing to route
   across different link domains.  Combining IP-layer routing with link-
   layer routing in a mesh-under approach raises very significant
   challenges in dealing with multi-layer path computation and recovery,
   inconsistent functionality, and increased implementation and
   management costs.

   Significant architectural issues exist even in cases where no IP-
   layer routing occurs (e.g. a stub LLN operating with a single link
   technology).  To effectively operate in the resource constraints and
   lossy links of LLNs, applications and the IP-based protocols that
   implement them must have the ability to constrain the scope of
   addressing devices and how far a message may propagate.  They must
   have visibility into link characteristics that can vary due to
   network density, diameter, and dynamic internal and environmental
   conditions.  They must have the ability to distinguish immediate
   neighbors from other devices in the LLN.

   In this document, we presented a number of architectural issues that
   arise with a mesh-under approach and why a route-over approach does
   not suffer from these issues.  As a result, a route-over approach
   provides the best path forward to an efficient and scalable Internet
   of Things.


10.  IANA Considerations

   This document includes no request to IANA.


11.  Security Considerations

   This document includes no security considerations.


12.  Informative References

   [I-D.chakrabarti-6lowpan-ipv6-nd]
              Chakrabarti, S. and E. Nordmark, "LowPan Neighbor
              Discovery Extensions",
              draft-chakrabarti-6lowpan-ipv6-nd-05 (work in progress),
              July 2008.

   [I-D.daniel-6lowpan-commissioning]



Hui & Vasseur          Expires September 29, 2011              [Page 14]


Internet-Draft        Routing Architecture in LLNs            March 2011


              Kim, K., Lim, C., Yoo, S., Park, S., and G. Mulligan,
              "Commissioning in 6LoWPAN",
              draft-daniel-6lowpan-commissioning-02 (work in progress),
              July 2008.

   [I-D.daniel-6lowpan-load-adhoc-routing]
              Park, S., "6LoWPAN Ad Hoc On-Demand Distance Vector
              Routing (LOAD)",
              draft-daniel-6lowpan-load-adhoc-routing-03 (work in
              progress), June 2007.

   [I-D.ietf-6man-rpl-option]
              Hui, J. and J. Vasseur, "RPL Option for Carrying RPL
              Information in Data-Plane Datagrams",
              draft-ietf-6man-rpl-option-02 (work in progress),
              February 2011.

   [I-D.ietf-6man-rpl-routing-header]
              Hui, J., Vasseur, J., Culler, D., and V. Manral, "An IPv6
              Routing Header for Source Routes with RPL",
              draft-ietf-6man-rpl-routing-header-02 (work in progress),
              March 2011.

   [I-D.ietf-core-coap]
              Shelby, Z., Hartke, K., Bormann, C., and B. Frank,
              "Constrained Application Protocol (CoAP)",
              draft-ietf-core-coap-05 (work in progress), March 2011.

   [I-D.ietf-roll-rpl]
              Winter, T., Thubert, P., Brandt, A., Clausen, T., Hui, J.,
              Kelsey, R., Levis, P., Pister, K., Struik, R., and J.
              Vasseur, "RPL: IPv6 Routing Protocol for Low power and
              Lossy Networks", draft-ietf-roll-rpl-19 (work in
              progress), March 2011.

   [RFC1142]  Oran, D., "OSI IS-IS Intra-domain Routing Protocol",
              RFC 1142, February 1990.

   [RFC3031]  Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
              Label Switching Architecture", RFC 3031, January 2001.

   [RFC4861]  Narten, T., Nordmark, E., Simpson, W., and H. Soliman,
              "Neighbor Discovery for IP version 6 (IPv6)", RFC 4861,
              September 2007.

   [RFC4944]  Montenegro, G., Kushalnagar, N., Hui, J., and D. Culler,
              "Transmission of IPv6 Packets over IEEE 802.15.4
              Networks", RFC 4944, September 2007.



Hui & Vasseur          Expires September 29, 2011              [Page 15]


Internet-Draft        Routing Architecture in LLNs            March 2011


   [RFC5340]  Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF
              for IPv6", RFC 5340, July 2008.


Authors' Addresses

   Jonathan Hui
   Cisco Systems, Inc.
   170 West Tasman Drive
   San Jose,   95134
   USA

   Email: jonhui@cisco.com


   JP Vasseur
   Cisco Systems, Inc
   11, Rue Camille Desmoulins
   Issy Les Moulineaux,   92782
   France

   Email: jpv@cisco.com





























Hui & Vasseur          Expires September 29, 2011              [Page 16]