DTN Research Group V. Cerf
INTERNET-DRAFT Worldcom/Jet Propulsion Laboratory
<draft-irtf-dtnrg-arch-00.txt> S. Burleigh
March 2003 A. Hooke
Expires September 2003 L. Torgerson
NASA/Jet Propulsion Laboratory
R. Durst
K. Scott
The MITRE Corporation
K. Fall
Intel Corporation
H. Weiss
SPARTA, Inc.
Delay-Tolerant Network Architecture
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This document was produced within the IRTF's Delay Tolerant
Networking Research Group (DTNRG). See http://www.dtnrg.org
Abstract
This document describes an architecture for delay-tolerant networks,
and is a generalization of the architecture designed for the
Interplanetary Internet: a communication system to provide Internet-
like services across interplanetary distances in support of deep
space exploration. This generalization addresses networks with
operational and performance characteristics make conventional
networking approaches either unworkable or impractical. We define a
message-based overlay that exists above the transport layer of the
networks on which it is hosted. The document presents an
architectural overview followed by discussions of services, topology,
routing, security, reliability and state management.
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
Table of Contents
Status of this Memo................................................1
Abstract...........................................................1
Table of Contents..................................................2
1 Introduction................................................4
2 Why an Architecture for Delay-Tolerant Networking?..........4
3 DTN Architectural Description...............................5
3.1 The DTN Architecture is Based on Virtual Message
Switching..............................................5
3.2 DTN Classes of Service Mimic Postal Operation..........5
3.3 DTN Postal-Style Delivery Options......................6
3.4 Nodes..................................................7
3.5 Regions and Gateways...................................8
3.6 Tuples.................................................9
3.7 Late Binding...........................................9
3.8 Routing...............................................10
3.9 Bundle Fragmentation and Reassembly...................12
3.10 Bundle Layer Reliability and Custodianship............13
3.11 Time Synchronization..................................13
3.12 Congestion and Flow Control at the Bundle Layer.......14
3.13 Security..............................................15
4 State Management Considerations............................16
4.1 Demultiplexing and Binding State......................16
4.2 Bundle Retransmission State...........................17
4.3 Bundle Routing State..................................17
4.4 Security-Related State................................17
5 Bundle Header Information..................................18
6 Application Structuring Issues.............................19
7 Convergence Layer Considerations for Use of Underlying
Protocols..................................................20
8 Summary....................................................20
9 Informative References.....................................20
10 Security Considerations....................................21
11 Authors' Addresses.........................................22
12 Intellectual Property Notices..............................23
13 Copyright Notices..........................................23
Cerf, et al. Expires September 2003 [Page 2]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
Acknowledgments
John Wroclawski, David Mills, Greg Miller, James P. G. Sterbenz, Joe
Touch, Steven Low, Lloyd Wood, Robert Braden, Deborah Estrin, Stephen
Farrell and Craig Partridge all contributed useful thoughts and
criticisms to previous versions of this document. We are grateful
for their time and participation.
This work was performed in part under DOD Contract DAA-B07-00-CC201,
DARPA AO H912; JPL Task Plan No. 80-5045, DARPA AO H870; and NASA
Contract NAS7-1407.
Release Notes
draft-irtf-ipnrg-arch-00.txt, May 2001:
Original Issue.
draft-irtf-ipnrg-arch-01.txt, August 2002:
-Restructured document to generalize architecture for delay-tolerant
networking.
-Refined DTN classes of service and delivery options. Added a
"reply-to" address to have response information such as error
messages or end-to-end acks directed to a source-specified third
party.
-Further defined the topological elements of delay tolerant networks.
-Elaborated routing and reliability considerations.
-Initial model for securing the delay tolerant network
infrastructure.
-Expanded discussion of flow and congestion control.
-Added section discussing state information at the bundle layer.
-Updated bundle header information and end-to-end data transfer.
draft-irtf-dtnrg-arch-02.txt, March 2003:
-Revised model for delay tolerant network infrastructure security.
-Introduced fragmentation and reassembly to the architecture.
-Removed significant amounts of rationale and redundant text.
-Moved bundle transfer example(s) to separate draft(s).
Cerf, et al. Expires September 2003 [Page 3]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
1 Introduction
This document describes an architecture for Delay-Tolerant
interoperable networking. The architecture embraces the concepts of
occasionally-connected networks that suffer from frequent partitions
and that may be comprised of more than one divergent set of protocol
families. The basis for this architecture lies with that of the
Interplanetary Internet, which focused primarily on the issue of deep
space communication in high-delay environments. We expect the
current DTN architecture described here to be utilized in various
high-delay and unusual environments; the case of deep space is one of
these, and is still being pursued as a specialization of this
architecture (http://www.ipnsig.org). Other networks to which we
believe this architecture may apply include sensor-based networks
using scheduled intermittent connectivity, classes of terrestrial
wireless networks that cannot ordinarily maintain end-to-end
connectivity, and more "conventional" internets with long delays. A
DTN tutorial [FW03], aimed at introducing DTN and the types of
networks for which it is designed is available to introduce new
readers to the fundamental concepts and motivation.
The particular approach we employ is that of an end-to-end message-
based overlay that exists above the transport layers of the networks
on which it is hosted. The overlay employs intermediate storage at
message switches, and includes a hop-by-hop transfer of reliable
delivery responsibility known as "custody transfer." It also
includes a flexible naming format and a security model aimed at
protecting infrastructure from unauthorized use.
2 Why an Architecture for Delay-Tolerant Networking?
The reason for pursuing an architecture for delay tolerant networking
stems from several factors. These factors are summarized below; much
more detail on their rationale can be explored in [SB03,KF03,DFS02].
The existing Internet Protocols do not work well for some
environments. Some environments violate these inherent assumptions
in the TCP/IP approach:
- that an end-to-end path between source and destination exists for
the duration of the communication session;
- (for reliable communication) that the maximum round-trip time over
that path is not excessive and not highly variable from packet to
packet; and
- that the end-to-end loss is relatively small
- that all routers and end stations support the TCP/IP protocols
- that applications need not worry about communication performance
In light of these issues, the DTN architecture is conceived based on
a number of design principles that are summarized here (and further
discussed in [KF03], as mentioned above):
- use messages (not streams or packets) as the communication
abstraction
- encourage applications to minimize round-trip exchanges
Cerf, et al. Expires September 2003 [Page 4]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
- guide application design to cope with application restart after
failure while network transactions remain pending
- use storage within the network to support store-and-forward
operation over potentially long timescales (i.e. to support
operation in environments where no end-to-end path may ever exist)
- provide security mechanisms that protect the infrastructure from
unauthorized use; if appropriate, provide access to these
mechanisms by applications for use in their own application-level
security protocols
- provide a coarse-grained class of service and delivery options
based on services provided by the US (and other) postal systems
3 DTN Architectural Description
The previous section presented the design principles that guide the
definition of the DTN architecture. This section presents a
description of the design decisions that result from those design
principles.
3.1 The DTN Architecture is Based on Virtual Message Switching
A DTN transmits application-layer "bundles" that contain whatever the
requesting application wishes to send. Bundles are sent by and
delivered to applications in an atomic fashion, although they may be
split up during transmission. Bundles are also called "messages" in
this document. Message senders and recipients are identified by
(variable-length) source and destination names called tuples
(described below). Messages may also contain an optional reply-to
tuple used when special diagnostic operations are requested to direct
diagnostic output to an entity other than the sender. Generally,
messages are transferred in an overlay above the transport layer
called the "bundle layer."
A message-switched abstraction provides the network with a priori
knowledge of the size and performance requirements of requested data
transfers. When there is a significant amount of queuing that can
occur prior to transmission over an outbound route (as is the case in
the DTN version of store-and-forward) the advantage provided by
knowing this information may be significant for making scheduling
decisions. An alternative abstraction (i.e. of stream based
delivery) would make such scheduling very difficult. Although
packets provide some of the same benefits as messages, larger
aggregates provide a way for the network to apply scheduling and
buffer management to entire units of data that are useful to
applications.
3.2 DTN Classes of Service Mimic Postal Operation
The (U.S. or similar) Postal Service provides a strong metaphor for
the services that a Delay-Tolerant Network offers. Traffic is
generally not interactive. It may be one-way in nature. There are
generally no strong guarantees of timely delivery, yet there are some
forms of class of service and security.
Cerf, et al. Expires September 2003 [Page 5]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
The DTN Architecture, like the Postal Service, offers *relative*
measures of priority (low, medium, high). It may offer basic
notifications, for example: "the intended recipient has accepted
delivery of the message," "the route taken by this message is as
follows..." and "the message has been transmitted."
An essential element of Postal Service operation for networking is
that messages have a place to wait in queue until an outbound
communication opportunity is available. This highlights the
following assumptions:
1. that storage is available and well-distributed throughout the
network
2. that storage is sufficiently persistent and robust to store
messages until forwarding can occur, and
3. (implicitly) that this 'store-and-forward' model is a better
choice than attempting to effect continuous connectivity or other
alternatives
For a network to effectively support the DTN architecture, these
assumptions must be considered and must be found to hold.
We have currently defined three specific classes of service in the
DTN architecture:
- Bulk - In bulk class, bundles are shipped on a "best effort"
basis. No bulk class bundle will be shipped until all complete
bundles of other classes bound for the same next hop destination
have been shipped.
- Normal - Normal class bundles that are in queue and bound for the
same next hop destination are shipped prior to any complete bulk
class bundles that are in queue.
- Expedited - Expedited bundles, in general, are shipped prior to
bundles of other classes. However, the bundle layer *may*
implement a queue service discipline that prevents starvation of
the normal class, which may result in some normal bundles being
shipped before expedited bundles bound for the same next hop
destination as the normal class bundles.
Applications specify their requested class of service. This request,
coupled with policy applied at message switches, affects the overall
likelihood and timeliness of message delivery.
3.3 DTN Postal-Style Delivery Options
Applications may request any of the following five delivery options:
- Custodial Delivery - a bundle will be transmitted by the bundle
layer using reliable transport protocols (if available), and the
point of reliable delivery responsibility (i.e. retransmission
buffer) will advance through the network from one custodian to
another until the bundle reaches its destination. The bundle
Cerf, et al. Expires September 2003 [Page 6]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
layer depends on the underlying transport protocols of the
networks that it operates over to provide the primary means of
reliable transfer from one bundle layer to the next. However,
when custodial delivery is requested, the bundle layer
additionally provides a coarse-grained timeout and retransmission
mechanism and an accompanying (bundle-layer) hop-by-hop
acknowledgment mechanism. When a bundle application does *not*
request custodial delivery, this bundle layer timeout and
retransmission mechanism is not employed, and successful bundle
layer delivery depends solely on the reliability mechanisms of the
underlying transport layers
- Return Receipt - a return-receipt bundle is issued by the
receiving bundle layer implementation when the (arriving) subject
bundle is consumed *by the destination application* (not simply
the destination bundle layer). The receipt is issued to the
entity specified in the source tuple of the subject bundle or the
source's designated alternate (reply-to field), which would
typically be located on a different host. The return receipt uses
the same custodial delivery option setting used in the subject
bundle. (Return receipts are never issued with the return receipt
delivery option requested, to avoid "return receipt storms.")
- Forwarding Indication - sent by a bundle router when the last
fragment of a bundle has been forwarded. The indication is sent
to the reply-to destination if specified, and to the source of the
subject bundle otherwise
- Custody Transfer Indication - same as forwarding indication, but
sent when a custodial transfer has successfully completed
- Secured Delivery - indicates the application has provided
authentication material along with its message send request. To
operate under general circumstances, applications should be
prepared to supply authentication credentials and request secured
delivery. Local policy determines whether any bundles may be sent
lacking the security option, and regions beyond the originating
region may require security even if the originating region does
not.
3.4 Nodes
A DTN node (or "node" in this document) is an engine for sending and
receiving DTN messages (bundles). DTN nodes may act as sources,
destinations, or forwarders of bundles. Each node is uniquely
identified by at least one tuple containing a region name and an
entity name; a node that is reachable within multiple regions will
have at least one identifying tuple for each region in which it is
reachable.
The name for a DTN node itself, as opposed to an application *using*
that node, is specified in a region-specific manner using all or part
of the entity identifier. This is necessary for generating custody
acknowledgments to the bundle layer itself rather than to a specific
application.
Cerf, et al. Expires September 2003 [Page 7]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
3.5 Regions and Gateways
The DTN architecture defines a "network of internets" Comprised of
"DTN regions." Assignment of DTN nodes to particular regions is an
administrative decision, and may be influenced by differences in
protocol families, connection dynamics, or administrative policies.
Regions may also be delimited based upon other criteria, such as
trust boundaries [NEWARCH]. Each DTN region has a unique name that
is known (or knowable) among all regions of the DTN. Thus, a DTN-
wide repository for region names is required.
All inter-region communication takes place via DTN *gateways*, which
provide the interconnection points between regions. These correspond
to "waypoints" in [META], and to the gateways described in the
original ARPANET/Internet designs [CK74]. DTN gateways differ from
ARPANET gateways because they operate above the transport layer and
are focused on message switching rather than packet switching.
However, both DTN gateways and ARPANET gateways provide
interoperability between the protocols specific to one region and the
protocols specific to another.
Regions are key concepts in the DTN architecture. DTN bundles that
originate outside the destination region are first transmitted toward
one of the DTN gateways that connect the source region with one or
more other regions. Routing outside the destination region is based
solely on the destination region's name, and not on the completely
formed destination name (see below).
The following list identifies the requirements for DTN regions:
- Each DTN region shall have an identifier space that is shared by
all DTN nodes within the region. A region must specify the naming
conventions to be employed within that region for entity
identification.
- Each node that is a member of the region shall have a unique
identifier drawn from that identifier space. (Note that for some
types of regions, a "node" may be made up of a collection of
computational elements, possibly geographically distributed. A
single unique identifier may collectively refer to them. Further,
the unique identifier requirement only applies to nodes that are
intended to *receive* data from other DTN nodes.)
- To be considered a member of a region, each prospective member of
the region shall have the ability to reach every other member of
the region without depending on any DTN nodes outside the region
using some protocol(s) known or knowable to each node. (Although
a DTN node may not necessarily be *directly* reachable. This may
require forwarding and/or store-and-forward operation by other DTN
nodes inside the same region.)
Cerf, et al. Expires September 2003 [Page 8]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
3.6 Tuples
The region name described above (plus some forwarding state) is
necessary and sufficient to route a bundle of data to its destination
region, but not to deliver it to the specific endpoint(s) for which
it is intended. The DTN architecture uses a tuple comprised of the
region name and a region-specific entity name to identify a
particular set of entities in the DTN. The entity name is *opaque*
outside the region of definition. An entity might be a host, a
protocol, an application, some aggregate of these, or something else
depending on the nature of the addressing and naming structures used
in the containing region. In the Internet, for example, an entity
name could be as general as a URI or URL. In any case, some form of
binding mechanism (local to the containing region) is required to
associate communication endpoints (and their local addressing or
naming ID) with a DTN tuple. The details of the binding mechanism
are region-specific and not discussed here. However, such a
mechanism must provide a way for a requesting application to bind to
a prefix of a fully specified destination tuple.
Regions are named by applications using syntax identical to that used
in the domain name system (DNS). (That is, hierarchical tree
structure, dot-separated text node names, left to right progresses
from leaf to root, sibling nodes must have different names.) The
bundle layer may translate a region name to a bundle-layer-specific
region address for transmission. The scope of the region name space
(and region address space, if used) spans an entire DTN.
Discovery of valid DTN region names is the responsibility of bundle-
layer routing. Knowledge of appropriate entity identifiers is out of
the scope of this document, but corresponds directly to the Internet
problem of knowing port numbers a-priori.
3.7 Late Binding
The opacity of entity names outside their local region enforces
another key concept in the DTN Architecture: that of late binding.
Late binding refers to the fact that the entity name of a tuple is
not interpreted (e.g., used to form an address for delivery within
the region) outside its local region. This avoids having a universal
name-to-address binding space (and its associated database and
synchronization issues).
This approach preserves a significant amount of autonomy within each
region. The entity names used in bundles might be built on DNS
names, or URLs, but they might also be "expressions of interest" or
forms of database-like queries as in a directed diffusion-routed
network [IGE00] or intentional naming [WSBL99].
Additionally, late binding avoids the delay-related issue that the
binding information might be highly ephemeral relative to the transit
time of a bundle. We assume that the internal details of a
Cerf, et al. Expires September 2003 [Page 9]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
destination region will be sufficiently stable for the duration of a
transmission of a message within that region, or that delay-tolerant
mechanisms will be employed *within* the region to compensate. (This
is, by definition, a local issue within the region and may be
accommodated in whatever manner is most practical for that region.)
3.8 Routing
The bundle layer provides routing among DTN nodes, including DTN
gateways. There may be many DTN gateways that interconnect adjacent
regions, and there may be multiple bundle routing "hops" within a
region (particularly if intra-regional connectivity is intermittent).
The distinction between a router and a gateway relates to the late
binding of names, as described above. In particular, a region's
gateways are the first in an inter-region store-and-forward chain to
utilize the region-local entity identifier portion of tuples for
forwarding decisions. The DTN nodes are responsible for using
whatever reliability mechanisms exist in the underlying (transport-
and-below) layers, and augmenting those mechanisms with bundle-layer
mechanisms to implement the required reliability.
3.8.1 Types of Routes
DTNs may be required to function in the presence of any or all of the
following types of connectivity. Routes are comprised of a sequence
of "contacts" that indicate the duration, endpoints, and forwarding
capacity of a link in the topology graph. They are generally assumed
to be describing edges on a *directed* graph, as communication
capabilities cannot be assumed to be symmetric.
Persistent Contacts
Persistent contacts are edges with a neighboring DTN node that are
always available, with no connection establishment required. In the
IP world, a Digital Subscriber Line (DSL) or other "always-on"
connection is an example.
OnDemand Contacts
OnDemand contacts are contacts that require some action in order to
instantiate, but then otherwise function as persistent contacts until
terminated. Dial-up connections are an example of OnDemand contacts
(at least, from the viewpoint of the dialer; they may be viewed as an
Opportunistic Contact û below û from the viewpoint of the dial-up
service provider).
Intermittent - Scheduled Contacts
Scheduled contacts are those where there is an agreement to establish
a link between two points at a particular time, for a particular
duration. An example of a scheduled route is a link that uses a low-
earth orbiting satellite. A schedule of view times and durations can
Cerf, et al. Expires September 2003 [Page 10]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
be constructed when next-hop neighbors are accessible via the
satellite.
Intermittent - Opportunistic Contacts
Opportunistic contacts are those that are not scheduled, but rather
present themselves unexpectedly. Such contacts could be with an
aircraft flying overhead and beaconing, advertising its availability
for communication. Another type of opportunistic contacts might be
via infrared communication link between a personal digital assistant
(PDA) and a kiosk in an airport concourse offering bundle service as
the PDA's owner walks by. If the PDA's owner authorizes it, the PDA
could communicate the owner's planned path and a desire for contacts
with subsequent kiosks in the concourse, resulting in a set of
probable communication opportunities for which bundle transfers can
be scheduled.
Intermittent - Predicted Contacts
Predicted contacts are those that are based on no fixed schedule, but
rather a history of opportunistic contacts that suggests that contact
with an intermittently-connected neighbor will probably occur within
a certain period of time and will probably last for some inferred
duration. Given a great enough certainty that the contact will
occur, a DTN node may allocate bundles to that predicted contact
period that would be allocated to different contacts otherwise. In
the previous example, the probable contacts in the airport concourse
would result in the establishment of a set of predicted contacts of a
given duration (perhaps based on the PDA owner's walking speed and
the kiosk's coverage area). The PDA could decide how to use those
contacts for transmitting waiting bundles, as well as perhaps to
request bundles that were awaiting delivery at any of a number of
store-and-forward points to which the user had access.
The algorithms for establishing the predicted time and duration of a
contact, the degree of uncertainty in those estimates, the time at
which to abandon the wait for a predicted contact, and the guidelines
for allocating bundles to such contacts are all open research areas.
3.8.2 Bundle Storage for Store-and-forward operation
While IP networks are based on "store-and-forward" operation, there
is an assumption that the "storing" will not persist for more than a
modest amount of queuing and transmission delay. In contrast, the
DTN architecture does not expect that an outbound link will be
available when a bundle is received, and expects to store that bundle
for some time until a link does become available. We anticipate that
most DTN nodes will use some form of persistent storage for this --
disk, flash memory, etc.
All DTN forwarding nodes ("DTN routers"), even those that do not
provide custodial operations as described in section 3.3, must be
able to queue bundles until outbound contacts are available. Each
Cerf, et al. Expires September 2003 [Page 11]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
DTN node that is also willing to provide custody transfer operations
should provision for longer-term storage of bundles, committing to
store bundles for which it takes custody for the entire remainder of
their lifetimes, if necessary.
It is entirely possible that a custodian will need to "take a break"
from further custodianship while it completes pending custodial
operations and recovers persistent storage. Two mechanisms support
this: First, the node can simply forward incoming bundles without
taking custody. The fact that a node is a potential custodian is no
guarantee that it will take custody of any given bundle that is
routed to it. Second, the node can revise its advertisement of
custodial capability in routing updates. This is a longer-term
solution, but has the attractive property that DTN nodes searching
for a custodian do not route a bundle out of its way vainly in search
of custodianship at the node in question. Also, see section 3.12 on
DTN flow and congestion control.
The availability of long-term storage for bundles allows the next-hop
forwarding decision to potentially be revoked. In particular, if a
future contact is chosen to carry a bundle and some other superior
contact becomes known, the bundle could be re-assigned. Details of
this re-assignment operation are local to a particular bundle router
and influenced by the times at which contact schedules become known.
3.9 Bundle Fragmentation and Reassembly
There are two forms of fragmentation/reassembly in Bundling:
Any DTN router may ûproactively- choose to divide a block of data
into multiple self-identifying smaller blocks and transmit each
such block as a bundle. In this case the *final destination(s)*
are responsible for extracting the smaller blocks from incoming
bundles and reassembling them into the original large block. This
form of fragmentation is analogous to IP fragmentation.
A bundle router may ûreactively- choose to fragment a bundle on
receipt. This situation arises when a portion of a bundle may
have been received successfully. In this case, the receiving node
modifies the incoming bundle to indicate it is a fragment, and
forwards it normally. The previous-hop sender may learn that only
a portion of the bundle was delivered to the next hop, and
optimistically continue sending the remaining portion of the
original bundle when subsequent contacts become available.
Reactive fragmentation is specifically designed to handle cases in
which a router is faced with forwarding a bundle for which no
single contact provides sufficient data transfer volume.
Cerf, et al. Expires September 2003 [Page 12]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
3.10 Bundle Layer Reliability and Custodianship
The bundle layer provides an end-to-end reliable message delivery
service that employs in-network retransmission between (possibly non-
network-layer-adjacent) DTN nodes. The bundle layer makes use of the
reliability mechanisms available from the underlying transport layers
of each region, and a single bundle-layer hop *may* span an entire
region. The bundle layer supports end-to-end reliability derived
solely from the custody transfer mechanism, but also provides a true
end-to-end acknowledgement for application use. Applications wishing
to utilize this indication for their own end-to-end bundle
retransmission mechanisms are free to do so.
The bundle layer provides three types of delivery services, with
various levels of reliability-enhancing mechanisms: end-to-end
acknowledgment of a bundle, custodial transmission (with in-network
retransmission if required), and unacknowledged bundle transmission.
Custody transfer allows the source to delegate retransmission
responsibility and recover its retransmission-related resources
relatively soon after sending the bundle (on the order of a round-
trip time to the first bundle hop). While not every node in a DTN
must be capable of providing custodial services, all DTN routers
(that span networks that may be frequently disconnected) are expected
to be able to be custodians. This expectation supports custodial
operation along the primary path without forcing custodial bundles to
make routing diversions in order to locate a custodian.
3.11 Time Synchronization
The DTN architecture depends on time synchronization (supported by
external, region-local protocols) for two primary purposes: routing
with scheduled or predicted contacts and bundle "time to live"
computations.
Routing based on schedules and dependent upon coordination of shared
assets (such as directional antennas) creates a requirement for time
synchronization to achieve contact rendezvous.
Time to live computations are achieved by including a source time
stamp and an explicit time to live field (in time units after the
time in the source time stamp). Its sole use is for purging data
from the network, so the synchronization requirements posed here are
not strict. This approach allows a source time stamp to be used for
a number of purposes, such as unique identification of individual
messages from a particular source. DTN nodes must ensure that
timestamps in bundles they send never decrease.
Applications specify an expiration time (actually, a time to live in
seconds) for bundles they send. If not supplied, or if the user-
supplied value is larger than local policy permits, the bundle layer
will provide a value. Note that this value is treated as an actual
Cerf, et al. Expires September 2003 [Page 13]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
"time to live" -- it is added to the time that a bundle was submitted
by the application to determine the time at which the bundle will be
purged from the network. Appropriate values depend on the network
and the data, and could conceivably vary widely (e.g. from
milliseconds to weeks).
3.12 Congestion and Flow Control at the Bundle Layer
The subject of congestion control and flow control at the bundle
layer is one on which the authors of this document have not yet
reached consensus. We have unresolved concerns about the efficiency
and efficacy of congestion and flow control schemes implemented
across long and/or highly variable delay environments.
One view of congestion control is as follows: "Congestion control is
concerned with allocating the resources in a network such that the
network can operate at an acceptable performance level when the
demand exceeds or is near the capacity of the network resources.
These resources include bandwidths of links, buffer space (memory),
and processing capacity at intermediate nodes. Congestion occurs
when the demand is greater than the available resources." [RJ90]
For the purposes of this document, we define "flow control" as a
means of assuring that the rate at which a sending node transmits
data to a receiving node does not exceed the maximum rate at which
the receiving node is prepared to receive data from that sender.
(Note that this is a generalized notion of flow control, rather than
one that applies only to end-to-end communication. In particular,
the concept of flow control between the two ends of a single link may
be indispensable in such DTN regions as the "interplanetary
backbone.") We define "congestion control" as a means of assuring
that the aggregate rate at which all traffic sources inject data into
a network does not exceed the maximum aggregate rate at which the
network can deliver data to destination nodes. If flow control is
propagated backward from destination nodes to routers and eventually
back to traffic sources, then flow control can be at least a partial
solution to the problem of congestion as well.
DTN flow control decisions must be made within the bundling layer
itself based on information about resources (in this case, primarily
persistent storage) available within the bundle node. However, the
bundle layer *might* be able to delegate the implementation of those
decisions to the underlying transport protocols, as follows.
We have not yet considered multipoint communication at or below the
bundle layer, so each individual flow control problem involves just
two nodes. Because inter-region traffic must pass through inter-
region gateways, any two nodes between which flow control is an issue
must inhabit a common DTN region and be using a common transport
protocol below the bundle layer (otherwise they could not be
communicating and there would be no flow to control). Therefore, it
may be possible to accomplish DTN flow control by invoking whatever
Cerf, et al. Expires September 2003 [Page 14]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
flow control mechanisms are already provided within the region by the
common transport protocol, if such mechanisms exist.
Alternatively, a new, supplementary flow control protocol could be
developed at the bundling layer; this would have the advantage of
reducing a DTN's reliance on capabilities provided by the underlying
protocols. At this time it's still unclear which approach is
preferable, and some combination of the two may eventually be
declared to be the best choice.
In either case, the problem of flow control between nodes separated
by very large signal propagation times remains to be solved: TCP's
flow control and congestion control facilities could be leveraged
within regions characterized by small round-trip times, but at this
time no comparable protocol exists for very long delay paths.
It remains as an exercise for us to demonstrate that a hop-by-hop
flow control scheme in long and/or highly variable delay environments
can effect end-to-end congestion control in a fair and efficient
manner.
3.13 Security
Resource scarcity in delay-tolerant networks dictates that some form
of access control to the network itself is required in many
circumstances. It is not acceptable for an unauthorized user to be
able to easily flood the network with traffic, possibly preventing
the network's mission from being accomplished. Implicit in this
statement is a requirement for some form of admission control and/or
in-network authentication that is sensitive to the class of service
that a user has requested, and the means to verify that the user is
authorized to make that request. In a low delay environment, this
would be tedious for performance reasons. In a high/variable delay,
and possibly low data rate environment, it is potentially much worse:
remote access control lists are difficult to update, query/response
keying protocols are not resource-efficient, and routers or end nodes
might be compromised for significant periods of time before being
noticed.
To implement the security model, each message includes an immutable
"postage stamp" (a type of capability) containing a verifiable
identity of the sender (or role), an approval (and approving
authority) of the requested class of service (CoS) associated with
the message, and other conventional cryptographic material to verify
accuracy of the message content. Routers check credentials at each
DTN hop, and discard traffic as early as possible if authentication
fails. This approach has the associated benefit of making denial-of-
service attacks considerably harder to mount as compared with
conventional Internet routers.
The current approach uses public key cryptography as a starting point
for keying. Routers and principals are issued public/private key
pairs, and a principal sending a message must obtain a signed copy of
its public key from a certificate authority known to DTN routers.
Cerf, et al. Expires September 2003 [Page 15]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
(All routers in a DTN are assumed to be pre-equipped with copies of
one or more certificate authority public keys and their own
public/private key pairs). A user then presents her signed public key
along with a message to be carried signed using her private key. At
the first DTN router, the signed public key is used to validate the
sender and requested CoS against an access control list stored in the
router. Accepted messages are then re-signed in the key
of the router for transit. Using this approach, only first-hop
routers need cache per-user certificates, and then only for adjacent
users. Non-edge "core" routers can rely on the authentication of
upstream routers to verify the authenticity of messages. We believe
this approach will help to improve the scalability of key management
for these networks, as it will limit the number of cached public key
certificates to a function of the number of adjacent routers rather
than the number of end-users. This should provide both the obvious
advantage of space savings, but also an improvement to system
management as router keys are expected to be changed less frequently
than end-user keys. As DTN routers are likely to be deployed in
remote areas, re-keying operations may be comparatively burdensome
system management tasks, so limiting the number and frequency of
certificate updates should provide additional savings.
The current approach is partially susceptible to compromised routers.
If an otherwise-legitimate router is compromised, it would be able to
utilize network resources at an arbitrary CoS setting and send
traffic purportedly originating from any user who's identity is known
to the router. However, if the message signature is carried end-to-
end (an option for DTN security), a legitimate user could repudiate
the origin of any traffic generated in this manner. Thus, we believe
a reasonable trade-off is to admit the possibility that a compromised
router could launch a denial-of-service attack in order to gain the
scalability benefits of not checking end-user credentials at every
hop.
4 State Management Considerations
An important aspect of any networking architecture is its management
of state information. This section describes the state information
that is managed at the bundle layer and discusses the conditions
under which that state information is established and how it is
removed.
4.1 Demultiplexing and Binding State
In long/variable delay environments, an asynchronous application
interface seems to be the only sensible approach. Such interfaces
involve callback registration actions that create state information.
This information is typically created by explicit request of the
application, and is removed by a separate explicit request, and is
thus "hard" state. In most cases, there must be provision for
retaining this state information across application
termination/restart operations, and across operating system
termination/restart operations because a client/server message round-
Cerf, et al. Expires September 2003 [Page 16]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
trip time may exceed the requesting application's execution time (or
hosting system's uptime).
In addition, the tuples for which an application wishes to receive
data must be associated with the protocol endpoint identifier
information needed to reach the application itself in the region it
exists. This operation (analogous to the socket bind() operation)
may result in a client/server type of interaction within a region
because regional gateways must extract and resolve incoming bundle
entity names to information required to perform delivery within the
destination region (compare, for example with other name/addressing
mapping services such as dynamic DNS [RFC2136]).
4.2 Bundle Retransmission State
State information to support bundle retransmission is created at a
DTN node when a bundle is received from a DTN user application
requesting custodial transmission) or when a bundle is received from
a peer DTN node and the receiving node intends to assume custody of
the bundle. The bundle's time-to-live field (possibly mitigated by
local policy) determines when this state is purged from the system in
the event that it is not purged explicitly due to acknowledgment.
4.3 Bundle Routing State
Forwarding tables, whether statically configured or maintained by
routing protocols, introduce state information that must be managed
in a manner that is dependent upon the specific routing mechanisms
that are employed. While routing protocols have not yet been
developed for DTN networks, in order to provide forwarding a DTN
router will be required to have available a forwarding table
containing region names and next-hops. In addition, it seems highly
useful to also include a method for routers to learn about potential
custodians. This information would enlarge the overall forwarding
state, but probably not significantly.
We have not yet seriously considered the routing protocols or metrics
that we will use, so we do not have an estimate for the size of each
routing entry, whether it be for inter-region or intra-region
routing. This remains work to be done.
4.4 Security-Related State
The infrastructure protection model described in this architecture
requires maintenance of state in all DTN nodes. All nodes are
required to store their own public/private key pairs and their
network access credentials, signed by an appropriate approving
authority. Additionally, all nodes are required to cache public keys
for one or more certificate authorities and. Further, in most cases,
DTN nodes will cache the public keys (and possibly the credentials)
of their next-hop (in bundle-space) neighbors. All keys will have
expiration times, and nodes are responsible for acquiring and
distributing newly signed copies of their public keys and credentials
Cerf, et al. Expires September 2003 [Page 17]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
prior to the expiration of the old set (in order to avoid a
disruption in network service).
Some DTNs may implement security boundaries at various points in the
network, at which point end-user credentials are checked in addition
to checking router credentials. User application public keys will
typically be cached at these points in the network.
5 Bundle Header Information
The bundle layer must carry some information end-to-end. The full
details of the fields present in a bundle header are specified in
[BundleSpec]. This section documents the meta-data information that
must be carried end-to-end, and notes which of those data elements
may be supplied by the application using the bundle service.
* Version Identifier: an 8-bit bundle protocol
* Destination entity ID: a variable-length field containing the
destination tuple, as described above. It is supplied by the
local application when sending using the bundle service.
* Source entity ID: this is the identifier of the source bundle
application instance, and is a tuple. It is supplied by the
local bundle service, since a particular host may have multiple
names and one may be chosen based on routing decisions or other
criteria opaque to the application (as in multihomed hosts).
The source entity ID may be returned to the application to
support return receipt processing.
* Reply-to entity ID (optional): a source may anticipate not being
able to accept replies, and may use the reply-to entity ID to
specify the destination for return receipts and delivery
records.
* Current custodian ID (optional): Entity ID of the current
custodian. It is necessary to identify the upstream node that
currently has custody of a bundle, in order to acknowledge
correct receipt or custody transfer of a bundle or bundle
fragments.
* Class of service flags:
- Flags: custody, return receipt, delivery record
- CoS Selector: bulk, normal, expedited
- Security: presence of authentication and/or encryption
* Send timestamp: the time that a bundle was presented by the
sending application to the bundle layer for transmission.
* Time to live: an offset, in seconds, from the send timestamp
that indicates when the bundle shall be purged from the DTN.
* Authentication information (optional): authentication data used
to prove that this bundle should be forwarded in the network.
* Fragmentation information (optional): for a bundle fragment,
indicates the place in the original bundle this fragment
belongs.
Some bundles (or events) cause a status indication to be generated by
the bundle layer. Bundle layer indications are sent as bundles with
Cerf, et al. Expires September 2003 [Page 18]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
the user data portion replaced by a Status Report, consisting of the
following information:
* Source entity ID of the subject bundle: a copy of the source
tuple from the subject bundle.
* Send timestamp of the subject bundle: used to disambiguate
status reports for different bundles from the same source entity
* Status flags, indicating whether or not a bundle was:
. received correctly by the sender of the Status Report
. Custodially transferred to the sender of the Status Report
. Forwarded by the sender of the Status Report
* Time of Receipt (optional): the time at which the sender of the
Status Report received the bundle.
* Time of Forward (optional): the time at which the sender of the
Status Report forwarded the bundle
6 Application Structuring Issues
DTN bundle delivery is intended to operate in a *delay-tolerant*
fashion over a broad range of network types. That does not mean that
there *must* be delays in the network, but that there *may* be very
significant delays. The protocols providing the services exposed by
the bundle layer are delay tolerant, so to take best advantage of
them, applications using them should be, too.
Message-oriented communication differs from conversational
communication. In general, applications should attempt to include
enough information in a bundle so that it may be treated as an
independent unit of work by the remote entity (a form of "application
data unit" [CT90] or "transaction", although transactions carry
connotations of multi-phase commitment that we do not intend here).
The goal is to minimize conversation between applications that are
separated by a network that presents long and possibly highly
variable delays, and to constrain any conversation that does occur to
be asynchronous in nature. A single file transfer request bundle,
for example, might include authentication information, file location
information, and requested file operation. Comparing this style of
operation to a classic FTP transfer, one sees that the bundled model
can complete in one round trip time, whereas an FTP file "put"
operation can take as many as eight round trips to get to a point
where file data can flow [DFS02].
Delay-tolerant applications must consider additional factors beyond
the conversational implications of long delay paths. Application
instances may terminate (voluntarily or not) between the time that a
bundle is sent and the time its response is received. If an
application has anticipated this possibility, it is possible to re-
instantiate the application instance with state information saved in
persistent storage. This is an implementation issue, but also an
application design consideration.
Some consideration of delay-tolerant application design can result in
applications that work reasonably well in low-delay environments, and
Cerf, et al. Expires September 2003 [Page 19]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
that do not suffer extraordinarily in high or highly-variable delay
environments.
7 Convergence Layer Considerations for Use of Underlying Protocols
Implementation experience with the DTN architecture has revealed an
important architectural construct and interface for DTN routers. Not
all transport protocols in different protocol families provide the
same exact functionality, so some additional adaptation or
augmentation on a per-stack basis may be require. This adaptation is
accomplished by a set of convergence layers placed between the bundle
layer and underlying transport protocols. The convergence layers
manage the protocol-specific details of interfacing with a particular
transport service and present a consistent interface to the bundle
layer.
The complexity of a convergence layer may vary substantially
depending on the type of protocol stack it adapts. For example, a
TCP/IP convergence layer for use in the Internet might only have to
add message boundaries to TCP streams, whereas a convergence layer
for some network where no reliable transport protocol exists may have
a considerable amount of work to do, at least if custody transfer is
to be supported.
8 Summary
The DTN architecture addresses many of the problems of networks that
must operate in environments subject to poor performance and non-
continuous end-to-end connectivity. It is based on asynchronous
messaging, and uses as a model of service classes those offered by
the postal mail system. It accommodates many different forms of
connectivity, including scheduled, predicted, and opportunistically
connected links. It introduces a novel approach to end-to-end
reliability across frequently partitioned and unreliable networks.
It also proposes a scheme for securing the network infrastructure
against unauthorized access.
It is our belief that this architecture is applicable to many
different types emerging environments. In subsequent documents, we
intend to specify profiles of this architecture that address specific
environments in detail.
9 Informative References
[SB03] S. Burleigh et al, "Delay-Tolerant Networking û An Approach to
Interplanetary Internet," To appear in IEEE Communications Magazine
approximately May 2003.
[CT90] D. Clark, D. Tennenhouse, "Architectural Considerations for a
new generation of protocols," Proc. SIGCOMM 1990.
[FW03] F. Warthman, "Delay-Tolerant Networks (DTNs): A Tutorial",
Wartham Associates, Available from http://www.dtnrg.org
Cerf, et al. Expires September 2003 [Page 20]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
[KF03] K. Fall, "A Delay-Tolerant Network Architecture for Challenged
Internets," Intel Research, IRB-TR-03-003. Available from
http://www.intel-research.net/publications.asp
[IGE00] C. Intanagonwiwat, R. Govindan, D. Estrin, "Directed
Diffusion: A scalable and robust communication paradigm for
sensor networks", MobiCOM 2000, Aug 2000.
[WSBL99] William Adjie-Winoto, Elliot Schwartz, Hari Balakrishnan,
Jeremy Lilley, The design and implementation of an intentional naming
system, Proc. 17th ACM SOSP, Kiawah Island, SC, Dec. 1999.
[NEWARCH] NewArch Project: Future-Generation Internet Architecture.
http://www.isi.edu/newarch
[META] Wroclawski, John T., "The Metanet," Workshop on Research
Directions for the Next Generation Internet, May 12-14, 1997, Vienna,
VA. http://www.cra.org/Policy/NGI/papers/wroklawWP.
[CK74] V. Cerf, R. Kahn, "A Protocol for Packet Network
Intercommunication", IEEE Trans. on Comm., COM-22(5), May 1974
[DFS02] R. Durst, P. Feighery, K. Scott, "Why not use the Standard
Internet Suite for the Interplanetary Internet?", MITRE White Paper,
http://www.ipnsig.org/reports/TCP_IP.pdf
[RJ90] R. Jain, "Congestion Control in Computer Networks: Issues and
Trends," IEEE Network Magazine, May 1990.
[RFC2136] P. Vixie, ed., "Dynamic Updates in the Domain Name System
(DNS UPDATE)," Internet Request for Comments, RFC2136, Apr. 1997
[BundleSpec] S. Burleigh, et al, "Bundle Protocol Specification" work
in progress, (draft-irtf-dtnrg-bundle-spec-00.txt), March 2003.
Available from http://www.dtnrg.org.
10 Security Considerations
Security is an integral concern of the Delay Tolerant Network
Architecture. Section 3.13 of this document presents an approach for
securing the DTN architecture. These capabilities may also be useful
in providing facilities to DTN applications to construct their own
end-to-end security protocols.
Cerf, et al. Expires September 2003 [Page 21]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
11 Authors' Addresses
Dr. Vinton G. Cerf
MCI WorldCom
22001 Loudoun County Parkway
Building F2, Room 4115, ATTN: Vint Cerf
Ashburn, VA 20147
Telephone +1 (703) 886-1690
FAX +1 (703) 886-0047
Email vcerf@mci.net
Scott C. Burleigh
Jet Propulsion Laboratory
4800 Oak Grove Drive
M/S: 179-206
Pasadena, CA 91109-8099
Telephone +1 (818) 393-3353
FAX +1 (818) 354-1075
Email Scott.Burleigh@jpl.nasa.gov
Robert C. Durst
The MITRE Corporation
1820 Dolley Madison Blvd.
M/S W650
McLean, VA 22102
Telephone +1 (703) 883-7535
FAX +1 (703) 883-7142
Email durst@mitre.org
Dr. Kevin Fall
Intel Research, Berkeley
2150 Shattuck Ave., #1300
Berkeley, CA 94704
Telephone +1 (510) 495-3014
FAX +1 (510) 495-3049
Email kfall@intel-research.net
Adrian J. Hooke
Jet Propulsion Laboratory
4800 Oak Grove Drive
M/S: 303-400
Pasadena, CA 91109-8099
Telephone +1 (818) 354-3063
FAX +1 (818) 393-3575
Email Adrian.Hooke@jpl.nasa.gov
Dr. Keith L. Scott
The MITRE Corporation
1820 Dolley Madison Blvd.
M/S W650
McLean, VA 22102
Telephone +1 (703) 883-6547
Cerf, et al. Expires September 2003 [Page 22]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
FAX +1 (703) 883-7142
Email kscott@mitre.org
Leigh Torgerson
Jet Propulsion Laboratory
4800 Oak Grove Drive
M/S: T1710-
Pasadena, CA 91109-8099
Telephone +1 (818) 393-0695
FAX +1 (818) 354-9068
Email Leigh.Torgerson@jpl.nasa.gov
Howard S. Weiss
SPARTA, Inc.
9861 Broken Land Parkway
Columbia, MD 21046
Telephone +1 (410) 381-9400 x201
FAX +1 (410) 381-5559
Email hsw@sparta.com
Please refer comments to dtn-interest@mailman.dtnrg.org
12 Intellectual Property Notices
The IETF takes no position regarding the validity or scope of
any intellectual property or other rights that might be claimed to
pertain to the implementation or use of the technology
described in this document or the extent to which any license
under such rights might or might not be available; neither does
it represent that it has made any effort to identify any such
rights. Information on the IETF's procedures with respect to rights
in standards-track and standards-related documentation
can be found in BCP-11. Copies of claims of rights made
available for publication and any assurances of licenses to be made
available, or the result of an attempt made
to obtain a general license or permission for the use of such
proprietary rights by implementors or users of this
specification can be obtained from the IETF Secretariat.
The IETF invites any interested party to bring to its
attention any copyrights, patents or patent applications, or
other proprietary rights which may cover technology that may be
required to practice this standard. Please address the
information to the IETF Executive Director.
13 Copyright Notices
Copyright (C) The Internet Society (2003). All Rights Reserved.
This document and translations of it may be copied and
furnished to others, and derivative works that comment on or
otherwise explain it or assist in its implmentation may be
prepared, copied, published and distributed, in whole or in
Cerf, et al. Expires September 2003 [Page 23]
Internet Draft draft-irtf-dtnrg-arch-02.txt March 2003
part, without restriction of any kind, provided that the above
copyright notice and this paragraph are included on all such
copies and derivative works. However, this document itself may
not be modified in any way, such as by removing the copyright
notice or references to the Internet Society or other Internet
organizations, except as needed for the purpose of developing
Internet standards in which case the procedures for copyrights
defined in the Internet Standards process must be followed, or as
required to translate it into languages other than English
The limited permissions granted above are perpetual and will
not be revoked by the Internet Society or its successors or
assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE
OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY
IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A
PARTICULAR PURPOSE.
Cerf, et al. Expires September 2003 [Page 24]
