IPN Research Group V. Cerf
INTERNET-DRAFT Worldcom/Jet Propulsion Laboratory
<draft-irtf-ipnrg-arch-01.txt> S. Burleigh
August 2002 A. Hooke
Expires February 2003 L. Torgerson
NASA/Jet Propulsion Laboratory
R. Durst
K. Scott
The MITRE Corporation
K. Fall
Intel Corporation
H. Weiss
SPARTA, Inc.
Delay-Tolerant Network Architecture:
The Evolving Interplanetary Internet
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Abstract
This document describes an architecture for delay-tolerant networks,
and is a generalization of the architecture designed for the
Interplanetary Internet: a communication system to provide Internet-
like services across interplanetary distances in support of deep
space exploration. This generalization addresses networks whose
operational characteristics make conventional networking approaches
become either unworkable or impractical. We define a message-based
overlay that exists above the transport layer of the networks on
which it is hosted. The draft presents an architectural overview
followed by discussions of services, topology, routing, security,
reliability and state management. It concludes with a discussion of
end-to-end information exchange, including an example.
Cerf, et al. Expires February 2003 [Page 1]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
Table of Contents
Status of this Memo................................................1
Abstract...........................................................1
Table of Contents..................................................2
Copyright Notice...................................................3
Foreword...........................................................5
1 Introduction................................................6
2 Why an Architecture for Delay-Tolerant Networking?..........6
2.1 Extreme Environments...................................7
2.2 Problems with Internet Protocols and Applications.....11
2.3 DTN Principles of Design..............................14
3 DTN Architectural Overview.................................16
3.1 The DTN Architecture is Based on Message Switching....16
3.2 DTN Classes of Service Mimic Postal Operation.........17
3.3 Regions and Region Identifiers........................17
3.4 Tuples................................................18
3.5 Late Binding..........................................18
3.6 The "Bundle Layer" Terminates Local Transport Protocols
and Operates End-to-End...............................19
3.7 Routing...............................................19
3.8 Bundle Layer Reliability and Custodianship............19
3.9 Security..............................................20
3.10 Time Synchronization..................................21
4 Service Considerations: Application Instances and Bundles.21
4.1 Networking Style Issues...............................21
4.2 Bundle Lifetime.......................................22
4.3 Classes of Service....................................23
4.4 Delivery Options......................................23
4.5 Naming and Addressing.................................24
5 Topological Considerations: Nodes, Regions, and Gateways..24
5.1 Nodes.................................................25
5.2 Regions...............................................25
5.3 Gateways..............................................26
5.4 Discussion............................................26
6 Routing considerations.....................................26
6.1 Types of Routes.......................................27
6.2 Store-and-forward operation...........................28
6.3 Contact-oriented routing..............................29
6.4 Routing protocols.....................................29
7 Security considerations....................................30
7.1 User-oriented security services.......................30
7.2 Infrastructure security services......................32
8 Reliability Considerations.................................35
8.1 Custodial Operation...................................35
8.2 End-to-end Retransmission.............................36
8.3 Congestion and Flow Control at the Bundle Layer.......37
9 State Management Considerations............................38
9.1 Application Interface State...........................39
9.2 Bundle retransmission state...........................39
Cerf, et al. Expires February 2003 [Page 2]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
9.3 Bundle routing state..................................40
9.4 Transmission queue state..............................41
9.5 Receive queue state...................................41
9.6 Network management state..............................41
10 Convergence Layer Considerations for Use of Underlying
Protocols..................................................42
11 Bundle Header Information..................................42
12 An Example Bundle Transfer.................................44
12.1 Rules for forming tuples in the example network.......44
12.2 Example Network Topology at the Region Level..........45
12.3 DTN Gateway routing...................................47
12.4 Systems participating in example bundle data transfer.48
12.5 End-to-end Transfer...................................50
12.6 Error Conditions at the Bundle Layer..................53
13 Summary....................................................55
14 References.................................................56
15 Security Considerations....................................57
16 Authors' Addresses.........................................58
Copyright Notice
Copyright (C) The Internet Society (2002). All Rights Reserved.
Cerf, et al. Expires February 2003 [Page 3]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
Acknowledgments
John Wroclawski, David Mills, Greg Miller, James P. G. Sterbenz, Joe
Touch, Steven Low, Lloyd Wood, Robert Braden, Deborah Estrin, Stephen
Farrell and Craig Partridge all contributed useful thoughts and
criticisms to this document. We are grateful for their time and
participation.
This work was performed in part under DOD Contract DAA-B07-00-CC201,
DARPA AO H912; JPL Task Plan No. 80-5045, DARPA AO H870; and NASA
Contract NAS7-1407.
Cerf, et al. Expires February 2003 [Page 4]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
Foreword
"The term 'sonata' can be applied to a large work in three movements
or to the three sections of a single movement: exposition,
development, recapitulation. They resonate with the three acts of a
movie or play, the three elements of a primal myth or legend (life,
death, rebirth), the cycle of days (daylight, night, then daylight
again) and of the seasons in temperate climates (nice weather, then
winter, then it gets nice again), the Star Wars trilogy, etc.
Basically, in any human experience we start off brightly, full of
energy and hope; then we experience reality and become thoughtful and
reflective, darker and slower; then Something Happens and hope is
reborn, our energy returns, and we reach that happy synthesis of
Innocence and Experience that is maturity. Allegro, andante,
rondo..."
-- Scott Burleigh
Release Notes
draft-irtf-ipnrg-arch-00.txt, May 2001: Allegro.
Original Issue.
draft-irtf-ipnrg-arch-01.txt, August 2002: Andante.
Restructured document to distinguish between architecture for delay-
tolerant networking and the *application* of that architecture to a
number of different environments, potentially including
interplanetary internetworking, military tactical networking, and
sensor internetworking.
Refined DTN classes of service and delivery options. Added a "reply-
to" address to have response information such as error messages or
end-to-end acks directed to a source-specified third party.
Further defined the topological elements of delay tolerant networks.
Elaborated routing and reliability considerations.
Presented a model for securing the delay tolerant network
infrastructure.
Expanded discussion of flow and congestion control.
Added section discussing state information at the bundle layer.
Updated bundle header information and end-to-end data transfer.
Cerf, et al. Expires February 2003 [Page 5]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
1 Introduction
This document describes an architecture for delay-tolerant networks.
We present this as a generalization and extension of our ongoing work
on the Interplanetary Internet (http://www.ipnsig.org). We have come
to the realization that there are a number of different environments
that share essential characteristics for which the architecture
presented herein is appropriate. For example, sensor-based networks
may have scheduled intermittent connectivity with a "conventional"
internet with long delays between contacts. These delays may result
from limited availability of communication assets (e.g. Low Earth
Orbiting satellite contacts) or from limited availability of other
communication assets (e.g., power to run a transmitter).
Applications using the network may perceive long, highly variable
end-to-end delays, particularly if multiple instances of intermittent
connectivity affect a path from source to destination. This fully
terrestrial example may appear no different to an application than an
interplanetary communication, even though each individual link may be
low-delay.
The particular approach we employ is that of a message-based overlay
that exists above the transport layers of the networks on which it is
hosted.
2 Why an Architecture for Delay-Tolerant Networking?
The existing TCP/IP-based internet, while fabulously successful in
many environments, does not suit all environments. The ability of
the "TCP/IP suite" to provide service depends on a number of
important assumptions:
. that an end-to-end path between source and destination exists for
the duration of the communication session;
. (for reliable communication) that the maximum round-trip time over
that path is not excessive and not highly variable from packet to
packet; and
. that the end-to-end loss is relatively small.
A class of "challenged networks," which may violate one or more of
these assumptions, is becoming important and may not be well served
by the current end-to-end TCP/IP model. These networks typically
serve environments in which it is either impractical or impossible to
configure a communication environment that supports the assumptions
on which the TCP/IP suite depends.
This section examines some of the constraints posed by extreme
environments that may result in "challenged networks," and considers
the problems that these environments might cause for common Internet
protocols and applications. Finally, we derive some "principles of
design" for the DTN.
Cerf, et al. Expires February 2003 [Page 6]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
2.1 Extreme Environments
2.1.1 Constraints Posed by Extreme Environments
The kinds of extreme environments that this DTN architecture
addresses constrain both the communication system and the
applications using that communication system. This section describes
some of the characteristics that make environments "extreme."
Clearly, not all environments that would be considered "extreme"
exhibit all of these characteristics.
Long and/or Variable Delays
Delay affects networks in at least two different ways. First,
interactivity is directly affected by delay. For example, telephone
conversations over a terrestrial telephone line are markedly
different than telephone conversations that are routed over a
geostationary satellite hop. Second, variability in delay can affect
applications and protocols. Consider a stream of packets carrying
voice data over a network. If the delay in the network varies
greatly from one packet to the next, a buffer is required to avoid
the perception of pauses in the reconstructed voice track. If the
application is hosted on a network with greater delay variability
than the application was designed for, the stream will again have
pauses, possibly of sufficient magnitude to make the data
unintelligible.
Delays are comprised primarily of three components: propagation delay
through the medium; queuing delay within relay points, source, and
destination; and clocking delays associated with transmitting an
atomic unit of data onto the medium.
Propagation delays can be long due to speed-of-light delays to cross
long distances (e.g., deep space). Alternatively, propagation delays
could be long due to the propagation medium (e.g.,
acoustic/underwater). How long is long? Light time (round trip)
between Earth and Mars ranges from ~8 minutes to ~40 minutes.
(Mars's average distance from the sun is 1.5AU, and light in a vacuum
travels at roughly 8 minutes per AU.)
Queuing delays are affected by traffic and service rate. In extreme
environments, the service rate of a node may be low due to power
limitations requiring a slow processor or a long wait while the unit
is quiescent. Arrival distributions may cause significant
variability in delays, particularly for heavy-tailed distributions
[KP97]. (Heavy-tailed distributions are those that asymptotically
follow a power law. That is, P[X > x] ~ x^-alpha as x approaches
infinity, for 0 < alpha < 2. For alpha < 2, the variance of a heavy-
Cerf, et al. Expires February 2003 [Page 7]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
tailed distribution is infinite. For alpha < 1, the mean is also
infinite. "Thus, as alpha decreases, a large portion of the
probability mass resides in the tail of the distribution." [KP97])
Clocking delays accrue each time a packet is transmitted if the
packet was fully received prior to transmission. "Cut-through"
routing, in which the first bits/bytes of a data unit are operated
upon before the last bits/bytes have been received, is a fairly rare
operation. For many types of data, the latency advantages of cut-
through routing are more than offset by the reluctance of network
designers to forward corrupt data. (Although some types of data are
useful even if partially corrupted.) For data that are not forwarded
before being fully received (and error checked), clocking delays
accumulate at each hop in the network. For relatively slow, multi-
hop networks, this can result in high per-packet delays, particularly
if packet sizes are large. (Which can, of course, increase queuing
delays for other packets, increasing both the round trip times and
the variability of delay.)
We assume that processing delay, another contributor to overall
delay, is comparatively low. However for some simple processors,
cryptographic processing, and in particular key generation, can
contribute sufficiently to processing delay and in some circumstances
make it become a noticeable contributor to overall delay.
Variability of delay can have a significant effect on communication
in addition to absolute delay. Many protocols attempt to provide
reliability through retransmission (ARQ) mechanisms. Timers are a
critical element of most retransmission mechanisms, and establishing
a retransmission time out value is an important aspect of the
mechanism. Some protocols, particularly those at the link layer,
have the ability to eliminate all components of round trip time other
than propagation and clocking delays from their timings, and are able
to produce retransmission time out estimates that are quite close to
the actual round trip time. When queuing delay must be considered
and the timing does not have direct knowledge of the delays at all of
the queues, estimating a reasonable retransmission time out value
becomes more complex. A timeout value that is too low will cause
unnecessary retransmission, while one that is too high will cause
excessive delay in data delivery.
Frequent Partitioning (Intermittent Reachability)
Network partitioning typically adds to data loss, because data that
cannot be forwarded essentially immediately is generally discarded.
If losses become too severe, applications typically fail. In addition
to contributing to loss, network partitioning adds significantly to
overall delay if nodes are configured to store data until the
outbound link is restored. This behavior can also contribute to
Cerf, et al. Expires February 2003 [Page 8]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
misordered data arriving at the destination, which can cause poor
performance in some protocols.
Data Rate Asymmetry
Data rate asymmetry measures the ratio of forward-path minimum
capacity versus reverse-path minimum capacity. Data rate asymmetry
adversely affects protocols using ARQ for reliable delivery by
altering the ACK or NACK return path. This effect has been studied
extensively with TCP, and is discussed below. At a higher layer,
request/response applications that have not been appropriately tuned
to balance the amount of request versus response traffic can time out
waiting for data to travel across the lower-capacity link. In the
most extreme case, the asymmetry is infinite (as is the case with
communications to radio-silent submarines, for example) ARQ is simply
not possible. In these cases, forward error correction and periodic
retransmission are typically used to improve communication
reliability. See [BLMR98] as an example of how such techniques are
used in the Internet.
Data rate asymmetry can arise due to routing path asymmetry,
different forward/reverse path link technologies, or intentional
engineering tradeoffs. Moderate asymmetries in the wired Internet
are found most commonly in asymmetric access technologies such as in
CATV or ADSL-based subscriber lines. The largest asymmetries for
wireless Internet access appear to be prevalent in in-flight Internet
access systems. The AirTV system, for example, being proposed for
2004 and beyond, could provide a ground-to-air speed of up to 45Mb/s
using DVB satellite technology with a comparatively anemic air-to-
ground bandwidth of a 128kb/s or less using INMARSAT [ARINC]. In the
case of deep space communications, downlink data rate is
intentionally engineered to be much greater than uplink data rate.
This tradeoff is not surprising given the goal of most science
missions: the capacity demands of downlinking high-resolution images
and telemetry from a spacecraft dwarf the capacity required to uplink
short commands to that spacecraft.
Packet Loss and Errors
Consider the following probability of success metric:
Pr = (1-pe)^(i*(8*m))
This metric is the probability of successful end-to-end delivery of a
particular message of size m bytes across i links assuming an
identical, independently distributed (IID) stationary loss process
with constant bit error rate pe across all intervening forwarders.
(In this equation, the carat -- ^ -- indicates exponentiation.) This
metric, as expressed, does not account for congestive loss but does
Cerf, et al. Expires February 2003 [Page 9]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
account for message size and number of cascaded links given a fixed
BER. For packet networks, most links employ some form of error
detection, implying that any bit loss creates an end-to-end packet
loss. As can be clearly seen from the formula, the end-to-end
probability of successful delivery decays exponentially with path hop
count. Any congestive loss would only worsen the performance.
For reliable transfer, excessive errors require repair using either
error correction or retransmission. If the path is sufficiently
lossy, end-to-end retransmission can become essentially useless. To
understand why this is so, consider a probability of packet loss pp =
1 ¡ Pr for some fixed m. Given the assumptions listed above, the
expected number of retransmissions required before successful
delivery is given by: (1-(1-pp)^i)/(1-pp)^i. Considering a packet
error probability of 0.3. In this case, 4 hops requires 3
retransmissions, 10 hops requires 34, and 20 hops requires about 1200
retransmissions. If a hop-by-hop retransmission scheme were used
instead, the total (network wide) number of retransmissions is given
by i*pp/(1-pp). For the same error probability of 0.3, the number of
retransmissions for 4, 10, and 20 hops would be 2, 5, and 9,
respectively. Thus, for very lossy environments, hop-by-hop
retransmission is preferable to end to end retransmission.
2.1.2 Interoperating Among Differently-Challenged Networks
Two complicating factors in building internetworks comprised of
networks operating in multiple different extreme environments are
1. Network adaptations necessary to achieve required operational
characteristics in one extreme environment may be mutually
exclusive with those required in another environment, and
2. Evolution to achieve interoperability with networks not
anticipated at design time may be technically or politically
infeasible.
There is no guarantee of support for any common communication
technology across networks designed for operation in extreme
environments. In many instances, at the time of deployment only the
barest minimum of capabilities can be fielded to support the
network's mission.
If these networks are successful, they evolve. In many cases, this
evolution leads to an expansion of scope and span for the network.
Often this can result in a requirement to interoperate with another
network in an environment that presents different challenges that
have been addressed by dramatically different solutions.
The ability to operate over many different types of networks that
have been adapted to support a range of benign and extreme
Cerf, et al. Expires February 2003 [Page 10]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
environments is both a challenge and a requirement for this Delay
Tolerant Networking Architecture.
2.1.3 Examples of some extreme environments
Several types of extreme environments currently exist, with more
appearing almost daily. If one considers as the norm the wired
Internet, with multi-gigabit backbones, almost no corruption-related
data losses and relatively short round trip times, then many
different types of environments seem extreme.
This architecture was originally conceived to support the
Interplanetary Internet, which exhibits round trip delays on the
order of tens of minutes and intermittent connectivity that can
result in disconnection for weeks.
Military tactical networks, in which data rates are low, error rates
are high, and link interruptions are frequent, can likely derive
great benefit from application of this architecture.
Similarly, sensor networks deployed in oceanic environments exhibit
long delays, significant data loss, and the potential for long link
outages.
An individual seeking a networking solution for a particularly
intriguing and challenging environment has recently approached us to
determine if the DTN architecture was right for the following
environment. The Sami people in Lapland live in widely dispersed
communities in remote areas and are not well served by either wired,
fixed wireless, or satellite internet service. They do, however,
frequently travel on snowmobiles from community to community, and
congregate at work sites and larger towns. To effectively serve this
environment, an architecture is required that can embrace
intermittent, probabilistic connectivity, store and forward
operation, and high and variable delays. (The architecture described
in this document meets these needs.)
2.2 Problems with Internet Protocols and Applications
This section discusses some of the issues associated with attempting
to serve challenged networks with the Internet suite of protocols.
2.2.1 Internet "Core" Protocols
The performance characteristics of challenged networks confound the
efficient operation of the core Internet protocols. By the 'core'
Internet protocols we mean IP, TCP, UDP, BGP, common IGPs (RIP, OSPF,
or EIGRP) and DNS. These protocols span the services of end-to-end
datagram delivery, reliable two-party stream delivery, regional
Cerf, et al. Expires February 2003 [Page 11]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
(aggregated) routing path discovery with policy, intra-domain path
selection and distributed support for name resolution. Although some
of these protocols are technically "application" protocols from a
layering point of view, we treat them here together as core protocols
for the purposes of discussion.
Excessive latency affects TCP directly by severely limiting its
throughput performance or interfering with connection establishment
[DMT96]. Any application-layer protocol using TCP as its underlying
transport is therefore affected (for example, BGP and DNS zone
transfers). Excessive latency also adversely affects the proper
operation of conventional routing protocols as links will be
incorrectly discovered to be non-operating when soft-state refreshes
are delayed too long (RIP), or a lack of response to link state
"hello" messages (OSPF, EIGRP) is observed. UDP is not sensitive to
excessive latency because it does not contain timers that affect its
operation. However, core application protocols that require it (for
example, DNS queries/responses; also SNMP, if used) will take too
long to complete when faced with excessive delay, triggering early
application failure (or the lack of ability to use DNS name/address
mappings in the case of address-to-name queries).
Data rate asymmetry adversely affects TCP by altering the smooth flow
of acknowledgments. Extensive studies have been conducted on TCP
using the normalized asymmetry ratio, which takes into account the
asymmetric message sizes between data packets and returning ACKS
[MLS00]. Results indicate that bandwidth asymmetry can lead to poor
performance of unidirectional transfers due to alterations in the
time series of the ACK channel. In particular, if ACKs are queued,
their spacing in time causes the TCP sender to clock out subsequent
packets less frequently. If ACKs are lost, burstiness, slow
congestion window growth, or defeat of the fast retransmit/recovery
algorithms can occur. (See [PILC-ASYM] characterization of this
problem and some possible approaches to ameliorate it). Any
application layer protocols attempting to use TCP over asymmetric
links are therefore affected as well.
Significant path loss affects the TCP transport most strongly,
causing it a number of problems. After multiple loss events it will
continue retrying with a backed-off retransmission timer until it
gives up on retransmitting altogether and closes the connection.
Somewhat more moderate losses will contribute to problems invoking
the fast retransmit and recovery algorithms, even in the presence of
SACK. IP performance can also be affected by path loss if fragmenta-
tion is required. IP provides no mechanism for fragment
retransmission, thereby causing the overall probability of successful
datagram delivery to be further reduced if datagrams are fragmented
[M95].
Cerf, et al. Expires February 2003 [Page 12]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
Node longevity affects the probability of end-to-end delivery, as
round-trip or even one-way delivery time of a particular message may
exceed the sender's lifetime. Clearly, in such cases it is useless
to arrange for immediate acknowledgements to be returned. In such
cases, any notification of successful or unsuccessful delivery needs
to be directed to some alternative node that remains functional.
2.2.2 Common Internet Applications and Supporting Protocols
The most common user services within the Internet today are the web,
e-mail, and perhaps FTP. These services employ a number of different
application protocols, including HTTP, SMTP, POP, IMAP, Microsoft's
Exchange Protocol and FTP. These protocols, or in some cases the
applications that implement them, are either severely degraded or
fail to operate entirely under challenging conditions. Two primary
reasons are application-level timeouts mismatched to the actual
network latency and excessive numbers of round-trip request/response
message exchanges required in order to accomplish a protocol
transaction. For example, web browsers and mail clients will often
time out during connection establishment after a minute or two. The
SMTP protocol requires peers to mutually identify themselves prior to
actually exchanging the mail payload, even though this early name
exchange is not fundamental to the process of delivering e-mail.
Such protocols have been called "chatty" due to their excessive
number of round-trip times required to accomplish simple tasks. FTP
has a similarly chatty interaction during its initial association as
user login and authentication information is exchanged between client
and server.
2.2.3 Discussion
When considering the use of existing Internet protocols as a basis
for interconnecting challenged networks, certain properties of the
operational environment seem insurmountable. The possibility of
intermittently infinite data rate asymmetry appears to preclude the
use of conventional TCP-like protocols based on ARQ for reliability
because an ACK channel may not be available. With exponentially
decaying probability of successful end-to-end delivery as path length
grows, end-to-end reliability should be avoided in favor of hop-by-
hop retransmission in lossy environments. Given the possibility of
very high round-trip times, any expectation of timely performance in
request/response protocols must also be avoided. Finally, the
standard Internet routing protocols assume the immediate existence of
an end-to-end path and do not accommodate the notion of future
(scheduled) routing opportunities that are needed for operation in
frequently partitioned networks.
Cerf, et al. Expires February 2003 [Page 13]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
2.3 DTN Principles of Design
After examining the environmental conditions cited above and seeing
the problems that the Internet protocols have with these conditions,
we derived the following principles of design to guide the definition
of a Delay Tolerant Networking Architecture.
2.3.1 Use transport and network layer technologies appropriate to the
local environment
Where Internet protocols work well, there is a distinct advantage to
using them. However, in other environments, such as sensor networks
or certain radio networks, the internet suite may not be the best fit
for the task at hand. Some sensor networks, for example, use very
different routing and node naming concepts (e.g., directed diffusion
[IGE00]). The DTN architecture should not inhibit designers from
building networks using transport, network, link, and physical layer
technologies that are best suited for their environments. Rather,
the DTN architecture should provide the means for dissimilar networks
to interoperate. The dissimilarity of the extreme environments
discussed precludes the use of a single end-to-end transport
protocol. Instead, end-to-end operations require us to provide a
common substrate above the *transport* layers of the constituent
networks. Thus, whereas IP provides the common substrate for the
Internet above dissimilar link layers, the DTN architecture creates a
"network of Internets" by providing an end-to-end layer above the
transport layer. We call this the "bundle layer."
2.3.2 Employ a "non-chatty" communication model
Because delays may be long and network partitioning may be frequent,
we suggest that highly interactive styles of communication be avoided
in the kinds of extreme environments described above. Rather, entire
application-layer messages, which we call "bundles," move through the
network as units. These bundles are intended to have all of the
primary data and meta data necessary for their use at the
destination.
Although this *may* mean that bundles are large (relative to packets
in a TCP/IP network), it does not require it. The intent simply is
to not have multiple exchanges between source and destination (e.g.
"user:", response, "password:", response, etc.) when such exchanges
can be "bundled" together into a single communication - a single
bundle or a unidirectional sequence of bundles.
2.3.3 Base transfers between nodes on store-and-forward techniques
The Internet Protocol (IP) is based on store and forward
transmission. However, in IP routers, the "store" portion of the
Cerf, et al. Expires February 2003 [Page 14]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
operation typically lasts only long enough to determine the next hop
destination and to wait for any packets to be transmitted that are
ahead of the current one. If no route to the destination exists at
the time that the routing engine examines the packet, that packet is
typically discarded.
In a Delay-Tolerant Network, it is considered *normal* for there to
be no path to the destination at the time a bundle is received.
Bundles are routed based not on current connectivity but on the
expectation of connectivity, drawn either from a firm schedule or
from previous experience. If an episode of connectivity doesn't
occur as expected, bundles are rerouted accordingly.
This approach assumes that there is a considerable amount of storage
available within the network, which is a fundamental assumption for
these types of delay-tolerant networks.
The result of this interpretation of store-and-forward operation is
that bundles of data can still make forward progress toward their
destination even if an end to end path never exists at any single
moment in time.
2.3.4 Advance the point of retransmission toward the destination
The retransmission of lost data in the Internet is (nominally) always
end-to-end: the source must commit resources to the retention of data
for possible retransmission by TCP until reception by the destination
is acknowledged, and retransmitted data must always traverse the
entire route from source to destination. In a Delay-Tolerant Network
such behavior might impose high performance costs due to the length
of time needed to traverse that route, so the point of data
retransmission is advanced toward the destination whenever possible.
This allows earlier release of retransmission resources at any given
node and enables retransmitted data to arrive sooner.
There is an obvious benefit to this approach if the point of
retransmission progresses across a *very* long delay hop (such as one
that spans interplanetary distance). However, in a network composed
of a series of lossy links (such as a congested multi-hop wireless
network), there is similar benefit, as discussed in section 2.1.1,
"Packet Loss and Errors."
Retransmission point advance occurs on two of the three levels at
which the DTN Architecture supports retransmission (the third being
the optional end-to-end retransmission discussed in section 8.2).
First, wherever transmission between pairs of DTN nodes is
accomplished by underlying reliable transport-layer protocols such as
TCP, the probability of correct end-to-end delivery at the bundle
Cerf, et al. Expires February 2003 [Page 15]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
layer is enhanced simply by the concatenation of episodes of point-
to-point reliability provided by those protocols. (The "points" of
transmission at the bundle layer look like "ends" at the transport
layer.) Data arrival at a transport-layer destination informally
advances the point of retransmission within the DTN: the destination
of one transmission that is reliable at the transport layer becomes
the source of the next one.
Second, upon receipt of a bundle, a node may *take custody* of it.
This is a commitment made by the node that it will shoulder the
burden of retransmission *at the bundle layer* to insure that the
bundle reaches its destination, and is accomplished by issuing a
bundle-layer acknowledgment to the previous custodian (which might be
the source). This allows the previous custodian to release the
resources necessary to ensure delivery and formally moves the point
of bundle-layer retransmission closer to the destination.
2.3.5 Provide security services to secure both the infrastructure and
the information
Networks in extreme environments may range from being completely
disposable to irreplaceably precious. Those that are disposable are
typically so resource starved that difficult tradeoffs must be made
regarding provision of service versus protection of infrastructure.
These tradeoffs must be made by those who instantiate the
architecture, and the architecture must provide for a range of
decisions.
Minimally, we have concluded that the architecture must provide for
infrastructure protection, up to and including mutual suspicion among
DTN routers. However, this must be optional at the discretion of
those who actually instantiate a particular delay tolerant network.
Security services offered to users should include key management,
authentication, integrity, and confidentiality, but availability and
use of those services may vary from DTN to DTN.
3 DTN Architectural Overview
The previous section presented the design principles that guide the
definition of the DTN architecture. This section presents an
overview of the decisions that result from those design principles.
3.1 The DTN Architecture is Based on Message Switching
In order to avoid unnecessary interaction between the source and
destination, a DTN transmits "bundles" that contain both user data
and relevant metadata. These bundles are the "messages" on which the
DTN architecture is based.
Cerf, et al. Expires February 2003 [Page 16]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
A message-switched architecture provides the network with a priori
knowledge of the size and performance requirements of requested data
transfers. When there is a significant amount of queuing that can
occur prior to transmission over an outbound route (as is the case in
the DTN version of store-and-forward) the advantage provided by this
information provided is significant. In an environment that
frequently experiences network partitioning, messages are efficient
units for queuing while awaiting network reconnection.
3.2 DTN Classes of Service Mimic Postal Operation
The (U.S. or similar) Postal Service provides a strong metaphor for
the services that a Delay-Tolerant Network offers. Traffic is
generally not interactive. It may be one-way in nature. There are
generally no strong guarantees of timely delivery.
The DTN Architecture, like the Postal Service, offers *relative*
measures of priority (low, medium, high). It may offer basic
notifications, for example: "the intended recipient has accepted
delivery of the message," "the route taken by this message is as
follows..." and "the message has been transmitted."
An essential element of Postal Service operation is that mail has a
place to wait in queue until an outbound hop is available. This
highlights the previously stated assumptions:
1. That there is storage available in the network,
2. that storage is well-distributed throughout the network,
3. that it sufficiently persistent to store bundles until custody
transfer can occur, and
4. (implicitly) that this 'store-and-forward' model is a better
trade-off than using resources to attempt to effect continuous
connectivity or other alternatives.
For a network to effectively support the DTN architecture, these
assumptions must be considered and must be found to hold.
3.3 Regions and Region Identifiers
The DTN architecture defines a "network of Internets." Each of these
Internets is called a "DTN region." Regions may be delimited by
differences in network protocol, transport protocol, or addressing
mechanism. Regions may also be delimited by other criteria, such as
trust boundaries [NEWARCH].
Each DTN region has a unique identifier that is known (or knowable)
among all regions of the DTN.
Cerf, et al. Expires February 2003 [Page 17]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
All inter-region communication takes place via DTN *gateways*, which
provide the interconnection points between regions. These correspond
to "waypoints" in [META], and to the gateways described in the
original ARPANET design [CK74]. DTN gateways differ from ARPANET
gateways because they operate above the transport layer and are
focused on message switching rather than packet switching. However,
both DTN gateways and ARPANET gateways provide conversions between
the protocols specific to one region and the protocols specific to
another.
Region definition and region identification are key concepts in the
DTN architecture. DTN bundles that originate outside the destination
region are first transmitted to one of the DTN gateways that connect
the source region with one or more other regions. Routing outside
the destination region is based on the destination region's
identification, meaning that there is a single region identifier
space that is common across the end-to-end DTN.
3.4 Tuples
The region identifier described above is sufficient to route a bundle
of data to its destination region, but not to deliver it to the
specific entity for which it is intended. The DTN architecture uses
a tuple of the region identifier and a region-specific entity
identifier to identify uniquely a particular entity in the DTN.
The tuple notion allows us to distinguish data that is necessary for
routing *across* regions from that which is necessary for delivery
within a region. As a result of this, the entity ID is *opaque*
outside the region of definition. The referenced entity might be a
host, a protocol, an application, or something else. Further, the
entity identifier might be a name, an address, or a descriptor such
as a URL. This is a local issue within the entity's region. (If the
entity ID is a name, there is an assumption that a name-to-address
binding function exists within the region to support eventual routing
of the bundle to the destination entity.)
3.5 Late Binding
The opacity of entity IDs outside their local region enforces another
key concept in the DTN Architecture: that of late binding. Late
binding refers to the fact that the entity ID of a tuple is not
interpreted (e.g., bound to an address) outside its local region.
This avoids having a universal name-to-address binding space (and its
associated database and synchronization issues).
This approach preserves a significant amount of autonomy within each
region. The entity IDs used in bundles might be built on DNS names,
Cerf, et al. Expires February 2003 [Page 18]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
or URLs, but they might also be "expressions of interest" as in a
directed diffusion-routed network [IGE00].
Additionally, late binding avoids the delay-related issue that the
binding information might be highly ephemeral relative to the transit
time of a bundle. We assume that the internal details of a
destination network will be sufficiently stable for the duration of a
transmission of a message within that region, or that delay-tolerant
mechanisms will be employed *within* the region to compensate. (This
is, by definition, a local issue within the region and may be
accommodated in whatever manner is most practical for that region.)
3.6 The "Bundle Layer" Terminates Local Transport Protocols and
Operates End-to-End
The bundle layer provides an end-to-end protocol that employs custody
in-network retransmission, both point-to-point retransmission between
transport-layer endpoints and also custody transfer between (possibly
non-adjacent) DTN nodes. The bundle layer makes use of the
reliability mechanisms available from the underlying transport layers
of each region, and a single bundle-layer hop *may* span an entire
region.
3.7 Routing
The bundle layer provides routing among DTN-capable nodes, including
DTN gateways. There may be many DTN gateways that interconnect
adjacent regions, and there may be multiple bundle routing "hops"
within a region (particularly if intra-regional connectivity is
intermittent).
Routing information exchange mechanisms may differ from one
instantiation of the Delay Tolerant Network Architecture to another,
reflecting different needs and constraints of the extreme
environments. Different routing protocols may be employed for intra-
region routing and for inter-region routing. We do not require that
the *same* inter-region routing protocol run among all (inter-region)
DTN gateways in a Delay Tolerant Network. (It is required that
necessary routing information be able to be propagated throughout the
DTN, but the mechanisms for doing that are neither defined nor
constrained by the DTN Architecture.)
3.8 Bundle Layer Reliability and Custodianship
The bundle layer provides three types of reliability services: end-
to-end acknowledgment of a bundle (and accompanying retransmission),
custodial transmission (with in-network retransmission if required),
and unacknowledged bundle transmission. End-to-end acknowledgment
and custodial transfers may be combined for additional reliability.
Cerf, et al. Expires February 2003 [Page 19]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
Reliability services at the bundle layer are relatively simple
because the bundle layer depends upon the reliability mechanisms in
each of the underlying internets. Bundles are delivered atomically
and are, generally, independent of other bundles. As a result,
positive acknowledgment is possible, but partial or negative
acknowledgment is not. Since delays may be long and/or highly
variable, retransmission timers must, in general, be coarse and may
be strongly affected by local policy.
Custody transfer, introduced in section 2.3.4, above, allows the
source to delegate retransmission responsibility and recover its
retransmission-related resources relatively soon after sending the
bundle. While not every node in a DTN must be capable of providing
custodial services, all DTN gateways (that span two or more DTN
regions) are expected to be able to be custodians.
3.9 Security
Resource scarcity in delay-tolerant networks dictates that some form
of access control is required. It is not acceptable for an
unauthorized user to be able to flood the network with high-priority
traffic, possibly preventing the network's mission from being
accomplished. Implicit in this statement is a requirement for some
form of admission control and/or in-network authentication that is
sensitive to the class of service that a user has requested, and the
means to verify that the user is authorized to make that request. In
a low delay environment, this would simply be tedious. In a
high/variable delay, low data rate environment, it is potentially
much worse: access control lists are difficult to update, re-keying
presents hard problems, and routers or end nodes might be
compromised.
Key management presents a number of dilemmas. Keys will almost
certainly require replacement in all but the most disposable of
networks. Re-keying and related operations on highly remote nodes is
challenging, and proper approaches to key distribution in this
environment remain active areas for research.
The DTN must be able to support end-to-end user services that include
verification of a message's integrity and the ability to provide
confidentiality of a user's message. It must do this in a way that
does not require a great deal of interaction across long-delay (or
resource-constrained) data links. "Support" for these services may
involve provision of a key management service to support user-level
confidentiality and integrity, or it may involve providing these
services as part of the bundle layer. This is an area still under
consideration.
Cerf, et al. Expires February 2003 [Page 20]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
3.10 Time Synchronization
The DTN architecture depends on time synchronization for two primary
purposes: routing, and "time to live" computations.
Routing that is based on schedules and dependent upon coordination of
shared assets (such as directional antennas) creates a requirement
for time synchronization. As with many requirements, just how *much*
time synchronization is required is an economic trade-off. The cost
to achieve a particular degree of time synchronization must be
weighed against the cost of not having it. The costs of not having
time synchronization include reduced overall efficiency of a
communication asset, potential loss of user data, power inefficiency,
and many others. For some communication assets, such as NASA's Deep
Space Network, the operational cost is many thousands of dollars per
hour, and the cost to achieve a modest level of time synchronization
between peers is quite justifiable. There is a study currently
underway [DM02] to examine the time synchronization issues across
NASA's Deep Space Network and to determine a particular time
synchronization scheme suited to that network.
Given that DTN routing depends to some degree on time
synchronization, the DTN architects have made the decision to exploit
its availability. In particular, time to live computations may be
done by including a source time stamp and an explicit time to live
field (in time units after the time in the source time stamp). This
requires some level of time synchronization, but since its sole use
is for purging data from the network, the synchronization
requirements are not strict. This approach allows a source time
stamp to be used for a number of purposes, such as to provide replay
protection and to identify individual messages.
4 Service Considerations: Application Instances and Bundles
This section describes the basic services available to applications
using the DTN Architecture's Bundle Service. The Bundle Service is
the message-oriented communication service provided by the "Bundle
Layer" described in section 3.6.
4.1 Networking Style Issues
Before discussing specific Bundle Services, a note about application
interaction style is in order. This service is intended to operate
in *delay-tolerant* networks. That does not mean that there *must*
be delays in the network, or that there *will* be delays, but that
there *may* be delays. The protocols providing the services exposed
by the bundle layer are delay tolerant. Applications using those
services should be, too.
Cerf, et al. Expires February 2003 [Page 21]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
Message-oriented communication differs from conversational
communication. In general, applications should attempt to include
enough information in a bundle so that it may be treated as an
independent unit of work by the remote entity (a "transaction",
although transactions carry connotations of multi-phase commitment
that we do not intend here). The goal is to minimize conversation
between application entities that are separated by a network that
presents long and possibly highly variable delays, and to constrain
any conversation that does occur to be asynchronous in nature. A
file transfer application, for example, might incorporate account
information, file location information, file operation information,
and file content within a bundle, allowing atomic operation on that
bundle. Comparing this style of operation to a classic FTP transfer,
one sees that the bundled model can complete in one round trip time,
whereas an FTP file "put" operation can take as many as eight round
trips to get to a point where file data can flow [WhynotTCP].
Delay-tolerant applications must consider additional factors beyond
the conversational implications of long delay paths. Application
instances may terminate (voluntarily or not) between the time that a
bundle is sent and the time its response is received. If an
application has anticipated this possibility, it is possible to
reinstantiate the application instance based on state information
saved in persistent storage. This is an implementation issue, but
also an application design consideration. Similarly, either host or
application mobility may result in the application's address having
changed by the time a response is received. If applications embed
physical addresses into their data, they will defeat the potential
benefits that late binding provides.
Some consideration of delay-tolerant application design can result in
applications that work well in low-delay environments, and that do
not suffer extraordinarily in high or highly-variable delay
environments.
4.2 Bundle Lifetime
Applications are requested to provide an expiration time (actually, a
time to live in seconds) for a bundle if the value of the data has a
finite duration. If not supplied, or if the user-supplied value is
larger than local policy permits, the bundle layer will provide a
value. Note that this value is treated as an actual "time to live" -
- it is added to the time that a bundle was submitted by the
application to determine the time at which the bundle will be purged
from the network. Appropriate values depend on the network and the
data, and may range from seconds to weeks.
Cerf, et al. Expires February 2003 [Page 22]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
4.3 Classes of Service
The DTN architecture provides for a range of classes of service.
These classes of service are requests to the bundle layer to provide
*relative* distinctions in the immediacy with which bundles are
transmitted.
We have currently defined three classes of service in the DTN
architecture:
. Bulk - In bulk class, bundles are shipped on a "best effort"
basis. No bulk class bundle will be shipped until all bundles of
other classes bound for the same next hop destination have been
shipped.
. Normal - Normal class bundles that are in queue and bound for the
same next hop destination are shipped prior to any bulk class that
are in queue.
. Expedited - Expedited bundles, in general, are shipped prior to
bundles of other classes. However, the bundle layer *may*
implement a queue service discipline that prevents starvation of
the normal class, which may result in some normal bundles being
shipped before expedited bundles bound for the same next hop
destination as the normal class bundles.
4.4 Delivery Options
The bundle layer offers a number of delivery options to users. First
and foremost is the option of custodial delivery. Custodial delivery
means that a bundle will be reliably transmitted by the bundle layer,
and that the point of retransmission at the bundle layer will advance
through the network from one custodian to another until the bundle
reaches its destination. The bundle layer depends on the underlying
transport protocols of the networks that it operates over to provide
the primary means of reliable transfer from one bundle layer entity
to the next. However, when custodial delivery is requested, the
bundle layer additionally provides a coarse-grained timeout and
retransmission mechanism and an accompanying acknowledgment
mechanism. When a bundle application does *not* request custodial
delivery, this bundle layer timeout and retransmission mechanism is
not employed, and the bundle depends solely on the reliability
mechanisms of the underlying transport layers.
A second delivery option is the option of a return receipt. An
application may request a return receipt for a bundle that is being
transmitted. When the subject bundle is received *by the destination
application instance* (not simply the destination bundle layer
entity), a return receipt is generated by the bundle layer entity and
returned to the source or the source's designated alternate (reply-
to) application instance, which would typically be located on a
Cerf, et al. Expires February 2003 [Page 23]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
different host. The return receipt uses the same custodial delivery
option used in the bundle that caused the return receipt to be sent.
(Return receipts are never issued with the return receipt delivery
option requested.)
The third and fourth delivery options are used for diagnostic
purposes, and may not be available to all users, subject to local
policy. These two delivery options result in notifications being
sent to the reply-to application. The first causes a notification to
be sent every time a bundle is forwarded. The second option causes a
notification to be sent every time a custody transfer occurs.
4.5 Naming and Addressing
As described in section 3.4, bundle application instances refer to
one another using tuples. The tuple consists of two parts: a region
identifier and an entity identifier. DTN regions will be discussed
in more detail in section 5.2, but they are named by applications
using syntax identical to that used in the domain name system (DNS).
(That is, hierarchical tree structure, dot-separated text node names,
left to right progresses from leaf to root, sibling nodes must have
different names.) When a region is served by the Internet's Domain
Name System distributed database, DTN region names may or may not be
stored in that database. The bundle layer may translate a region
name to a bundle-layer-specific region address for transmission. The
scope of the region name space and region address space spans an
entire DTN, and name-to-address translations of DTN regions must be
consistent and reversible throughout the DTN.
The other portion of a tuple (the entity identifier) is opaque
outside the region specified by the tuple's region identifier (the
"home region" for the entity). In internet-served regions, an entity
ID may be a hostname, protocol identifier, port (used to find the
bundle service on the destination host) and potentially a token used
for demultiplexing to a particular application using the bundle
service. These could potentially be combined into a URL (depending
on the region's internal rules). In diffusion-routed regions, they
may be an expression of interest [IGE00]. In any case, no attempt
shall be made by the bundle layer to bind the entity ID to an address
from any location outside the entity's home region.
Discovery of valid DTN region names and entity identifiers is out of
the scope of this document.
5 Topological Considerations: Nodes, Regions, and Gateways
This section describes the three key topological elements in the DTN
architecture and how they are related: DTN nodes, DTN regions, and
DTN gateways.
Cerf, et al. Expires February 2003 [Page 24]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
5.1 Nodes
A DTN node (or "node" in this document) is an engine for sending and
receiving DTN messages (bundles). DTN nodes may act as sources,
destinations, or forwarders of bundles. There is a one-to-one
mapping between DTN nodes and DTN entities. Each node is uniquely
identified by at least one tuple containing region identifier and
entity identifier; a node that is reachable within multiple regions
will have at least one identifying tuple for each region within which
it is reachable.
The identifier for a DTN node, meaning the bundle sending and
receiving engine itself as opposed to an application *using* that
engine, is specified in a region-specific manner using all of part of
the entity identifier.
5.2 Regions
The following list identifies the requirements for DTN regions:
. Each DTN region shall have an identifier space that is shared by
all DTN nodes within the region. A region must specify the naming
conventions to be employed within that region for entity
identification.
. Each node that is a member of the region shall have a unique
identifier drawn from that identifier space. (Note that for some
types of regions, a "node" may be made up of a collection of
computational elements, possibly geographically distributed. A
single unique identifier may collectively refer to them. Further,
the unique identifier requirement only applies to nodes that are
intended to receive data from other DTN nodes.)
. To be considered a member of a region, each prospective member of
the region shall have the ability to reach every other member of
the region without depending on any DTN nodes outside the region.
(Although a DTN node may not necessarily be *directly* reachable.
This may require forwarding and/or store-and-forward operation by
other DTN nodes inside the same region.)
A DTN region is a generalization of the notion of a "network" that
relaxes the assumption of real-time end-to-end connectivity.
DTN regions may be created for a number of reasons. Accommodation of
tailored support for a particularly challenging networking
environment is one reason for defining a DTN region. Delimiting a
particular security boundary may be another reason for defining a DTN
region boundary.
Cerf, et al. Expires February 2003 [Page 25]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
5.3 Gateways
A gateway is an entity that has membership in two or more DTN regions
and relays messages between regions.
Entities that reside (only) in one region can only send messages to
entities in other regions with the assistance of one or more DTN
gateways.
5.4 Discussion
DTN nodes may act as "hosts," meaning entities that send and receive
bundles, but do not forward them. DTN nodes may also act as
"routers", meaning they perform the functions of a host, but also
forward bundles within a single region. Finally, DTN nodes may act
as gateways, forwarding bundles across region boundaries. A single
DTN node may act in any subset of these roles simultaneously.
As members in a store-and-forward chain, DTN nodes have the
responsibility for resource allocation to support bundle transfers.
These resources include, among other things, buffer space and
transmission capacity.
Additionally, DTN nodes have the responsibility of actually executing
the bundle transfer. Reliability requirements for bundle transfers
are specified by the using application, and include both reliable and
unreliable transfers. The DTN nodes are responsible for using
whatever reliability mechanisms exist in the underlying (transport-
and-below) layers, and augmenting those mechanisms with bundle-layer
mechanisms to effect the required reliability.
We require that each DTN region have a unique identifier that is
known among all regions in the DTN or that can be discovered as
required.
The entity identification rules for a particular DTN region have
scope only within that region, and are opaque outside that region.
However, all DTN nodes within a region must know and conform to the
entity identification rules in order to successfully communicate.
6 Routing considerations
Communication within a DTN can be either intra-regional or inter-
regional. While this architecture does not prescribe specific
methods for the exchange of routing data, it acknowledges the fact
that mechanisms for the exchange of intra-regional bundle routing
data may well be quite different from the mechanisms to exchange
inter-regional routing information. Further, the architecture
Cerf, et al. Expires February 2003 [Page 26]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
recognizes that mechanisms for inter-regional transfer in one portion
of a DTN may be inappropriate for use in other portions of that DTN,
and does not require homogeneity of inter-region routing protocol
throughout the DTN.
6.1 Types of Routes
DTNs may be required to function in the presence of any or all of the
following types of routes. These are presented as information rather
than as requirements for all DTNs, although any of these may be
required for a *particular* DTN.
6.1.1 Persistent Contacts
Persistent contacts are sessions with a neighboring DTN node that are
always available or that can be made available on demand. In the IP
world, Digital Subscriber Line (DSL) connections are an example of
the former, while dial-up connections are an example of the latter.
6.1.2 Intermittent - Scheduled Contacts
Scheduled routes are those where there is an agreement to establish a
link between two points at a particular time, for a particular
duration. Attributes of that link, such as its data rate,
probability of loss, and the routing metrics to destinations via that
link, are routing protocol-specific.
An example of a scheduled route is a link that uses a low-earth
orbiting satellite. A schedule of view times and durations can be
constructed when next-hop neighbors are accessible via the satellite.
6.1.3 Intermittent - Opportunistic Contacts
Opportunistic contacts are those that are not scheduled, but rather
present themselves unexpectedly. Such contacts could be with an
aircraft flying overhead and beaconing, advertising its availability
for communication. Another type of opportunistic contacts might be
via infrared communication link between a personal digital assistant
(PDA) and a kiosk in an airport concourse offering bundle service as
the PDA's owner walks by. If the PDA's owner authorizes it, the PDA
could communicate the owner's planned path and a desire for contacts
with subsequent kiosks in the concourse, resulting in a set of
probable contacts for which bundle transfers can be scheduled.
6.1.4 Intermittent - Predicted Contacts
Cerf, et al. Expires February 2003 [Page 27]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
Predicted contacts are those that are based on no fixed schedule, but
a history of opportunistic contacts that suggests that contact with
an intermittently-connected neighbor will probably occur within a
certain period of time and will probably last for a certain duration.
Given a great enough certainty that the contact will occur, a DTN
node may allocate bundles to that predicted contact period that would
be allocated to different contacts otherwise. In the previous
example, the probable contacts in the airport concourse would result
in the establishment of a set of predicted contacts of a given
duration (perhaps based on the PDA owner's walking speed and the
kiosk's coverage area). The PDA could decide how to use those
contacts for transmitting waiting bundles, as well as perhaps to
request bundles that were awaiting delivery at any of a number of
store-and-forward points to which the user had access.
The algorithms for establishing the predicted time and duration of a
contact, the degree of uncertainty in those estimates, the time at
which to abandon the wait for a predicted contact, and the guidelines
for allocating bundles to such contacts are all open research areas.
6.2 Store-and-forward operation
While IP networks are based on "store-and-forward" operation, there
is an assumption that the "storing" will not persist for more than a
modest amount of queuing and transmission delay. In contrast, the
DTN architecture does not expect that an outbound link will be
available when a bundle is received, and expects to store that bundle
for some time until a link does become available. We anticipate that
most DTN nodes will use some form of persistent storage for this --
disk, flash memory, etc.
All DTN forwarding nodes ("DTN routers"), even those that do not
provide custodial operations as described in section 3.8, must be
able to queue bundles until outbound contacts are available. Each
DTN forwarding node and DTN gateway that also provides custody
transfer operations must provide for longer-term storage of bundles,
committing to store bundles for which it takes custody for the entire
remainder of their lifetime, if necessary. It is entirely possible
that a custodian will need to "take a break" from further
custodianship while it completes pending custodial operations and
recovers persistent storage. Two mechanisms support this: First, the
node can simply forward incoming bundles without taking custody. The
fact that a node is a potential custodian is no guarantee that it
will take custody of any given bundle that is routed to it. Second,
the node can revise its advertisement of custodial capability in
routing updates (see section 6.4). This is a longer-term solution,
but has the attractive property that DTN nodes searching for a
custodian do not route a bundle out of its way vainly in search of
custodianship at the node in question.
Cerf, et al. Expires February 2003 [Page 28]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
6.3 Contact-oriented routing
Making routing decisions to allocate bundles to future contacts
requires a DTN forwarding node to probabilistically estimate what the
connectivity to other points in the network will be via that contact.
We do not yet have a significant body of experience with this, but we
suspect that maintaining a weighted average of previous routing
metrics and developing an uncertainty metric will be useful.
Based on the expected connectivity to the remainder of the network
from each of these intermittent contacts, and similar
characterizations of connectivity for persistently connected
neighbors, a DTN forwarding node can make decisions about how to
allocate bundles to contacts.
One could think of this allocation of bundles to contacts as being
made by creating an ordered list of references to the bundles in
persistent storage and providing that list to the lower layer
protocol (or to the convergence layer) for transmission at the start
of a contact. (We refer to such a list of bundles as a "contact
list.") Bundles not transmitted during the contact could be returned
on the list for allocation to subsequent contacts. With such an
approach, the bundle layer is certainly free to adjust the allocation
of bundles to future contacts, based on the receipt of new, possibly
high priority bundles. It is also possible that the bundle layer
could make changes to a contact list during the course of a contact,
if the implementation permitted.
6.4 Routing protocols
We have not yet developed routing protocols for this architecture,
but envision the need for at least two: one to support inter-region
routing, and at least one to support intra-region routing.
Realistically, there will probably be more than two. It is likely
that an inter-region routing protocol for a network having one or
more links with 40-minute one way delays and two-week link outages
(such as might be encountered in a deep-space mission) would be
wholly different than an inter-region routing protocol to support a
sensor network connected to the Internet via a low-earth orbiting
satellite.
Our current thinking in routing protocols to support intermittent
connectivity is to maintain one-hop link state information and a
distance-vector type of aggregation beyond one hop. We feel that
attempting to maintain an "accurate" picture of network connectivity
beyond one hop when routes are composed of scheduled or probabilistic
connections of uncertain capacity will be futile. However, we
Cerf, et al. Expires February 2003 [Page 29]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
believe that a probabilistic view of connectivity can be built and
advertised in a scalable way that permits effective use of these
intermittent contacts. The fact that there is no expectation of a
contemporaneous end-to-end path makes the overall job seem more
feasible.
As previously mentioned, this area is teeming with interesting
research topics. We intend to address some of these in coming
months, but we certainly solicit the participation of interested
researchers.
7 Security considerations
While the particular security problems presented by delay-tolerant
networks do not necessarily differ from those presented by other
networks, the environment has a significant effect on the available
solution space.
In addition to typical end-user oriented security, providing writer-
to-reader services such as message confidentiality or end-to-end user
authentication, protection of the network infrastructure and
controlling access to that infrastructure tend to be important in
delay-tolerant networks, which are typically resource-challenged and
are critical components to mission success. Along with high round-
trip times and frequent partitioning, delay-tolerant networks often
have low data rate links, making efficiency a necessary consideration
in any solution. It is not unusual for delay-tolerant networks to be
deployed in places where a portion of the network might become
compromised. Such compromise, should it occur, must be limited in
scope and the effect finite in duration.
7.1 User-oriented security services
We have not received explicit user requirements for the following
security services, but anticipate their need. We are as yet
undecided whether to provide confidentiality and user authentication
as explicit bundle-layer services (a la IPSEC) or to simply make the
key management and distribution mechanisms we require available to
user applications for them to provide their own confidentiality and
authentication (a la PGP, SSL, etc.). Current discussions have
focused on using a combination of public key and symmetric key
cryptographic methods for the security services.
7.1.1 Confidentiality
The confidentiality service attempts to insure that the information
content of a bundle is not disclosed to anyone except the intended
recipient. If the bundle layer provides the confidentiality service
(a la IPSec Encapsulating Security Payload) the intended recipient
Cerf, et al. Expires February 2003 [Page 30]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
will be the bundle layer entity at the destination (rather than any
specific application). In a public key based system, this requires a
high confidence that the sender has the public key of the intended
recipient and that said key is unique. The confidentiality service
will render all affected portions of the bundle unreadable, so it
cannot be applied to fields in the bundle header that are needed for
delivery. In general, the bundle user data is covered by the
confidentiality service but none of the rest of the bundle header is
covered.
7.1.2 User Authentication
User authentication is a capability to prove that the bundle has been
sent by the entity that claims to have sent it. Typically, an
authentication service ensures that the content of the bundle is
unaltered as a means to prevent replay attacks. Authentication is
accomplished in a public key system by the use of a digital
signature. A digital signature is created by computing a hash of the
data to be covered by the authentication, and then encrypting that
information with the private key of the sender. If the sender's
public/private key pair is uncompromised and the cryptographic
algorithm is sufficiently strong, the recipient can apply the
purported sender's public key to the encrypted data, recover the
hash, and compare it to a hash of the corresponding information in
the bundle. This allows the recipient to determine if the sender's
key is correct (authenticating the sender's identity), and if the
data received is identical to the data that was sent (verifying the
integrity of the data). Note that this service is required to
support the infrastructure security model described in section 7.2,
and hence could be made available to end-users to authenticate them
to the remote bundle entity at relatively low cost. However, the
remote bundle entity will have to request a trusted copy of the
purported sender's public key, potentially adding considerable delay
to the receipt of the first bundles until a signed copy of the
sources public key is received, verified, and cached.
7.1.3 Key Management and Distribution
The DTN Architecture requires that a key management service be
available that can produce, upon request, a set of bytes that are
useful to the recipient in verifying the cryptographic correctness of
an information object.
An additional service is required that allows the verifier to
determine the status of the above-mentioned cryptographic
verification information.
Cerf, et al. Expires February 2003 [Page 31]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
This description is intentionally abstract, as these services may
vary with the specific cryptographic mechanisms that are selected
(e.g., Kerberos tickets, CRLs with X.509, etc.).
7.2 Infrastructure security services
The following model describes an approach to provide infrastructure
and data security for a delay tolerant network. It is based on the
sensitivity to the effects of long delays, constrained bandwidth,
constrained data rates, and intermittent connectivity. Like the rest
of this architecture, this security model is preliminary and is still
a work in progress.
The security architecture is loosely based on the model of secure
electronic mail. In the secure electronic mail environment, a source
sends secured electronic mail to a destination only knowing the
destination's email address. The security attributes of the
electronic mail may be that the contents have been encrypted,
digitally signed, or both encrypted and signed.
Encryption provides "writer-to-reader" confidentiality. That is,
only the source and the destination will be able to read the contents
of the message. Digital signatures provide authentication of the
source of the message as well as message content integrity. In this
way, the receiver of the message is assured that the message came
from the claimed source and that the message contents have not been
tampered with in transit ¡ what is received is exactly what was sent.
In the secure electronic mail environment, few assumptions are made
about the receiver of the mail. When a message is digitally signed,
the sender's private key is utilized. In order to assure that the
receiver will be able to authenticate the signature, the sender's
authenticated public key is sent along with the message to the
receiver. The receiver is able to extract the sender's public key
from the message, verify the public key's authenticity based on its
being signed by a trusted-third party (e.g., a Certificate Authority
(CA)), and then use the verified public key to verify the message
digital signature. The receiver needs to know nothing, security-
wise, about the sender other than having a copy of the public key of
the certificate authority (or equivalent).
In order to encrypt the message contents, the sender must have the
receiver's public key. By virtue of public key cryptography, only
the receiver, using its private key, will be able to decrypt what has
been encrypted in the receiver's public key. Typically, the message
contents are encrypted in a randomly selected symmetric key. The
symmetric key is then encrypted using the public key of the receiver.
The receiver is then able to use its private key to decrypt the
symmetric key and then use the symmetric key to decrypt the message
Cerf, et al. Expires February 2003 [Page 32]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
payload. However, in this case, unlike the digital signature-only
case, the sender must have some information about the receiver other
than a destination address. The sender must have previously obtained
the receiver's public key via a previous exchange, or must have
retrieved it from a key server.
In a DTN, key servers may or may not exist. In addition, bandwidth
to communicate with key servers may be extremely limited or the delay
to accomplish a round-trip may be intolerable. Nevertheless, there
still needs to be a means by which public keys may be obtained or
distributed.
For this architecture, we make the assumption that DTN systems that
require public keys will have the means to acquire them. They may be
able to generate their own public/private key pairs (as in the PGP
model) and then have the public key certified as belonging to the
owner by a trusted third-party. In this way, a DTN system may only
need to expose its public key to the trusted third-party and does not
have to trust the third party with its private key at all. The
downside to this is that the pedigree of the key generation software
may be suspect. That is, the key generators may exist on the
specific DTN system but may have been compromised or corrupted. As a
result, the keys generated on directly on a DTN system might be weak
or otherwise bad keys.
Alternatively, a DTN system's keys may be generated and certified by
a trusted third-party agent (e.g., Verisign, Thawte, Baltimore).
Because the trusted third parties base their livelihood on the
goodness of the keys they generate and the trust they provide through
their certification of keys, the danger of a corrupted key generator
producing weak keys is minimized.
Key pairs are required by both end-systems (e.g., principal
investigator workstations, space-based instruments, terrestrial-based
instruments in a sensor web) as well as intermediate DTN systems
(e.g., DTN routers). The end-systems will use their key pairs for
both admission control to a DTN as well as for writer-to-reader
confidentiality, authentication, and integrity when required. DTN
routers will make use of their keys in order to provide assurance
that only certified DTN systems have the ability to route data
through a DTN. In this manner, DTN systems implement "mutual
suspicion." That is, they do not trust each other without each
providing proof of identity. It is desirable that all DTN router
interactions are authenticated, however that choice will have to be
left up to the DTN implementer/operator.
Four types of DTN elements have security mechanisms and services
associated with them: an end-user system, a DTN router, a DTN
Cerf, et al. Expires February 2003 [Page 33]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
interregional gateway (DTN gateway for short), and a DTN certificate
authority (CA).
As was already stated, each of the DTN elements will have a
public/private key pair in place either self-generated and then
certified by a DTN CA, or generated by a DTN CA and then populated as
necessary in the DTN elements. The final decisions about key
management, key/certificate distribution, and key/certificate
revocation have not been decided yet and are therefore left for
future debates. The key pairs may be pre-placed in the DTN elements.
Alternatively, they may be exchanged during a "neighbor discovery"
process. Or finally, they may be exchanged in the same manner as
with secure electronic mail. That is, they may be sent in a
certificate along with the payload. The certificates should be
cached by the recipient systems. The "safe" method to ensure that
recipients always have the most up-to-date keys would be to send the
certificate with every payload. However, because of bandwidth
constraints, this may prove to be impractical and therefore an
intelligent mode of operation should be employed where state
information is maintained regarding communicating end-points to
determine if the end-points already have the necessary certificates
from previous communications.
When an end-system requires admission to a DTN, that is, when it
needs to inject data into a DTN, it first presents its credentials to
a DTN "gatekeeper" which is analogous to a present day Radius (or
other flavor) access control server. The end-system's credentials,
based on a certified public key are checked for authenticity and then
an access control list is consulted to determine if the end-system
should be allowed access.
If the end-system is authenticated and access is allowed to a DTN,
the end-system must decide whether or not end-to-end security
services are required. That is, the end-system must decide whether
"write-to-reader" confidentiality, authentication, or integrity is
required. If end-to-end services are required, then the end-system
may encrypt the payload with a randomly chosen key and then encrypt
the key using the public key of the intended recipient of the
payload. This would be the equivalent of using Secure Sockets Layer
(SSL) or its IETF-version, Transport Layer Security (TLS) at the
application layer. Alternatively, the end-system could request and
end-to-end service from the bundle layer, if it is chosen to
implement such a service at the bundle layer. If this is done, the
ends of the end-to-end services would be the sender and receiver
bundle agents rather than the actual end-systems. This is because
the bundle agents have acted on behalf of the end-system to provide
end-to-end security services.
Cerf, et al. Expires February 2003 [Page 34]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
Regardless of whether end-to-end security services are required or
not, the initial DTN router or gateway that receives the payload from
an end-system applies its own digital signature to the bundle so that
subsequent DTN routers and gateway can be assured of hop-by-hop
authenticity. Likewise, when DTN routers and gateways perform
housekeeping chores (e.g., routing updates, etc.) those bundles are
also digitally signed in order to assure that only secured
transactions between the DTN entities occurs.
When a DTN router or DTN gateway receives a bundle, and before it
forwards it onward to a neighbor DTN router or gateway, it first
validates the bundle by authenticating digital signature from its
precursor hop. As the bundle progresses through a DTN, each
subsequent DTN device performs an authentication function and then
forwards the bundle onwards with its own digital signature to its
next hop.
8 Reliability Considerations
The bundle layer depends on underlying transport services to provide
hop-by-hop reliability. In this context, "hop-by-hop" means from one
DTN node to the next. Those nodes may be separated by *many* hops in
an underlying network, and the DTN architecture intends to exploit
the reliability, flow control, and congestion control capabilities of
those underlying networks. However, some significant reliability
issues remain at the bundle layer. This section discusses the issues
that we have currently identified.
8.1 Custodial Operation
One of the fundamental tenets of the DTN architecture is that we must
be able to provide reliable end-to-end message transfer via an
infrastructure that may *never* be connected end-to-end
contemporaneously.
Because no single transport-layer protocol operates end-to-end across
the IPN, end-to-end reliability can only be assured at the bundle
layer. At each node along an end-to-end route, the bundle-layer
protocol entity passes bundle data to the Transport layer for
transmission. Each bundle layer entity is highly confident that the
transport layer will successfully convey the data entrusted to it to
the next bundle-layer protocol entity (which may "take custody" of
the data or merely relay it; a single hop). But failures are
possible (e.g., a host computer does an unplanned reboot).
A bundle custodian makes a commitment to store the subject bundle
until one of two events occurs: another DTN node accepts custody of
the bundle, or the bundle's time to live expires. The bundle layer
attempts to deal with bundles atomically and independently of other
Cerf, et al. Expires February 2003 [Page 35]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
bundles. In this model, positive acknowledgement of a bundle is
possible, but negative acknowledgment is not. We consider the
opportunity to *not* deal with partial bundles a significant plus.
Since the underlying transport protocols are working on our behalf,
we do not consider the lack of a negative acknowledgment capability a
significant minus. (Note that there actually *is* a circumstance in
which a negative acknowledgment can be sent -- when the time to live
for a bundle expires. However, at that point, *all* copies of the
bundle, including those stored at a custodian, are purged from the
network. A "timed-out" error can be sent to the source, which
*might* be able to cause either an application layer retransmission
or a bundle-layer retransmission (see section 8.2, below).)
Without a negative acknowledgment mechanism, retransmission must be
accomplished solely via timer expiration. The bundle layer's
confidence in the effectiveness of the underlying transport-layer
protocols is reflected in the design of the timers for bundle-layer
reliability. These timers are highly optimistic ¡ that is, they
expire as late as possible ¡ in order to give the transport protocols
every opportunity to complete reliable transmission. The effect of
this optimism is to minimize the chance of unnecessary bundle-layer
retransmission, which could seriously degrade DTN performance by
consuming valuable bandwidth.
8.2 End-to-end Retransmission
Even custody transfer operating on a bundle-hop by bundle-hop basis
does not provide guaranteed end-to-end reliability. This can only be
achieved with an end-to-end reliability mechanism operating between
source and destination. In most cases, custodial reliability should
suffice. However, the architecture also provides for end-to-end
reliability using retransmission mechanisms that are in addition to
those used for custody transfers.
The availability of timer-based retransmission and acknowledgment
mechanisms to support custody transfers means that the mechanisms to
support end-to-end retransmission at the bundle layer are essentially
available for free. If a DTN user has requested both custodial
transfer and the return receipt delivery option, it is reasonable to
assume that end-to-end retransmission would be appropriate if, for
some reason, the custody transfer operation failed. However, for
timer-based retransmission, we must set the retransmission timer to
some reasonable value to account for the diligent efforts and
ultimate failure of the underlying transport protocols to succeed in
either the end-to-end transfer or the end-to-end return receipt.
For custody transfers, appropriate retransmission timer setting is
less of an issue, since presumably the transfer between the current
custodian and the next-hop custodian traverses fewer hops carried by
Cerf, et al. Expires February 2003 [Page 36]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
underlying transport protocols. One could reasonably expect better
predictions of this round trip time than one could for an end-to-end
transfer.
The result is that a conservative (optimistic) retransmission time
out value for an end-to-end transfer will likely be VERY large. And
that value plus the anticipated end-to-end delay for a retransmitted
bundle must be less than the time to live for the user's data or else
the retransmitted data will time out in transit. If a bundle timed
out in transit and the source received the indication of that error,
the bundle layer could respond with an end-to-end retransmission,
assuming that the remainder of the application's statement of the
time-to-live for that bundle to have a reasonable chance of end-to-
end transfer success.
So, while we believe that the mechanisms are available to effect end-
to-end retransmission, we are unsure whether it will be of value in
all DTN environments. It is possible that it may provide value in
certain environments, and we will continue to consider whether it
should be provided as an implicitly-requested capability when users
request both custody transfer and return receipt.
8.3 Congestion and Flow Control at the Bundle Layer
The subject of congestion control and flow control at the bundle
layer is one on which the authors of this draft have not yet reached
consensus. We have unresolved concerns about the efficiency and
efficacy of congestion and flow control schemes implemented across
long and/or highly variable delay environments.
One view of congestion control is as follows: "Congestion control is
concerned with allocating the resources in a network such that the
network can operate at an acceptable performance level when the
demand exceeds or is near the capacity of the network resources.
These resources include bandwidths of links, buffer space (memory),
and processing capacity at intermediate nodes. Congestion occurs
when the demand is greater than the available resources." [RJ90]
For the purposes of this document, we define "flow control" as a
means of assuring that the rate at which a sending node transmits
data to a receiving node does not exceed the maximum rate at which
the receiving node is prepared to receive data from that sender.
(Note that this is a generalized notion of flow control, rather than
one that applies only to end-to-end communication. In particular,
the concept of flow control between the two ends of a single link may
be indispensable in such DTN regions as the "interplanetary
backbone.") We define "congestion control" as a means of assuring
that the aggregate rate at which all traffic sources inject data into
a network does not exceed the maximum aggregate rate at which the
Cerf, et al. Expires February 2003 [Page 37]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
network can deliver data to destination nodes. If flow control is
propagated backward from destination nodes to routers and eventually
back to traffic sources, then flow control can be at least a partial
solution to the problem of congestion as well.
DTN flow control decisions must be made within the bundling layer
itself based on information about resources available within the
bundle node. However, the bundle layer *might* be able to delegate
the implementation of those decisions to the underlying Transport
protocol, as follows.
We have not yet considered multipoint communication at or below the
bundle layer, so each individual flow control problem involves just
two nodes. Since inter-region traffic must pass through inter-region
gateways, any two nodes between which flow control is an issue must
inhabit a common DTN region and be using a common Transport protocol
below the bundle layer (otherwise they could not be communicating and
there would be no flow to control). Therefore, it may be possible to
accomplish DTN flow control by invoking whatever flow control
mechanisms are already provided within the region.
Alternatively, a new, supplementary flow control protocol could be
developed at the bundling layer; this would have the advantage of
reducing DTN's reliance on capabilities provided by the underlying
protocols. At this time it's still unclear which approach is
preferable.
In either case, the problem of flow control between nodes separated
by very large signal propagation times remains to be solved: TCP's
flow control and congestion control facilities could be leveraged
within regions characterized by small round-trip times, but at this
time no comparable protocol exists for very long delay paths, such as
the "interplanetary backbone". [The Long-haul Transmission Protocol
referenced below has yet to be developed.]
It remains as an exercise for us to demonstrate that a hop-by-hop
flow control scheme in long and/or highly variable delay environments
can effect end-to-end congestion control in a fair and efficient
manner.
9 State Management Considerations
An important aspect of any networking architecture is its management
of state information. This section describes the state information
that is managed at the bundle layer and discusses the conditions
under which that state information is established and how it is
removed.
Cerf, et al. Expires February 2003 [Page 38]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
9.1 Application Interface State
Although it is implementation-specific, it is very likely that
application-programming interfaces (APIs) to the bundle layer will be
stateful. In long/variable delay environments, an asynchronous
interface seems to be the only sensible approach, and such interfaces
involve registration actions that create state information. This
information is typically created by explicit request of the
application, and is removed by a separate explicit request (or,
possibly, upon termination of the application).
Since operation in a DTN environment means that delays might be quite
long, it is reasonable to expect that an application instance might
terminate (voluntarily or not) between the time that a bundle is sent
and the time its response is received. The application interface to
the bundle layer will, in most cases, provide a mechanism to
reinstantiate applications that have terminated, assuming the
application has been written to take advantage of such a service.
When a DTN node that is the destination of a bundle receives it, the
bundle layer attempts to deliver it to the destination application
instance indicated in the destination tuple. If the application
instance is unavailable for immediate delivery (for example, it runs
as a result of a "cron" job), then state is created pending delivery
of the bundle. That state is removed on successful delivery of the
bundle to the application instance or on expiration of the time to
live in the bundle. Note that if the return receipt delivery option
is enabled for a bundle, that return receipt is not generated until
the bundle has been successfully delivered to the destination
application.
Note that in some DTN environments, delays might REALLY be long, and
bundle layer operations may be required to operate across system
reboots, host changes, or possibly even operating system upgrades.
To facilitate this, some bundle layer implementations will employ
various forms of persistent storage for their state information, and
will avoid operating system attempts to "clean up" state information.
9.2 Bundle retransmission state
State information to support bundle retransmission is created at a
DTN node as a result of either of two events: First, state is created
when a bundle is received from a DTN user application requesting
custodial transmission (assuming that DTN node is capable of
supporting custodial operation). Second, state is created when a
bundle is received from a peer DTN node and the receiving node
intends to assume custody of the bundle.
Cerf, et al. Expires February 2003 [Page 39]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
In the first case, recall that applications may indicate a bundle
time to live that is greater than that which is allowed in the
network. (For example, a DTN application may legitimately feel that
its data has value for a century or more, while the DTN that is
carrying it doesn't want a particular bundle to potentially be in
transit for that long.) The bundle layer may reduce the time to live
value in the bundle itself to a value that is consistent with network
operation rules. The bundle layer at the source *should* retain the
information about the application's view of the bundle's valuable
lifetime. In the event that a bundle *transmission* times out before
custody is assumed, the bundle layer *may* restart the bundle
transmission procedure. This highlights the need to be careful to
ensure that bundle acknowledgments have a high probability of
receipt, to avoid spurious retransmissions of bundles. At the
source, the state associated with a custodially transmitted bundle is
removed when a custody transfer acknowledgment is received, or when a
period of time subject to local policy has elapsed.
For the second case (that of an in-network custodian), state
information is removed when a custodial acknowledgment for the bundle
is received or when the bundle's time to live has been exceeded.
9.3 Bundle routing state
Routing tables, whether statically configured or maintained by
routing protocols, introduce state information that must be managed.
If the general approach to maintaining routing information is as
described in section 6.4 (that is, maintaining link state information
for next-hop neighbors, and distance vector information for points
beyond), then we can make some basic statements about the amount of
state information that must be maintained. For persistent links, in
the worst case the volume of routing information for inter-region
routing scales by (n*R), where n is the number of next hop neighbors
and R is the number of regions in the DTN. For intermittent links,
the volume of information scales by (c*n*R), where c is the number of
contact periods maintained for planning purposes. These values are
worst-case. Pruning and aggregation mechanisms can be used reduce
the volume of information if necessary. In particular, it is likely
that the intermittent links may be reduced to order (n*R) by simply
assuming that the distance vector information is constant for all
contacts. (There probably will not be enough useful information to
do otherwise.) We have not yet seriously considered the routing
metrics that we will maintain, so we do not have an estimate for the
size of each routing entry. This remains work to be done.
Additionally, we have not given significant consideration to intra-
region routing, which will likely have different scaling properties.
Intra-region routing information will be affected by the number of
DTN gateways that are members of the region and by the routing
Cerf, et al. Expires February 2003 [Page 40]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
approach used within the region. (By routing approach we mean
broadly either proactive, meaning that routes to all possible
destinations are maintained, or reactive, meaning that routes are
discovered only when necessary and discarded after some period of
time.) This also directly affects how and when routing state
information is removed from a DTN forwarding node.
9.4 Transmission queue state
Transmission queues are maintained within DTN nodes to manage bundles
awaiting transmission. Although implementation dependent, this state
information will likely include a list of next hop destinations, and
for intermittently connected next-hop destinations, a sublist of
upcoming contacts. These lists will likely contain lists of bundles
for transmission on those contacts. When a new contact is scheduled
or predicted with an intermittently-connected next-hop neighbor, a
new list is created and made available for population with bundles.
Bundles are removed from these lists upon successful transmission.
If a contact ends with bundles remaining on the list to be
transmitted, those bundles are allocated to subsequent contact lists
and the list for the completed contact is removed.
9.5 Receive queue state
Incoming bundles are received from lower layer entities and either
placed on a transmission queue (for bundles to be forwarded) or
placed in a receive queue (for local delivery). Receive queues are
maintained to support demultiplexing of incoming bundles. Again,
although implementation-specific, receive queues are likely
associated with particular local application entities. A bundle is
removed from its receive queue when it has been successfully
delivered to its destination application entity. The receive queue
itself is removed when an application explicitly requests that its
information be purged from the bundle layer (as part of an un-
registration action), or when the application entity has terminated
without leaving instructions for the disposition of remaining
bundles.
9.6 Network management state
We acknowledge that network management information is likely to be a
necessary part of the operation of delay tolerant networks. We have
not, at this point, done a meaningful amount of work in identifying
the network management requirements for DTNs or in defining the state
to support such management. This remains an item for future work.
Cerf, et al. Expires February 2003 [Page 41]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
10 Convergence Layer Considerations for Use of Underlying Protocols
The DTN Architecture provides for the existence of a convergence
layer between the bundle layer and underlying transport protocols.
The convergence layer manages the protocol-specific details of
interfacing with a variety of underlying transport services and
presents a consistent interface to the bundle layer. The convergence
layer also allows additional capability to be inserted as necessary
to augment the services of the underlying transport protocols.
The convergence layer provides an abstraction to the bundle layer in
which bundles are delivered atomically. Partial bundle delivery,
especially at the end of a contact, is accommodated within the
convergence layer. Additionally, the convergence layer may provide
performance enhancement services such as the ability to resume a
partially-received bundle on a subsequent contact with the same next-
hop neighbor (versus starting the bundle transmission over).
11 Bundle Header Information
The bundle layer must carry some information end-to-end. This
section documents our current thinking on the information that must
be carried end-to-end, and notes which of those data elements may be
supplied by the application using the bundle service.
* Version Identifier: this is small integer that indicates which
version of the bundle protocol is being invoked.
* Destination entity ID: this is the identifier of the destination
bundle application instance, and is a tuple, as described above.
It is supplied by the local application using the bundle
service.
* Source entity ID: this is the identifier of the source bundle
application instance, and is a tuple. It is supplied by the
local bundle service, since a particular host may have multiple
names and one may be chosen based on routing decisions or other
criteria opaque to the application (as in multihomed hosts).
The source entity ID may be returned to the application to
support return receipt processing.
* Reply-to entity ID (optional): a source may anticipate not being
able to accept replies, and may use the reply-to entity ID to
specify the destination for return receipts and delivery
records.
* Current custodian ID (optional): the bundle protocol supports
store-and-forward operation in which the custody of a bundle
(that is, the responsibility for ensuring reliable delivery) may
transfer from one DTN node to another as the bundle progresses
through the DTN. There is not a requirement for *each* DTN node
encountered to assume custody of a bundle. As a result, it is
necessary to identify the upstream node that currently has
Cerf, et al. Expires February 2003 [Page 42]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
custody of the bundle, in order to acknowledge correct receipt
of the bundle and to accept custody of the bundle.
* Class of service flags:
. an indication that custody transfer is requested,
. an indication that a return receipt is requested,
. an indication that a delivery record for custody transfers is
requested,
. an indication that a delivery record for all forwarding
operations is requested,
. an indication of the priority of the bundle (bulk, normal, or
expedited), and
. an indication of whether authentication information is present
and its type.
* Send timestamp: the time that a bundle was presented by the
sending application to the bundle layer for transmission.
* Time to live: an offset, in seconds, from the send timestamp
that indicates when the bundle shall be purged from the DTN.
* Authentication information (optional): access control
information to prove that this bundle should be forwarded in the
network.
* User data: this is intended to be all of the data that the
remote entity requires to perform whatever operation is
requested. Since the environmental characteristics of a DTN
tend to make interactivity difficult, the notion is that all of
the information that is required to perform a particular
"transaction" would be provided in a single bundle or
unidirectional sequence of bundles.
Some bundles cause a response to be generated by the bundle layer.
Bundle layer responses are sent as bundles with the user data portion
replaced by a Status Report, consisting of the following information:
* Source entity ID of the subject bundle: this field partially
identifies the bundle that is the subject of the Status Report.
* Send timestamp of the subject bundle: this field completes the
identification of the bundle that is the subject of the Status
Report.
* Status flags, indicating whether or not:
. The subject bundle was received correctly by the source of the
Status Report
. The source of the Status Report has taken custody of the
subject bundle
. The source of the Status Report has forwarded the subject
bundle
. The Status Report contains the time at which the source of the
Status Report received the subject bundle
. The Status Report contains the time at which the source of the
Status Report forwarded the subject bundle
.
Cerf, et al. Expires February 2003 [Page 43]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
* Time of Receipt (optional): the time at which the sender of the
Status Report received the bundle.
* Time of Forward (optional): the time at which the sender of the
Status Report forwarded the bundle.
12 An Example Bundle Transfer
We provide the following example of an end-to-end bundle transfer
across a Delay-Tolerant Network. This particular example is based on
the Interplanetary Internet: A host on Earth sends a bundle to a
destination on Mars. Figure 2 illustrates the network that is the
subject of this example ¡ the Interplanetary Internet in this example
has five distinct regions interconnected by four DTN Gateways. This
example highlights the actions taken by the bundle layer and the
interactions of the bundle layer with applications and with
underlying transport protocols.
12.1 Rules for forming tuples in the example network
As noted in section 4.5, application instance ID tuples consist of
two parts: a region ID, an entity ID. Section 5.2 requires that each
DTN region have an entity identifier space shared by all DTN nodes
within that region. Section 5.4 requires that each DTN region have a
unique identifier that is known (or discernable) by all regions in
the DTN. This particular delay-tolerant network is the
Interplanetary Internet. In this section we will establish the
naming rules that permit interoperation within this network. Note
that this is only an example, and that not all DTNs must use this
particular region identifier space (subject to the requirements of
section 4.5).
12.1.1 Region identification
All regions in *this* DTN (the example Interplanetary Internet) must
share a region identifier space. Per section 4.5, the DTN region
*name* space is hierarchical, dot-separated text, structured such
that left to right is leaf-to-root in the tree. The *identifier*
space may be exactly this name space, or a DTN may define a
translation from the name to a particular identifier, in the same way
that names in the DNS may be translated to Internet addresses. For
this example, we will use the *names* as the *identifiers*.
The DTN region name space is shown in Figure 1:
Cerf, et al. Expires February 2003 [Page 44]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
Root .
|
The Interplanetary Internet int
|
The Solar System sol
+-----+-----+-+---+-----+
| | | | |
Region jupiter mars earth venus ipn
Figure 1. An Example Interplanetary Internet Region Name Space
12.1.2 Intra-region identification
For simplicity in this example, we will assume that all regions use a
well-known TCP port in combination with DNS host names as the portion
of their entity identifier that locates the application providing the
bundle service. The bundle layer application resides above this
well-known port (which we arbitrarily choose to be TCP port 6769, a
currently unassigned port in the registered port space). The
interplanetary backbone region, labeled "ipn" in Figure 1, will
certainly NOT use TCP as its underlying transport protocol. For the
sake of simplicity in this example, let us assume that the protocol
used within the interplanetary backbone region uses the same entity
identifier space (IP addresses and port numbers) that the remainder
of the network uses.
The mechanism used to demultiplex bundles received by a bundle node
is entity-specific, and for the sake of simplicity, we will assume
that this is a process ID that has been incorporated into the entity
ID. [Note: this is a simple, but quite limiting decision, as it has
implications on process reinstantiation and process
portability/migration from one entity to the next. We choose it only
for simplicity.]
12.2 Example Network Topology at the Region Level
It is important to have some understanding of the routing that is
required at the DTN Gateways, so it is important to understand the
topology of the network at the region level. Unlike typical
terrestrial communications, the Interplanetary Internet's (IPN's)
*long-haul* communication links are directional, mobile, and highly
scheduled. This is important, because directionality combined with
mobility means that a transmitter and receiver must track each other
in order to establish and maintain a communication link. In the IPN,
much of the mobility is due to orbital mechanics and is therefore
relatively predictable. However, this means that nodes that we would
normally consider to be fixed, such as antennas on the surface of the
Earth, are actually highly mobile as a result of the Earth's rotation
and its revolution around the Sun. (In this example, we confine
Cerf, et al. Expires February 2003 [Page 45]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
ourselves to our local solar system, and do not consider the motion
of our sun relative to celestial bodies outside our solar system.)
We can describe the predictable aspects of node mobility with an
ephemeris, a table of the positions of celestial bodies at specified
intervals of time. Both a directional sender and receiver must each
know the other's position in order to establish a link between the
pair.
+-------------------------+
| Earth's Internet |
| DTN Region: earth.sol |
| +---+ |
+-----------/ /|--------+
+---+ |
|G/W| +
+----------| 1 |/---------+
| +---+ |
| The "Backbone" |
| DTN Region: ipn.sol |
+---+ +---+ +---+
/ /|------/ /|-------/ /|
+---+ | +---+ | +---+ |
|G/W| + |G/W| + |G/W| +
+----------------| 3 |/ +---| 4 |/-----+ | 2 |/-------------------+
| +---+ | +---+ | +---+ |
| Venus's Internet | | Jupiter's | | Mars's Internet |
| DTN Region | | Internet | | DTN region: |
| venus.sol | | DTN Region | | mars.sol |
+------------------+ | jupiter.sol | +----------------------+
+--------------+
Figure 2. An Interplanetary Internet of Five IPN Regions
In addition, these communication resources are highly scheduled. It
is not sufficient for a receiver to point at a prospective target and
just wait ¡ for example, a terrestrial node will typically have to
point at several targets sequentially, and an interplanetary node
will typically not have enough power to just wait for incoming
messages. Rather, a schedule of communication *opportunities* must
be calculated and then refined with planned communication *contacts*.
A communication opportunity establishes that the endpoints could
establish a link if they were pointing at each other at the proper
times. We refer to a planned communication contact as an agreement
(although not irrevocable) between the two parties to establish
contact and communicate for a defined period of time.
Cerf, et al. Expires February 2003 [Page 46]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
The protocols for establishing the schedule of communication contacts
between all pairs of possible communicants will evolve -- from
something primarily done manually to something more automated as the
real Interplanetary Internet grows.
The scheduled nature of connectivity in the Interplanetary Internet,
particularly across the deep-space links, means that at the time of a
bundle's arrival at a DTN Gateway, some or all of the possible
outbound routes may be "down." The gateway must store the bundle
until the appropriate link is available and then transmit the bundle
over that link. One of the fundamental differences between the
Interplanetary Internet and the terrestrial Internet is this inherent
use of store-and-forward mechanisms in routing bundles. With that
in mind, let us consider the routing decisions made at some of the
DTN Gateways in Figure 3.
12.3 DTN Gateway routing
Bundle routing at a DTN gateway will typically have to deal with a
mix of continuously available links and intermittently available
links. Routing across a continuously available link is a relatively
straightforward activity. Routing in the presence of intermittently
available links can be significantly more complex, as the gateway
will need to decide not only the next hop destination but also the
time at which to send the bundle. Conditions elsewhere in the
network may make it more desirable to send a bundle to a next-hop
destination that is not yet accessible, even when an alternative
route is currently available. This decision process is discussed in
section 6.3.
The intermittent links in this example provide connectivity among the
DTN Gateways that are part of the ipn.sol.int region. The contacts
among gateways in this region are scheduled. As is currently the
case, Gateway 1 (the Earth gateway) has scheduled contacts with each
of the other DTN gateways in the ipn.sol.int region (the "backbone"),
but there is no direct contact among any of the DTN gateways on
Venus, Jupiter, or Mars. Recall that this communication uses
directional antennas, so it is generally not possible to communicate
with two different entities at once unless they are in the same field
of view. Power availability on the remote planets is an issue, so
transmitters and receivers are turned on only during a contact.
Further, the speed of light delays are nontrivial, skewing transmit
and receive times between remote gateways. In Table 1, a schedule of
contacts is presented that will be used in the example. All times
are referenced to Gateway 1. The information in this table is
completely hypothetical, and the speed of light delays are plausible,
but completely back-of-the-envelope. One-way light times between
Gateway 1 and Gateways 3, 4, and 2 are 4 minutes, 32 minutes, and 10
Cerf, et al. Expires February 2003 [Page 47]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
minutes, respectively. Those details are perhaps interesting but not
the point of the example.
Note that any bundles sent by GW3 after 1156 GW1 time cannot be
acknowledged before the next contact, because the bundle will arrive
at GW1 after the end of GW1's transmission time (1200). Since the
next contact between GW1 and GW3 might be the subsequent Monday, the
acknowledgement delay might be VERY long. Bundles sent by GW4 after
1358 cannot be acknowledged during the current contact, because they
will not be received before GW1's transmission time ends at 1430.
Similarly, bundles sent by GW2 after 1150 cannot be acknowledged
during the contact.
Table 1. Contact schedule for Gateway 1.
+---------+------------+-----------+-------------+------------------+
| Day |Destination | GW1 Send | GW1 Receive | Destination Send |
| | | Time | Time | /Receive Time |
+---------+------------+-----------+-------------+------------------+
| Monday | GW3 | 0900-1200 | 0908-1208 | 0904-1204 |
+---------+------------+-----------+-------------+------------------+
| Tuesday | GW4 | 1300-1430 | 1404-1534 | 1332-1502 |
+---------+------------+-----------+-------------+------------------+
|Wednesday| GW2 | 1000-1200 | 1020-1220 | 1010-1210 |
+---------+------------+-----------+-------------+------------------+
DTN Gateways 2, 3, and 4 have entries in their contact schedules for
the contacts with Gateway 1.
12.4 Systems participating in example bundle data transfer
Figure 3 is a revision of Figure 2 that highlights those portions of
the Interplanetary Internet that participate directly in the bundle
transfer example. Also shown in Figure 3 are the source and
destination for the transfer and the Domain Name System equivalents
in the terrestrial Internet (DNS 1), in the IPN "Backbone" (DNS 2),
and in the Mars Internet (DNS 3). This figure will serve as the
basis for the bundle data transfer example.
Table 2 provides the full host names of each of the primary elements
in Figure 4. Recall that in this example all bundle layer
Cerf, et al. Expires February 2003 [Page 48]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
+---+
/ /|
+------------------------+---+ |
+---+ Earth's Internet |DNS| +
/ /|IPN Region: earth.sol| 1 |/
+---+ | +---+ +---+
|SRC| +---------/ /|--------+
| |/ +---+ |
+---+ |G/W| +
+----------| 1 |/---------+
| +---+ |
| The "Backbone" |
| IPN Region: ipn.sol |
+---+ +---+ +---+
/ /|-------------------/ /| / /|
+---+ | +---+ | +---+ |
|DNS| + |G/W| + |DST| +
| 2 |/ | 2 |/-------------| |/+
+---+ +---+ +---+ |
| Mars's Internet |
+---+ IPN region: |
/ /| mars.sol |
+---+ |---------------------+
|DNS| +
| 3 |/
+---+
Figure 1. Interplanetary Internet showing principal systems.
Table 1. Host name tuples (entity ID demultiplexing info omitted).
+------------+------------------+---------------------------+
| Host | IPN Regions | Host name tuples |
+------------+------------------+---------------------------+
| SRC | earth.sol |{src.jpl.nasa.gov:6769, |
| | | earth.sol} |
+------------+------------------+---------------------------+
| IPN G/W1 | earth.sol |{ipngw1.jpl.nasa.gov:6769, |
| | | earth.sol} |
| | ipn.sol |{ipngw1.jpl.nasa.gov:6769, |
| | | ipn.sol} |
+------------+------------------+---------------------------+
| IPN G/W2 | ipn.sol |{ipngw2.nasa.mars.org:6769,|
| | | ipn.sol} |
| | mars.sol |{ipngw2.nasa.mars.org:6769,|
| | | mars.sol} |
+------------+------------------+---------------------------+
| DST | mars.sol |{dst.jpl.nasa.gov:6769, |
| | | mars.sol} |
+------------+------------------+---------------------------+
Cerf, et al. Expires February 2003 [Page 49]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
applications reside on TCP port 6769. The portion of the entity
identifier used to demultiplex to the application using the bundle
service has been omitted until the applications are discussed in
section 12.5.1.
12.5 End-to-end Transfer
12.5.1 Step 0: Application instance registration
Before the beginning of this example, all of the bundle nodes in the
network exchange their signed key material and credentials with next
hop neighbors. These materials are cached for subsequent use. The
applications at the source and destination register with their local
bundle layers, providing similar key material and credentials to the
bundle layer, and receiving in return their application instance
identifiers. The destination has registered its IPN-standardized
well-known demultiplexing id of "8," which becomes part of the entity
ID used to refer to this particular application. The source has
registered a demultiplexing identifier "0x1763421A" (a hexadecimal
number) that (arbitrarily) corresponds to its process ID.
12.5.2 Step 1: Bundle creation and first-hop transmission
The application on the source host in Figure 3 has data that it
wishes to send to the destination on Mars. The exact content of this
data is opaque to the bundle transfer, but assume that it contains
all of the information necessary to accomplish some desired function.
That is, assume that application-specific instruction for storage,
handling, error processing, and disposal accompanies whatever data
object is to be operated upon. The application invokes the bundle
layer, supplying it the information shown in Table 2.
The bundle agent verifies the signature, then creates a bundle and
stores it in persistent storage (on disk or other non-volatile
memory). There are some fields of the bundle header that the bundle
agent must supply: the Sending Timestamp, the Source Host name tuple,
the Custodian name tuple, and the time to live. (The application may
state a time after which the data are obsolete, but the actual time-
to-live field in the bundle header uses the application's data in
combination with network restrictions on time-to-live to initialize
this field properly.) The bundle layer consults routing tables notes
that its next-hop destination to reach the mars.ipn.sol region within
the earth.ipn.sol region is ipngw1.jpl.nasa.gov:6769. (Since a host
may reside in multiple IPN Regions, the source host name tuple is a
function of the outbound route selected. The bundle layer uses this
information to complete the Source Host and Custodian name tuples
prior to transmission.) Having verified the application's signature,
it incorporates this into the authentication information of the
Cerf, et al. Expires February 2003 [Page 50]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
bundle header and appends its own credentials, as described in
section 7.2. It further notes that it has a continuous connection to
that gateway, and transmits the bundle to it, retaining a copy for
custodial storage. The bundle layer starts a retransmission timer
for the bundle and awaits a custodial transfer acknowledgment.
Table 2. Information supplied by source application.
+-------------+---------------------+------------------------------+
| Item | Value | Description |
+-------------+---------------------+------------------------------+
| Destination | {mars.sol, | IPN Name tuple of the |
| DTN | dst.jpl.nasa.gov, | destination. Note that appl.|
| tuple | 8} | demuxing ID 8 is supplied. |
+-------------+---------------------+------------------------------+
| Source | 0x1763421A | Value used to identify the |
| application | | appropriate instance of the |
| demultiplex | | source application for |
| identifier | | response processing (becomes |
| | | part of the source entity ID)|
+-------------+---------------------+------------------------------+
| Class of | Custodial delivery, | The services requested from |
| service | normal priority, | the bundle layer. |
| | data obsolete in 36 | |
| | hours. | |
+-------------+---------------------+------------------------------+
| Signature | Opaque | Digital signature of the |
| | | app.-supplied information |
+-------------+---------------------+------------------------------+
| User data | N/A | |
+-------------+---------------------+------------------------------+
12.5.3 Step 2: Bundle processing at first-hop destination
When the IPN Gateway {ipngw1.jpl.nasa.gov, earth.sol} receives the
bundle via TCP, it checks the signature of the previous router and
determines that the bundle has been forwarded by a legitimate source.
Further, since this is the security boundary for the Interplanetary
Internet, it verifies the signature of the sending application,
requesting a copy of the application's public key from a secure
distribution service if it does not already have one cached. Having
verified that the application is authorized to access the deep-space
portion of the Interplanetary Internet, the bundle layer replaces the
signature block of the bundle layer entity with its own, leaving the
application's signature block intact. It stores the received bundle
on persistent storage (disk). The bundle layer consults the contact
schedule and determines that the appropriate next-hop destination is
in the "ipn.sol" IPN Region: {ipngw2.nasa.mars.org, ipn.sol}, and
Cerf, et al. Expires February 2003 [Page 51]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
that the next contact is the following Wednesday. The bundle layer
manifests the bundle on the contact for that Wednesday.
In considering alternative contacts for the bundle, the dispatcher
checks the time-to-live in the bundle, which was 36 hours from the
time of initial submission to the bundle agent at the source, to
ensure that the route selected is consistent with the time-to-live
requirements of the bundle. The bundle transport functionality of
the bundle agent in ipngw1 accepts custody of the bundle, updates
this information in the bundle header, and informs the source that
has done so. The sources bundle agent deletes its custodial copy of
the bundle. When the time arrives for contact with the
ipngw2.nasa.mars.org DTN gateway to be established, the convergence
layer establishes that contact via the Long Haul Transport Protocol
(LTP). When informed that the contact is available, the bundle layer
posts its bundles to the convergence layer for transmission via LTP.
12.5.4 Step 3: Bundle processing at gateway to destination IPN Region
The Mars gateway, {ipngw2.nasa.mars.org, mars.sol}, receives the
bundle from the Earth gateway via LTP. It verifies the signature
block of the Earth gateway, then replaces that signature block with
its own. It stores the bundle in persistent storage and accepts
custody of the bundle, signaling back to the Earth gateway that it
has done so. When the notification of acceptance reaches the Earth
gateway, ipngw1 deletes its custodial copy. The Mars gateway
consults its routing tables to find an outbound contact to forward
the bundle over. It determines that the appropriate next hop is the
destination itself, that the proper transport protocol is TCP, and
that the destination is accessible immediately. The gateway verifies
that the time-to-live has not expired, and forwards the bundle to the
destination.
12.5.5 Step 4: Bundle processing at destination
The bundle layer at the destination entity receives the bundle via
TCP, verifies the signature block of the IPN G/W2, stores it on its
own persistent storage, and accepts custody of the bundle from IPN
G/W2. The bundle agent "awakens" the destination application process
identified by the Destination Application demultiplexing portion of
the entity ID. This may involve creating a new instance of a server
from a daemon process, signaling an idle running process, or
reinstantiating a process that has been suspended with its state
stored on persistent storage. The bundle agent deletes the copy of
the bundle from persistent storage when the application has received
it. The destination application may generate an application-layer
acknowledgment in a new bundle and send it to the source.
Cerf, et al. Expires February 2003 [Page 52]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
12.6 Error Conditions at the Bundle Layer
This section describes the error conditions that may arise at the
bundle layer during bundle creation and transport. When these errors
occur within the sender's DTN region, it may be possible to conduct a
near-real-time dialog to correct them before the bundle is forwarded.
We say 'may be possible' because even if two nodes are in the same
IPN domain, they may not have real-time connectivity. An example of
such a situation would be if a lander were on the opposite side of
the planet from its DTN gateway, and used bundles to communicate with
the gateway through a low altitude orbiter, with the orbiter itself
serving as a bundle agent.
Table 3: Error conditions at the bundle layer.
+-------+---------------------------+------------------------------+
| Error | Description | Places where Error can Occur |
+-------+---------------------------+------------------------------+
| 1* | Unknown destination region| Source Bundle Node |
+-------+---------------------------+------------------------------+
| 2* | Invalid Source App. | Source Bundle Node |
+-------+---------------------------+------------------------------+
| 3* | Bundle Parameter Syntax | Source Bundle Node |
| | Error | |
+-------+---------------------------+------------------------------+
| 4* | Bundle Parameter Semantic | Source Bundle Node |
| | Error | |
+-------+---------------------------+------------------------------+
| 5 | Insufficient buffer space | Any bundle node |
+-------+---------------------------+------------------------------+
| 6 | DNS unreachable | Any bundle node |
+-------+---------------------------+------------------------------+
| 7* | Time exceeded | Any bundle node other than |
| | | the source |
+-------+---------------------------+------------------------------+
| 8* | Source Entity Access | Any bundle node other than |
| | Denied | the source |
+-------+---------------------------+------------------------------+
| 9 * | Invalid Entity ID in | IPN gateway serving |
| | Destination Name | destination DTN region |
+-------+---------------------------+------------------------------+
| 11* | Invalid Destination App. | Destination |
+-------+---------------------------+------------------------------+
| 12* | End-to-end Access Denied | Destination |
+-------+---------------------------+------------------------------+
The errors that can occur at the bundle layer are shown in Table 3.
Error numbers marked with an asterisk (*) are reported back to the
sending application.
Cerf, et al. Expires February 2003 [Page 53]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
* Unknown Destination Region: This error occurs when the source
bundle node is directed to create a bundle destined for an DTN
Region that is not recognized. Note that only the DTN Region part
of the destination name has to be interpretable outside the
destination's DTN Region. In particular, the entity identifier of
the destination name need not be interpretable to the source
(assuming the source and destination are in different DTN
Regions), so it cannot necessarily be checked when the bundle is
created.
* Invalid Source Application: If the source authentication
information fails, the source bundle layer responds with an
Invalid Source Application error.
* Bundle Parameter Syntax Error: The source bundle layer may check
the syntax of some of the bundle-creation parameters (i.e. it may
ensure that the end-to-end and DTN access security certificates
are well-formed, etc.) If a parameter is found to be
syntactically incorrect or obviously and definitely erroneous, the
bundle layer will report a Bundle Parameter Syntax Error back to
the source that includes, at a minimum, the parameter that caused
the error.
* Bundle Parameter Semantic Error: If the source bundle layer can
identify a particular bundle creation parameter as being well-
formed but unserviceable, it will report a Bundle Parameter
Semantic Error to the source application that includes, at a
minimum, the parameter that caused the error.
* Insufficient Buffer Space: If a bundle node does not have
sufficient buffer space to accept a bundle, it drops the bundle
and generates an Insufficient Buffer Space error. Note that a
bundle node may choose to drop lower priority bundles in order to
make room for higher priority ones. This error is not propagated
back to the source.
* DNS Unreachable: If a bundle node needs access to its DNS (or DNS-
equivalent) and cannot obtain information from it, it generates a
DNS Unreachable error. This information is not propagated back to
the source application.
* Time Exceeded: If the time-to-live field (either the source-
provided TTL or the internal bundle TTL) expires, the source is
notified with a Time Exceeded message. These errors are
propagated to the source application.
* Source Entity Access Denied: This error indicates that the source
entity does not have access to a needed resource at a DTN node.
The source might not be authorized to use the node at all, or it
might not have authorization to use a particular interface
required by the bundle. Source Entity Access Denied errors
indicate that the source is not *authorized* to use a particular
resource; other errors (e.g. Insufficient Buffer Space) indicate
that a particular resource is *unavailable* to a bundle. For
example, an entity on the surface of a planet might be authorized
to communicate, using the bundle protocol, with another entity on
Cerf, et al. Expires February 2003 [Page 54]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
the other side of the planet via a low-altitude orbiter that is
also an IPN gateway. The sender might not, however, be authorized
to send bundles across interplanetary space. In this case bundles
sent to the orbiter destined for the other side of the planet
would not cause errors, while any bundles with off-planet
destination addresses would. Source Entity Access Denied errors
are propagated back to the source application.
* Invalid Entity ID in Destination Name: Once a bundle has reached
its destination DTN Region, the Entity ID part of the destination
name can be parsed. If the Entity ID of the destination name is
not valid, the source is notified with an Invalid Administrative
Destination Name error message.
* Invalid Destination Application: If the destination bundle layer
cannot instantiate the destination application (based on the
demultiplexing information the region-specific entity ID of the
bundle), it notifies the source application with an Invalid
Destination Application error message.
* End-to-End Access Denied: If the bundle destination does not
accept the bundle due to an authentication or access-control
error, the source is notified with an End-to-End Access Denied
Message.
13 Summary
This architecture addresses many of the problems of networks that
must operate in extreme environments. It is message based, and uses
as a model of service classes those offered by the postal mail
system. It accommodates many different forms of connectivity,
including scheduled, predicted, and opportunistically connected
links. It introduces a novel approach to end-to-end reliability
across frequently partitioned and unreliable networks. It also
proposes a scheme for securing the network infrastructure against
unauthorized access.
It is our belief that this architecture is applicable to many
different types of extreme environments. In subsequent documents, we
intend to specify profiles of this architecture that address specific
environments in detail. We plan to develop profiles of this
architecture for Interplanetary Internetworking (the genesis of the
architecture), for Sensor Internetworking, for Military Tactical
Internetworking, and for "Snowmobile" Internetworking. In addition
to these profiles, our upcoming tasks include precise specification
of the constituent protocols in concert with their prototyping and
testing. These activities are currently under way.
Cerf, et al. Expires February 2003 [Page 55]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
14 References
[KP97] K. Park, G. Kim, and M. Crovella, "On the Effect of Traffic
Self-Similarity on Network Performance," Proceedings of the 1997 SPIE
International Conference on Performance and Control of Network
Systems.
[BLMR98] J. Byers, M. Luby, M. Mitzenmacher, A. Rege, "A Digital
Fountain Approach to Reliable Distribution of Bulk
Data", SIGCOMM 1998
[ARINC] Presentation by ARINC, see page at http://www.arinc.com
[DMT96] R. Durst, G. Miller, E. Travis, "TCP Extensions for Space
Communications", Wireless Networks 3(5):389-403, 1997.
[MLS00] U. Madhow, T. V. Lakshman, B. Suter, "TCP/IP Performance
with Random Loss and Bidirectional Congestion", IEEE/ACM
Transactions on Networking, 8(5), Oct 2000.
[PILC-ASYM] H. Balakrishnan, V. Padmanabhan, G. Fairhurst, M.
Sooriyabandara, "TCP Performance Implications of Network Path
Asymmetry", IETF draft-ietf-pilc-asym-07, (Work in Progress), Nov
2001.
[M95] J. Mogul, "Fragmentation Considered Harmful", SIGCOMM 1995
[IGE00] C. Intanagonwiwat, R. Govindan, D. Estrin, "Directed
Diffusion: A scalable and robust communication paradigm for
sensor networks", MobiCOM 2000, Aug 2000
[NEWARCH] NewArch Project: Future-Generation Internet Architecture.
http://www.isi.edu/newarch
[META] Wroclawski, John T., "The Metanet", Workshop on Research
Directions for the Next Generation Internet, May 12-14, 1997, Vienna,
VA. http://www.cra.org/Policy/NGI/papers/wroklawWP.
[CK74] V. Cerf, R. Kahn, "A Protocol for Packet Network
Intercommunication", IEEE Trans. on Comm., COM-22(5), May 1974
[DM02] D. Mills, "Timekeeping in the Interplanetary Internet", July
2002, http://www.eecis.udel.edu/~mills/database/brief/ipin/ipin.pdf
[WhynotTCP] "Why not use the Standard Internet Suite for the
Interplanetary Internet?" http://www.ipnsig.org/reports/TCP_IP.pdf
[RJ90] R. Jain, "Congestion Control in Computer Networks: Issues and
Trends," IEEE Network Magazine, May 1990.
Cerf, et al. Expires February 2003 [Page 56]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
15 Security Considerations
Security is an integral concern of the Delay Tolerant Network
Architecture. Section 7 of this document, also titled "Security
considerations" is devoted to examining the issues involved in
securing the DTN architecture and providing basic security services
to DTN applications.
Cerf, et al. Expires February 2003 [Page 57]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
16 Authors' Addresses
Dr. Vinton G. Cerf
MCI WorldCom
22001 Loudoun County Parkway
Building F2, Room 4115, ATTN: Vint Cerf
Ashburn, VA 20147
Telephone +1 (703) 886-1690
FAX +1 (703) 886-0047
Email vcerf@mci.net
Scott C. Burleigh
Jet Propulsion Laboratory
4800 Oak Grove Drive
M/S: 179-206
Pasadena, CA 91109-8099
Telephone +1 (818) 393-3353
FAX +1 (818) 354-1075
Email Scott.Burleigh@jpl.nasa.gov
Robert C. Durst
The MITRE Corporation
1820 Dolley Madison Blvd.
M/S W650
McLean, VA 22102
Telephone +1 (703) 883-7535
FAX +1 (703) 883-7142
Email durst@mitre.org
Dr. Kevin Fall
Intel Research, Berkeley
2150 Shattuck Ave., #1300
Berkeley, CA 94704
Telephone +1 (510) 495-3014
FAX +1 (510) 495-3049
Email kfall@intel-research.net
Adrian J. Hooke
Jet Propulsion Laboratory
4800 Oak Grove Drive
M/S: 303-400
Pasadena, CA 91109-8099
Telephone +1 (818) 354-3063
FAX +1 (818) 393-3575
Email Adrian.Hooke@jpl.nasa.gov
Dr. Keith L. Scott
The MITRE Corporation
1820 Dolley Madison Blvd.
Cerf, et al. Expires February 2003 [Page 58]
Internet Draft draft-irtf-ipnrg-arch-01.txt August 2002
M/S W650
McLean, VA 22102
Telephone +1 (703) 883-6547
FAX +1 (703) 883-7142
Email kscott@mitre.org
Leigh Torgerson
Jet Propulsion Laboratory
4800 Oak Grove Drive
M/S: T1710-
Pasadena, CA 91109-8099
Telephone +1 (818) 393-0695
FAX +1 (818) 354-9068
Email Leigh.Torgerson@jpl.nasa.gov
Howard S. Weiss
SPARTA, Inc.
9861 Broken Land Parkway
Columbia, MD 21046
Telephone +1 (410) 381-9400 x201
FAX +1 (410) 381-5559
Email hsw@sparta.com
Please refer comments to ipn-team@mailman.ipnrg.org
