Internet Engineering Task Force S. Burleigh
Internet Draft IPNGROUP
Intended status: Informational August 30, 2024
Expires: March 2025
Revisiting the Use of the IP Protocol Stack in Deep Space
draft-burleigh-deepspace-ip-assessment-00.txt
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on March 3, 2025.
Copyright Notice
Copyright (c) 2024 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License.
Burleigh Expires March 2025 [Page 1]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
Abstract
This document describes aspects of communication over interplanetary
distances that make the use of the Internet protocol stack in a deep
space network inadvisable.
Table of Contents
1. Introduction...................................................2
2. Conventions used in this document..............................4
3. Deep space communication considerations........................4
3.1. Connections...............................................4
3.2. Retransmission............................................5
3.3. Routing and Forwarding....................................6
3.4. Domain Name System........................................7
3.5. Time to live..............................................7
3.6. Flow control..............................................8
3.7. Congestion control........................................8
3.8. Security..................................................9
3.9. Network management........................................9
3.10. Electronic mail..........................................9
3.11. Clock alignment.........................................10
4. Conclusions...................................................10
5. Security Considerations.......................................11
6. IANA Considerations...........................................11
7. References....................................................11
7.1. Normative References.....................................11
7.2. Informative References...................................12
8. Acknowledgments...............................................12
1. Introduction
Marc Blanchet's article "Beyond Earth: Exploring the Future of Deep
Space Communications", in the May 2024 issue of IEEE Communications
Society Technology News [COMSOC], discusses a topic that grows in
importance as the world's space agencies begin plans for mission
operations at the Moon and beyond. Readers of that article might be
interested in some additional background on the challenges of using
the Internet protocol stack to operate a communication network over
deep space links.
It is well understood that entities exchanging information in deep
space must cope with the lengthy signal propagation latencies
directly imposed by the vast distances separating them, given the
limited speed of light in a vacuum. It is perhaps less well
Burleigh Expires March 2025 [Page 2]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
understood that those distances will indirectly impose additional
latencies that are both far greater and also highly variable.
Obviously there will be no wired infrastructure supporting this
information exchange; all communication will be wireless (either
radio or optical). Transmitted wireless signal strength is
attenuated by the square of the distance between transmitter and
receiver; no broadcast signal on Earth (for example) will be
powerful enough to be decoded by a spacecraft 20 light minutes away.
This means that all deep space communication must be by directed,
rather than broadcast, radiation.
This in turn means that in order for two entities to exchange
information they must be mutually visible and their antennae must be
pointed at each other. Planets rotate, satellites revolve around
the rotating planets, and transmission and reception equipment
powerful enough to enable interplanetary signal exchange is
expensive. In the general case, a given entity will not be able to
communicate continuously with all peer entities with which it must
interoperate.
So connectivity will routinely be interrupted for minutes, hours,
even days, and the durations of these interruptions will not be
computable by statistical analysis of traffic; they will be imposed
by orbital dynamics and by the operating schedules of the entities,
from which they may be computed far in advance of their occurrence.
Importantly, these interruptions are not instances of the link
shutting down for some indeterminate time while repairs are made;
they are readily anticipated increments in forwarding latency.
Connectivity is punctuated, not unstable.
Moreover, while the interval between the transmission of a signal
and reception of that signal is easily computed from knowledge of
the distance between the transmitter and the receiver, not all
message exchange within a deep space network will be exclusively
between the transmitter and receiver of a given signal; as in the
Internet, arrival of a message at a remote destination may require
multiple consecutive receptions and onward transmissions - "hops" -
by relay entities (gateways, routers).
Additionally, punctuated connectivity will frequently make it
impossible for a message received at a given relay point to be
forwarded immediately to the next relay on its end-to-end path
through the network: connectivity to that neighboring node may not
be established until hours or even days later. The relay will be
required to retain the message in storage until that opportunity
Burleigh Expires March 2025 [Page 3]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
arrives, and latency in the delivery of the message will be
increased accordingly.
Due to the nature of deep space communication, then, the end-to-end
latency in delivery of any message may by measured in hours or days,
as may the latency in delivery of any message issued in response to
that message, and the end-to-end latency in delivery of the next
message from the same source to the same destination may be very
different. The very brief, statistically predictable "round-trip"
times on which many Internet protocols rely for operational success
cannot be expected. This paper will discuss some of the
implications of this constraint.
2. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119 [RFC2119].
In this document, these words will appear with that interpretation
only when in ALL CAPS. Lower case uses of these words are not to be
interpreted as carrying RFC-2119 significance.
3. Deep space communication considerations
3.1. Connections
The values of parameters governing data exchange between network
peers must be agreed upon in order for data to flow, and those
values must remain in agreement as recorded in the state of each
peer in order for data flow to be sustained. When that agreement is
established by means of a round-trip message exchange between the
peers, we often term it a "connection".
A connection can be negotiated quickly while nodes are in close
proximity and the round-trip time between them is brief; when this
is the case, the communication governed by the connection can
normally be sustained for a lengthy period following connection
establishment. Connection-based protocols in the Internet succeed
because it is virtually certain that the opportunity for
communication will not lapse until long after the connection has
been established.
However, establishing new connections while the peers are many light
minutes apart and round-trip times are much longer may be so time-
consuming that communication opportunities lapse shortly after - or
even before - agreement is reached, limiting data flow. This effect
Burleigh Expires March 2025 [Page 4]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
could in some cases disable multi-hop end-to-end connections
altogether, but it would be troublesome even in connections between
topological neighbors that are separated by interplanetary
distances.
Such connection difficulties can be avoided if there is no need to
establish connections between peers that are separated by
interplanetary distances. One possible mitigation is to establish
connections between peers while they are co-located (e.g.,
spacecraft that have not yet been launched from Earth) and then
simply retain that connection state indefinitely while the distance
between the peers increases to interplanetary scale.
But in the general case that may not be sufficient. In an
interplanetary network, a given spacecraft might communicate with
any number of laboratories or operations centers on Earth or
(ultimately) on some other planet; adding another such "ground" peer
would require a new connection. A given spacecraft might
additionally communicate with any number of other spacecraft,
including spacecraft that were launched after it was already in
orbit; communicating with each such spacecraft would require a new
connection. Moreover, an existing connection must be re-established
whenever state is lost at one of the peers, as when mission
operations computers reboot or flight computers reboot or are simply
shut down to conserve power.
Further, even when the establishment of transport (e.g., QUIC
[RFC9000]) connections is minimized it may be necessary to establish
additional connections - agreement on additional data exchange
parameter values - at the application layer. Such connections are
by definition end-to-end and would need to be negotiated whenever
application peers begin or restart operation in the network.
3.2. Retransmission
Reliable message exchange in the Internet rests on protocols such as
TCP and QUIC that detect data loss and automatically retransmit lost
data. That retransmission may be triggered by negative
acknowledgments issued by the destination entity, but acknowledgment
messages may also be lost; in that event, retransmission is
triggered by the expiration of a timeout interval at the source
entity shortly after the time at which an end-to-end acknowledgment
is expected. Since message round-trip times over multi-hop paths in
deep space may be not only lengthy but also highly variable,
statistical computation of an accurate retransmission timeout
interval by, e.g., QUIC is in the general case not possible.
Retransmission will either occur too soon, wasting limited
Burleigh Expires March 2025 [Page 5]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
transmission opportunities, or too late, delaying both data arrival
and the release of retransmission buffer resources, and thereby
introducing congestion in the network.
This effect too can be avoided: end-to-end message exchange can be
accomplished over paths constructed by concatenating multiple
consecutive QUIC transmissions, each of which entails only signaling
between two mutually visible network nodes for which the round-trip
time is a function of the latency in direct signal propagation. But
to enable this model a new mechanism is needed that can concatenate
multiple successive QUIC transmissions in a manner that is secure
from end to end. (The Bundle Protocol [RFC9171] is one such
mechanism.)
Beyond unnecessary consumption of network resources, end-to-end
retransmission in an interplanetary internet also unnecessarily
retards data delivery to the destination. For a message on a 3-hop
path from node A to node D via nodes B and C, the corruption of a
single frame in the forward transmission from node C would cause
node D to send a negative acknowledgment back to node A, triggering
retransmission from A. If the hop from A to B or from B to C spans
20 light minutes of space, delivery of the message is delayed by at
least 40 minutes. Since the message was received successfully at C,
network resource utilization and delivery timeliness are both
improved if node C receives the negative acknowledgment and
retransmits the message itself.
3.3. Routing and Forwarding
Delivery of a message in the Internet is reliant on asserting the
address (location within the network topology) of the message's
destination and on providing network routers with sufficient
knowledge of network topology to enable the forwarding of the
message to that address.
The Internet responds dynamically to changes in network topology by
means of routing protocols, which convey information about those
changes to routers. When routing protocol message propagation is
very rapid, as in the Internet, a generally accurate understanding
of network topology can be sustained across the network.
But when it is not, due to lengthy and highly variable message
delivery latency as in a deep space network, not all nodes of the
network will agree on its topology at any moment. Changes in
network topology will result in routing errors, data loss, and poor
network performance.
Burleigh Expires March 2025 [Page 6]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
This effect can be avoided if dynamic routing in the deep space
network is abandoned and, instead, manually provisioned routing
tables are managed in coordination among network administrators.
But the scope of this management activity will grow as the network
grows. For a future deep space network potentially comprising
thousands of nodes (on Earth, in Earth orbit, on other planetary
surfaces, in other planetary orbits, at LaGrange points, etc.), the
canonical Internet routing and forwarding model would not be
satisfactory.
3.4. Domain Name System
Internet applications typically issue messages to named entities
rather than to network addresses, relying on the Domain Name System
(DNS) to resolve entities' names to their IP addresses. DNS name
resolution is performed by querying a DNS server and waiting for a
response, a round-trip communication. When the DNS server is in
close proximity (e.g., on the same planetary surface, as on Earth),
this works well. But lengthy round-trip times would severely retard
the domain name resolution on which many applications rely.
This effect can be addressed by deploying DNS servers to remote
locations in the solar system so that the round-trip time from any
node to the nearest DNS server is small. But this would entail
synchronization of DNS information among all servers in the network
via messages that may take hours or days to reach all destinations.
That synchronization would consume network bandwidth that might
better be used to convey scientific or industrial information, and
meanwhile inconsistencies among DNS servers would likely cause
routing errors.
3.5. Time to live
In the Internet, excessive consumption of network resources
(transmission bandwidth) by any single message is detected when the
message's "hop count" violates a designated limit. At that time the
message can be assumed to be in a routing loop and should be
discarded from the network.
This mechanism is effective because data in the Internet are in
generally continuous motion, residing only momentarily in the memory
buffers of any single forwarding node. A steadily decreasing count
of remaining authorized hops ("time to live", or TTL) is a rough
analog for the passage of time.
In an interplanetary network, however, a message may routinely be
required to wait at a forwarding node until the start of the next
Burleigh Expires March 2025 [Page 7]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
opportunity to transmit it to the next node on its network path.
While waiting it is consuming network resources (buffer space), and
that resource consumption might be excessive in an analogous way:
the next transmission opportunity might never begin, in which case
the message should be discarded from the network even if its TTL has
never been decremented at all.
Mechanisms for detecting expiration of this alternative "time to
live" will be needed in the implementation of the interplanetary
network.
3.6. Flow control
Limiting the rate at which a source inserts data into the network to
the rate at which the destination is able to receive and process it
- flow control - is critical to network performance. If the
destination node's buffers are filled by inbound traffic more
rapidly than they can be emptied by the application, buffers will
overflow; this will cause data loss, resulting in degraded bandwidth
utilization and delayed data delivery.
In the Internet that limit can be inferred from the rate at which
data arrival acknowledgments are received. In an interplanetary
network, however, signal propagation latency may be so great that
the receiver's buffers overflow long before the sender receives
enough acknowledgment information to compute a reduced transmission
rate.
Put simply, flow control in the interplanetary network must be
anticipatory rather than reactive.
3.7. Congestion control
Limiting the rates at which data sources in aggregate insert data
into the network so as to ensure that the network can successfully
convey all data to all destinations - congestion control - is
likewise critical to network performance. In the Internet, a
mismatch between the rates at which data are inserted and removed
from the network results in buffer overflow at relay points. Nodes
detect imminent congestion and signal one another to reduce rates of
application data acceptance, preventing congestion.
As with flow control, signal propagation latency in an
interplanetary network may be so great that congestion signals may
not arrive soon enough to trigger timely reduction in application
data insertion. Buffers will overflow not only at end systems but
also at relay points.
Burleigh Expires March 2025 [Page 8]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
Again, congestion control in the interplanetary network must be
anticipatory rather than reactive.
3.8. Security
Internet security protocols such as IPsec and TLS rely on
authentication and key agreements negotiated by means of handshakes
between end systems. These security agreements are subject to the
same constraints that limit the usability of transport-layer
connections as discussed in 3.2 above. End-to-end security
associations are especially difficult to establish over a deep space
network that includes multiple relay points. An alternative model
of managed security associations is needed.
Point-to-point security associations between topologically adjacent
nodes can be established more readily, but migrating to an
architecture based on achieving end-to-end communication by
concatenating multiple individually secured "hops" would raise an
additional problem. Internet security protocols protect data while
in transit via other protocols (IP, TCP, QUIC), but data may reside
in the memory of a relay node - no longer in transit, thus no longer
secured - for minutes, hours, or days while awaiting onward
transmission. A means of securing application data at rest is
needed. It would be preferable to avoid requiring each application
protocol to perform this function itself.
3.9. Network management
Commonly used Internet network management protocols - SNMP, NETCONF,
and RESTCONF - can be used over interplanetary distances, but care
must be taken. The potentially very long round-trip times between
clients and servers limit the utility of synchronous queries for
node state, such as the SNMP GetRequest/Response cycle; responding
information may be obsolete by the time it arrives.
Asynchronous notifications, such as SNMP's Trap protocol data units,
will be more suitable but will still be of limited value due to
signal propagation latency. In general it will be helpful to rely
increasingly on node autonomy to manage the network, using
asynchronous commands to configure autonomous operation.
3.10. Electronic mail
Email is a wholly delay-tolerant application, perfect for the
interplanetary network. However, at least three aspects of standard
email as it functions within the Internet are somewhat problematic
in deep space deployment.
Burleigh Expires March 2025 [Page 9]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
First, email is conveyed to POP3 and IMAP servers (for local access
by means of mail clients such as Outlook and Gmail) via the Simple
Mail Transfer Protocol (SMTP). Only SMTP servers that have been
adapted to use some transport protocol other than TCP will be
suitable for use in the interplanetary network, as standard TCP is
not useful over interplanetary distances as discussed in [DURST].
Other transport protocols that operate end-to-end as TCP does, such
as QUIC, raise once again the retransmission concerns discussed in
section 3.2 above.
Second, SMTP communication is performed in "sessions", each
comprising zero or more SMTP transactions. Implementations may need
to be modified to tolerate transaction closure latencies on the
order of hours or days.
Third, the headers of modern SMTP messages typically include
references to assets identified by domain names. As the identified
assets may be arbitrarily distant, resolution of those domain names
is subject to the DNS considerations discussed in 3.4 above.
3.11. Clock alignment
Agreement on the current time is important for many network
functions. In the Internet, the Network Time Protocol (NTP) serves
to minimize disagreement on time among network assets. However, NTP
relies on the statistically valid determination of precise timestamp
message transit latencies among NTP peers. Message transit
latencies among interplanetary network nodes will be lengthy; the
length of time required to converge on the current time may be so
great that the consensus time may already be incorrect, due to the
orbital motion of the peers, by the time it is computed.
4. Conclusions
All this said, the Internet protocol suite will certainly play a
vital role in deep space network communications, in much the same
way that Ethernet and WiFi play a vital role in the Internet:
enclaves of local Internet architecture will surely be established
wherever in the Solar System multiple co-located computing platforms
must communicate.
And with minor modifications the Internet protocol suite might
indeed be able to address concurrently all of the concerns
identified here, sustaining communications over deep space links,
for a network of sufficiently limited scope. That is, for a small,
static set of directly interconnected nodes among which no multi-hop
paths are necessary, all connections are permanent, and the efforts
Burleigh Expires March 2025 [Page 10]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
of network administrators suffice to manage all routing, perhaps
only a few additional expectations (e.g., flow control, time to
live) would need to be lowered.
But retaining in a Solar System-wide network the scope of automation
that was developed for the Internet, enabling comparable power and
scalability, will require either major extensions to the current
Internet stack or else a different architecture.
Note that such a "Delay-Tolerant Networking" (DTN) architecture
already exists [RFC4838]. It was first demonstrated in deep space
in 2008, on the EPOXI spacecraft en route to a comet; it has been
operating on-board the International Space Station since 2016; it is
in experimental operation aboard the KPLO (Danuri) spacecraft now in
lunar orbit; it is now in operational use on-board the PACE
spacecraft, conducting Earth observational science since early 2024.
Standardization of the DTN protocols is mature, both within the
Consultative Committee for Space Data Systems and within the
Internet Engineering Task Force (most notably RFCs 9171 and 9172).
Multiple open-source implementations of the protocols are widely
available.
Marc Blanchet's article has done the network communications
community a great service by providing a historical overview of deep
space communications research. Some of the conclusions reached in
the article merit clarification, but this is a conversation from
which we will all benefit.
5. Security Considerations
Security considerations raised in this paper are discussed in
section 3.8.
6. IANA Considerations
No IANA action is proposed in this paper.
7. References
7.1. Normative References
Not applicable.
Burleigh Expires March 2025 [Page 11]
Internet-Draft Bundle-in-Bundle Encapsulation August 2024
7.2. Informative References
[COMSOC] https://www.comsoc.org/publications/ctn/beyond-earth-
exploring-future-deep-space-communications, Blanchet, M., "Beyond
Earth: Exploring the Future of Deep Space Communications", May 2024.
[DURST] https://www.researchgate.net/publication/267721105 Durst,
R., P. Feighery, and K. Scott, "Why not use the Standard Internet
Suite for Interplanetary Internet?", 2000.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", RFC 2119, March 1997.
[RFC4838] Cerf, V., et al, "Delay-Tolerant Networking Architecture",
RFC 4838, April 2007.
[RFC9000] Iyengar, J., Ed. and Thomson, M., Ed., "QUIC: A UDP-Based
Multiplexed and Secure Transport", RFC 9000, May 2021.
[RFC9171] Burleigh, S., E. Birrane, and K. Fall, "Bundle Protocol
Version 7", RFC 9171, January 2022.
Acknowledgments
This document was prepared using 2-Word-v2.0.template.dot.
Authors' Address
Scott Burleigh
IPNGROUP
1435 Woodhurst Blvd.
McLean, VA 22102
United States of America
Email: sburleig.sb@gmail.com
Burleigh Expires March 2025 [Page 12]