Revisiting the Use of the IP Protocol Stack in Deep Space
draft-burleigh-deepspace-ip-assessment-00

Internet Engineering Task Force                             S. Burleigh
Internet Draft                                                 IPNGROUP
Intended status: Informational                          August 30, 2024
Expires: March 2025

         Revisiting the Use of the IP Protocol Stack in Deep Space
               draft-burleigh-deepspace-ip-assessment-00.txt

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other documents
   at any time.  It is inappropriate to use Internet-Drafts as
   reference material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   This Internet-Draft will expire on March 3, 2025.

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with
   respect to this document.  Code Components extracted from this
   document must include Simplified BSD License text as described in
   Section 4.e of the Trust Legal Provisions and are provided without
   warranty as described in the Simplified BSD License.

Burleigh                  Expires March 2025                   [Page 1]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

Abstract

   This document describes aspects of communication over interplanetary
   distances that make the use of the Internet protocol stack in a deep
   space network inadvisable.

Table of Contents

   1. Introduction...................................................2
   2. Conventions used in this document..............................4
   3. Deep space communication considerations........................4
      3.1. Connections...............................................4
      3.2. Retransmission............................................5
      3.3. Routing and Forwarding....................................6
      3.4. Domain Name System........................................7
      3.5. Time to live..............................................7
      3.6. Flow control..............................................8
      3.7. Congestion control........................................8
      3.8. Security..................................................9
      3.9. Network management........................................9
      3.10. Electronic mail..........................................9
      3.11. Clock alignment.........................................10
   4. Conclusions...................................................10
   5. Security Considerations.......................................11
   6. IANA Considerations...........................................11
   7. References....................................................11
      7.1. Normative References.....................................11
      7.2. Informative References...................................12
   8. Acknowledgments...............................................12

1. Introduction

   Marc Blanchet's article "Beyond Earth: Exploring the Future of Deep
   Space Communications", in the May 2024 issue of IEEE Communications
   Society Technology News [COMSOC], discusses a topic that grows in
   importance as the world's space agencies begin plans for mission
   operations at the Moon and beyond.  Readers of that article might be
   interested in some additional background on the challenges of using
   the Internet protocol stack to operate a communication network over
   deep space links.

   It is well understood that entities exchanging information in deep
   space must cope with the lengthy signal propagation latencies
   directly imposed by the vast distances separating them, given the
   limited speed of light in a vacuum.  It is perhaps less well

Burleigh                  Expires March 2025                   [Page 2]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

   understood that those distances will indirectly impose additional
   latencies that are both far greater and also highly variable.

   Obviously there will be no wired infrastructure supporting this
   information exchange; all communication will be wireless (either
   radio or optical).  Transmitted wireless signal strength is
   attenuated by the square of the distance between transmitter and
   receiver; no broadcast signal on Earth (for example) will be
   powerful enough to be decoded by a spacecraft 20 light minutes away.
   This means that all deep space communication must be by directed,
   rather than broadcast, radiation.

   This in turn means that in order for two entities to exchange
   information they must be mutually visible and their antennae must be
   pointed at each other.  Planets rotate, satellites revolve around
   the rotating planets, and transmission and reception equipment
   powerful enough to enable interplanetary signal exchange is
   expensive.  In the general case, a given entity will not be able to
   communicate continuously with all peer entities with which it must
   interoperate.

   So connectivity will routinely be interrupted for minutes, hours,
   even days, and the durations of these interruptions will not be
   computable by statistical analysis of traffic; they will be imposed
   by orbital dynamics and by the operating schedules of the entities,
   from which they may be computed far in advance of their occurrence.
   Importantly, these interruptions are not instances of the link
   shutting down for some indeterminate time while repairs are made;
   they are readily anticipated increments in forwarding latency.
   Connectivity is punctuated, not unstable.

   Moreover, while the interval between the transmission of a signal
   and reception of that signal is easily computed from knowledge of
   the distance between the transmitter and the receiver, not all
   message exchange within a deep space network will be exclusively
   between the transmitter and receiver of a given signal; as in the
   Internet, arrival of a message at a remote destination may require
   multiple consecutive receptions and onward transmissions - "hops" -
   by relay entities (gateways, routers).

   Additionally, punctuated connectivity will frequently make it
   impossible for a message received at a given relay point to be
   forwarded immediately to the next relay on its end-to-end path
   through the network: connectivity to that neighboring node may not
   be established until hours or even days later.  The relay will be
   required to retain the message in storage until that opportunity

Burleigh                  Expires March 2025                   [Page 3]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

   arrives, and latency in the delivery of the message will be
   increased accordingly.

   Due to the nature of deep space communication, then, the end-to-end
   latency in delivery of any message may by measured in hours or days,
   as may the latency in delivery of any message issued in response to
   that message, and the end-to-end latency in delivery of the next
   message from the same source to the same destination may be very
   different.  The very brief, statistically predictable "round-trip"
   times on which many Internet protocols rely for operational success
   cannot be expected.  This paper will discuss some of the
   implications of this constraint.

2. Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC-2119 [RFC2119].

   In this document, these words will appear with that interpretation
   only when in ALL CAPS. Lower case uses of these words are not to be
   interpreted as carrying RFC-2119 significance.

3. Deep space communication considerations

3.1. Connections

   The values of parameters governing data exchange between network
   peers must be agreed upon in order for data to flow, and those
   values must remain in agreement as recorded in the state of each
   peer in order for data flow to be sustained.  When that agreement is
   established by means of a round-trip message exchange between the
   peers, we often term it a "connection".

   A connection can be negotiated quickly while nodes are in close
   proximity and the round-trip time between them is brief; when this
   is the case, the communication governed by the connection can
   normally be sustained for a lengthy period following connection
   establishment.  Connection-based protocols in the Internet succeed
   because it is virtually certain that the opportunity for
   communication will not lapse until long after the connection has
   been established.

   However, establishing new connections while the peers are many light
   minutes apart and round-trip times are much longer may be so time-
   consuming that communication opportunities lapse shortly after - or
   even before - agreement is reached, limiting data flow.  This effect

Burleigh                  Expires March 2025                   [Page 4]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

   could in some cases disable multi-hop end-to-end connections
   altogether, but it would be troublesome even in connections between
   topological neighbors that are separated by interplanetary
   distances.

   Such connection difficulties can be avoided if there is no need to
   establish connections between peers that are separated by
   interplanetary distances.  One possible mitigation is to establish
   connections between peers while they are co-located (e.g.,
   spacecraft that have not yet been launched from Earth) and then
   simply retain that connection state indefinitely while the distance
   between the peers increases to interplanetary scale.

   But in the general case that may not be sufficient.  In an
   interplanetary network, a given spacecraft might communicate with
   any number of laboratories or operations centers on Earth or
   (ultimately) on some other planet; adding another such "ground" peer
   would require a new connection.  A given spacecraft might
   additionally communicate with any number of other spacecraft,
   including spacecraft that were launched after it was already in
   orbit; communicating with each such spacecraft would require a new
   connection.  Moreover, an existing connection must be re-established
   whenever state is lost at one of the peers, as when mission
   operations computers reboot or flight computers reboot or are simply
   shut down to conserve power.

   Further, even when the establishment of transport (e.g., QUIC
   [RFC9000]) connections is minimized it may be necessary to establish
   additional connections - agreement on additional data exchange
   parameter values - at the application layer.  Such connections are
   by definition end-to-end and would need to be negotiated whenever
   application peers begin or restart operation in the network.

3.2. Retransmission

   Reliable message exchange in the Internet rests on protocols such as
   TCP and QUIC that detect data loss and automatically retransmit lost
   data.  That retransmission may be triggered by negative
   acknowledgments issued by the destination entity, but acknowledgment
   messages may also be lost; in that event, retransmission is
   triggered by the expiration of a timeout interval at the source
   entity shortly after the time at which an end-to-end acknowledgment
   is expected.  Since message round-trip times over multi-hop paths in
   deep space may be not only lengthy but also highly variable,
   statistical computation of an accurate retransmission timeout
   interval by, e.g., QUIC is in the general case not possible.
   Retransmission will either occur too soon, wasting limited

Burleigh                  Expires March 2025                   [Page 5]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

   transmission opportunities, or too late, delaying both data arrival
   and the release of retransmission buffer resources, and thereby
   introducing congestion in the network.

   This effect too can be avoided: end-to-end message exchange can be
   accomplished over paths constructed by concatenating multiple
   consecutive QUIC transmissions, each of which entails only signaling
   between two mutually visible network nodes for which the round-trip
   time is a function of the latency in direct signal propagation.  But
   to enable this model a new mechanism is needed that can concatenate
   multiple successive QUIC transmissions in a manner that is secure
   from end to end.  (The Bundle Protocol [RFC9171] is one such
   mechanism.)

   Beyond unnecessary consumption of network resources, end-to-end
   retransmission in an interplanetary internet also unnecessarily
   retards data delivery to the destination.  For a message on a 3-hop
   path from node A to node D via nodes B and C, the corruption of a
   single frame in the forward transmission from node C would cause
   node D to send a negative acknowledgment back to node A, triggering
   retransmission from A.  If the hop from A to B or from B to C spans
   20 light minutes of space, delivery of the message is delayed by at
   least 40 minutes.  Since the message was received successfully at C,
   network resource utilization and delivery timeliness are both
   improved if node C receives the negative acknowledgment and
   retransmits the message itself.

3.3. Routing and Forwarding

   Delivery of a message in the Internet is reliant on asserting the
   address (location within the network topology) of the message's
   destination and on providing network routers with sufficient
   knowledge of network topology to enable the forwarding of the
   message to that address.

   The Internet responds dynamically to changes in network topology by
   means of routing protocols, which convey information about those
   changes to routers.  When routing protocol message propagation is
   very rapid, as in the Internet, a generally accurate understanding
   of network topology can be sustained across the network.

   But when it is not, due to lengthy and highly variable message
   delivery latency as in a deep space network, not all nodes of the
   network will agree on its topology at any moment.  Changes in
   network topology will result in routing errors, data loss, and poor
   network performance.

Burleigh                  Expires March 2025                   [Page 6]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

   This effect can be avoided if dynamic routing in the deep space
   network is abandoned and, instead, manually provisioned routing
   tables are managed in coordination among network administrators.
   But the scope of this management activity will grow as the network
   grows.  For a future deep space network potentially comprising
   thousands of nodes (on Earth, in Earth orbit, on other planetary
   surfaces, in other planetary orbits, at LaGrange points, etc.), the
   canonical Internet routing and forwarding model would not be
   satisfactory.

3.4. Domain Name System

   Internet applications typically issue messages to named entities
   rather than to network addresses, relying on the Domain Name System
   (DNS) to resolve entities' names to their IP addresses.  DNS name
   resolution is performed by querying a DNS server and waiting for a
   response, a round-trip communication.  When the DNS server is in
   close proximity (e.g., on the same planetary surface, as on Earth),
   this works well.  But lengthy round-trip times would severely retard
   the domain name resolution on which many applications rely.

   This effect can be addressed by deploying DNS servers to remote
   locations in the solar system so that the round-trip time from any
   node to the nearest DNS server is small.  But this would entail
   synchronization of DNS information among all servers in the network
   via messages that may take hours or days to reach all destinations.
   That synchronization would consume network bandwidth that might
   better be used to convey scientific or industrial information, and
   meanwhile inconsistencies among DNS servers would likely cause
   routing errors.

3.5. Time to live

   In the Internet, excessive consumption of network resources
   (transmission bandwidth) by any single message is detected when the
   message's "hop count" violates a designated limit.  At that time the
   message can be assumed to be in a routing loop and should be
   discarded from the network.

   This mechanism is effective because data in the Internet are in
   generally continuous motion, residing only momentarily in the memory
   buffers of any single forwarding node.  A steadily decreasing count
   of remaining authorized hops ("time to live", or TTL) is a rough
   analog for the passage of time.

   In an interplanetary network, however, a message may routinely be
   required to wait at a forwarding node until the start of the next

Burleigh                  Expires March 2025                   [Page 7]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

   opportunity to transmit it to the next node on its network path.
   While waiting it is consuming network resources (buffer space), and
   that resource consumption might be excessive in an analogous way:
   the next transmission opportunity might never begin, in which case
   the message should be discarded from the network even if its TTL has
   never been decremented at all.

   Mechanisms for detecting expiration of this alternative "time to
   live" will be needed in the implementation of the interplanetary
   network.

3.6. Flow control

   Limiting the rate at which a source inserts data into the network to
   the rate at which the destination is able to receive and process it
   - flow control - is critical to network performance.  If the
   destination node's buffers are filled by inbound traffic more
   rapidly than they can be emptied by the application, buffers will
   overflow; this will cause data loss, resulting in degraded bandwidth
   utilization and delayed data delivery.

   In the Internet that limit can be inferred from the rate at which
   data arrival acknowledgments are received.  In an interplanetary
   network, however, signal propagation latency may be so great that
   the receiver's buffers overflow long before the sender receives
   enough acknowledgment information to compute a reduced transmission
   rate.

   Put simply, flow control in the interplanetary network must be
   anticipatory rather than reactive.

3.7. Congestion control

   Limiting the rates at which data sources in aggregate insert data
   into the network so as to ensure that the network can successfully
   convey all data to all destinations - congestion control - is
   likewise critical to network performance.  In the Internet, a
   mismatch between the rates at which data are inserted and removed
   from the network results in buffer overflow at relay points.  Nodes
   detect imminent congestion and signal one another to reduce rates of
   application data acceptance, preventing congestion.

   As with flow control, signal propagation latency in an
   interplanetary network may be so great that congestion signals may
   not arrive soon enough to trigger timely reduction in application
   data insertion.  Buffers will overflow not only at end systems but
   also at relay points.

Burleigh                  Expires March 2025                   [Page 8]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

   Again, congestion control in the interplanetary network must be
   anticipatory rather than reactive.

3.8. Security

   Internet security protocols such as IPsec and TLS rely on
   authentication and key agreements negotiated by means of handshakes
   between end systems.  These security agreements are subject to the
   same constraints that limit the usability of transport-layer
   connections as discussed in 3.2 above.  End-to-end security
   associations are especially difficult to establish over a deep space
   network that includes multiple relay points.  An alternative model
   of managed security associations is needed.

   Point-to-point security associations between topologically adjacent
   nodes can be established more readily, but migrating to an
   architecture based on achieving end-to-end communication by
   concatenating multiple individually secured "hops" would raise an
   additional problem.  Internet security protocols protect data while
   in transit via other protocols (IP, TCP, QUIC), but data may reside
   in the memory of a relay node - no longer in transit, thus no longer
   secured - for minutes, hours, or days while awaiting onward
   transmission.  A means of securing application data at rest is
   needed.  It would be preferable to avoid requiring each application
   protocol to perform this function itself.

3.9. Network management

   Commonly used Internet network management protocols - SNMP, NETCONF,
   and RESTCONF - can be used over interplanetary distances, but care
   must be taken.  The potentially very long round-trip times between
   clients and servers limit the utility of synchronous queries for
   node state, such as the SNMP GetRequest/Response cycle; responding
   information may be obsolete by the time it arrives.

   Asynchronous notifications, such as SNMP's Trap protocol data units,
   will be more suitable but will still be of limited value due to
   signal propagation latency.  In general it will be helpful to rely
   increasingly on node autonomy to manage the network, using
   asynchronous commands to configure autonomous operation.

3.10. Electronic mail

   Email is a wholly delay-tolerant application, perfect for the
   interplanetary network.  However, at least three aspects of standard
   email as it functions within the Internet are somewhat problematic
   in deep space deployment.

Burleigh                  Expires March 2025                   [Page 9]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

   First, email is conveyed to POP3 and IMAP servers (for local access
   by means of mail clients such as Outlook and Gmail) via the Simple
   Mail Transfer Protocol (SMTP).  Only SMTP servers that have been
   adapted to use some transport protocol other than TCP will be
   suitable for use in the interplanetary network, as standard TCP is
   not useful over interplanetary distances as discussed in [DURST].
   Other transport protocols that operate end-to-end as TCP does, such
   as QUIC, raise once again the retransmission concerns discussed in
   section 3.2 above.

   Second, SMTP communication is performed in "sessions", each
   comprising zero or more SMTP transactions.  Implementations may need
   to be modified to tolerate transaction closure latencies on the
   order of hours or days.

   Third, the headers of modern SMTP messages typically include
   references to assets identified by domain names.  As the identified
   assets may be arbitrarily distant, resolution of those domain names
   is subject to the DNS considerations discussed in 3.4 above.

3.11. Clock alignment

   Agreement on the current time is important for many network
   functions.  In the Internet, the Network Time Protocol (NTP) serves
   to minimize disagreement on time among network assets.  However, NTP
   relies on the statistically valid determination of precise timestamp
   message transit latencies among NTP peers.  Message transit
   latencies among interplanetary network nodes will be lengthy; the
   length of time required to converge on the current time may be so
   great that the consensus time may already be incorrect, due to the
   orbital motion of the peers, by the time it is computed.

4. Conclusions

   All this said, the Internet protocol suite will certainly play a
   vital role in deep space network communications, in much the same
   way that Ethernet and WiFi play a vital role in the Internet:
   enclaves of local Internet architecture will surely be established
   wherever in the Solar System multiple co-located computing platforms
   must communicate.

   And with minor modifications the Internet protocol suite might
   indeed be able to address concurrently all of the concerns
   identified here, sustaining communications over deep space links,
   for a network of sufficiently limited scope.  That is, for a small,
   static set of directly interconnected nodes among which no multi-hop
   paths are necessary, all connections are permanent, and the efforts

Burleigh                  Expires March 2025                  [Page 10]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

   of network administrators suffice to manage all routing, perhaps
   only a few additional expectations (e.g., flow control, time to
   live) would need to be lowered.

   But retaining in a Solar System-wide network the scope of automation
   that was developed for the Internet, enabling comparable power and
   scalability, will require either major extensions to the current
   Internet stack or else a different architecture.

   Note that such a "Delay-Tolerant Networking" (DTN) architecture
   already exists [RFC4838].  It was first demonstrated in deep space
   in 2008, on the EPOXI spacecraft en route to a comet; it has been
   operating on-board the International Space Station since 2016; it is
   in experimental operation aboard the KPLO (Danuri) spacecraft now in
   lunar orbit; it is now in operational use on-board the PACE
   spacecraft, conducting Earth observational science since early 2024.
   Standardization of the DTN protocols is mature, both within the
   Consultative Committee for Space Data Systems and within the
   Internet Engineering Task Force (most notably RFCs 9171 and 9172).
   Multiple open-source implementations of the protocols are widely
   available.

   Marc Blanchet's article has done the network communications
   community a great service by providing a historical overview of deep
   space communications research.  Some of the conclusions reached in
   the article merit clarification, but this is a conversation from
   which we will all benefit.

5. Security Considerations

   Security considerations raised in this paper are discussed in
   section 3.8.

6. IANA Considerations

   No IANA action is proposed in this paper.

7. References

7.1. Normative References

   Not applicable.

Burleigh                  Expires March 2025                  [Page 11]
Internet-Draft      Bundle-in-Bundle Encapsulation          August 2024

7.2. Informative References

   [COMSOC]  https://www.comsoc.org/publications/ctn/beyond-earth-
   exploring-future-deep-space-communications, Blanchet, M., "Beyond
   Earth: Exploring the Future of Deep Space Communications", May 2024.

   [DURST] https://www.researchgate.net/publication/267721105 Durst,
   R., P. Feighery, and K. Scott, "Why not use the Standard Internet
   Suite for Interplanetary Internet?", 2000.

   [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
   Requirement Levels", RFC 2119, March 1997.

   [RFC4838] Cerf, V., et al, "Delay-Tolerant Networking Architecture",
   RFC 4838, April 2007.

   [RFC9000] Iyengar, J., Ed. and Thomson, M., Ed., "QUIC: A UDP-Based
   Multiplexed and Secure Transport", RFC 9000, May 2021.

   [RFC9171] Burleigh, S., E. Birrane, and K. Fall, "Bundle Protocol
   Version 7", RFC 9171, January 2022.

Acknowledgments

   This document was prepared using 2-Word-v2.0.template.dot.

Authors' Address

   Scott Burleigh

   IPNGROUP

   1435 Woodhurst Blvd.

   McLean, VA 22102

   United States of America

   Email: sburleig.sb@gmail.com

Burleigh                  Expires March 2025                  [Page 12]