PipeStream: A Recursive Entity Streaming Protocol for Distributed Processing over QUIC
draft-krickert-pipestream-03
This document is an Internet-Draft (I-D).
Anyone may submit an I-D to the IETF.
This I-D is not endorsed by the IETF and has no formal standing in the
IETF standards process.
| Document | Type | Active Internet-Draft (individual) | |
|---|---|---|---|
| Author | Kristian E Rickert | ||
| Last updated | 2026-07-23 | ||
| RFC stream | (None) | ||
| Intended RFC status | (None) | ||
| Formats | |||
| Additional resources |
GitHub Repository
|
||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-krickert-pipestream-03
Individual Submission K. Rickert
Internet-Draft PipeStream AI
Intended status: Standards Track 23 July 2026
Expires: 24 January 2027
PipeStream: A Recursive Entity Streaming Protocol for Distributed
Processing over QUIC
draft-krickert-pipestream-03
Abstract
This document specifies PipeStream, a recursive scatter-gather
streaming protocol for hierarchical task decomposition and
distributed processing over QUIC transport. PipeStream enables the
decomposition (scattering) of complex, arbitrary workloads into
constituent sub-tasks, their transmission across distributed
processing nodes, and subsequent rehydration (gathering) at
destination endpoints.
While application-layer protocols like gRPC provide stream
multiplexing, PipeStream embeds a hierarchical state machine directly
into the protocol. It employs a dual-stream architecture consisting
of a data stream for payload transmission and a control stream for
tracking completion status and maintaining distributed consistency.
PipeStream defines a generic 2-bit Data Layer field for entity
representation, leaving the concrete payload semantics to application
profiles. To ensure consistency across parallel processing
pipelines, the protocol implements checkpoint blocking, guaranteeing
that all constituent parts of a decomposed workload are successfully
processed before rehydration operations commence.
About This Document
This note is to be removed before publishing as an RFC.
Status information for this document may be found at
https://datatracker.ietf.org/doc/draft-krickert-pipestream/.
Discussion of this document takes place on the Individual Group
mailing list (mailto:kristian.rickert@pipestream.ai).
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Rickert Expires 24 January 2027 [Page 1]
Internet-Draft PipeStream July 2026
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 24 January 2027.
Copyright Notice
Copyright (c) 2026 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1. Problem Statement . . . . . . . . . . . . . . . . . . . . 5
1.1.1. The Limits of Existing Transport and RPC
Mechanisms . . . . . . . . . . . . . . . . . . . . . 6
1.2. Applicability . . . . . . . . . . . . . . . . . . . . . . 7
1.3. PipeStream Overview . . . . . . . . . . . . . . . . . . . 7
1.4. Design Philosophy . . . . . . . . . . . . . . . . . . . . 8
1.5. Protocol Layering . . . . . . . . . . . . . . . . . . . . 8
1.6. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1. Protocol Entities . . . . . . . . . . . . . . . . . . . . 9
2.2. Dehydration and Rehydration . . . . . . . . . . . . . . . 10
2.3. Consistency Mechanisms . . . . . . . . . . . . . . . . . 10
2.4. Resilience Mechanisms (Protocol Layer 2) . . . . . . . . 11
2.5. Data Representation . . . . . . . . . . . . . . . . . . . 11
2.6. Additional Terms . . . . . . . . . . . . . . . . . . . . 12
3. Protocol Layers . . . . . . . . . . . . . . . . . . . . . . . 12
3.1. Layer 0: Core Protocol . . . . . . . . . . . . . . . . . 12
3.2. Layer 1: Recursive Extension . . . . . . . . . . . . . . 13
3.3. Layer 2: Resilience Extension . . . . . . . . . . . . . . 13
Rickert Expires 24 January 2027 [Page 2]
Internet-Draft PipeStream July 2026
3.4. Capability Negotiation . . . . . . . . . . . . . . . . . 13
3.4.1. Version Negotiation . . . . . . . . . . . . . . . . . 14
3.4.2. Serialization Format Negotiation . . . . . . . . . . 15
4. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . 16
4.1. Design Goals . . . . . . . . . . . . . . . . . . . . . . 16
4.1.1. True Streaming Processing . . . . . . . . . . . . . . 16
4.1.2. Recursive Decomposition . . . . . . . . . . . . . . . 16
4.1.3. Checkpoint Consistency . . . . . . . . . . . . . . . 16
4.1.4. Control and Data Plane Separation . . . . . . . . . . 16
4.1.5. QUIC Foundation . . . . . . . . . . . . . . . . . . . 16
4.1.6. Multi-Layer Data Representation . . . . . . . . . . . 17
4.2. Architecture Summary . . . . . . . . . . . . . . . . . . 17
4.3. Connection Lifecycle . . . . . . . . . . . . . . . . . . 18
5. QUIC Stream Mapping . . . . . . . . . . . . . . . . . . . . . 18
5.1. Control Stream (Stream 0) . . . . . . . . . . . . . . . . 18
5.1.1. Stream Identification . . . . . . . . . . . . . . . . 18
5.1.2. Usage Rules . . . . . . . . . . . . . . . . . . . . . 18
5.1.3. Flow Control Considerations . . . . . . . . . . . . . 19
5.1.4. Heartbeat Mechanism . . . . . . . . . . . . . . . . . 19
5.1.5. Transport Session vs. Application Session Context . . 20
5.1.6. Interaction with QUIC Flow Control and Congestion
Control . . . . . . . . . . . . . . . . . . . . . . . 20
5.2. Entity Streams . . . . . . . . . . . . . . . . . . . . . 21
5.2.1. Stream Identification and Direction . . . . . . . . . 21
5.2.2. One Entity per Stream . . . . . . . . . . . . . . . . 21
5.3. Prohibition of 0-RTT Early Data . . . . . . . . . . . . . 22
5.4. Performance Considerations . . . . . . . . . . . . . . . 22
5.4.1. Entity Granularity and Stream Overhead . . . . . . . 22
5.4.2. Control Stream Priority . . . . . . . . . . . . . . . 23
5.5. Transport Error Mapping . . . . . . . . . . . . . . . . . 23
6. Frame Formats . . . . . . . . . . . . . . . . . . . . . . . . 24
6.1. Control Stream Framing (Stream 0) . . . . . . . . . . . . 24
6.1.1. UCF Header . . . . . . . . . . . . . . . . . . . . . 24
6.1.2. Frame Types . . . . . . . . . . . . . . . . . . . . . 25
6.2. Status Frames (Layer 0) . . . . . . . . . . . . . . . . . 25
6.2.1. Status Frame Format (0x50) . . . . . . . . . . . . . 25
6.2.2. Status Codes . . . . . . . . . . . . . . . . . . . . 26
6.2.3. Entity Status State Machine . . . . . . . . . . . . . 27
6.2.4. Cursor Update Extension . . . . . . . . . . . . . . . 29
6.2.5. Status Reporting Direction . . . . . . . . . . . . . 29
6.3. Scope Digest Frame (0x54) . . . . . . . . . . . . . . . . 30
6.4. Barrier Frame (0x55) . . . . . . . . . . . . . . . . . . 31
6.5. GOAWAY Frame (0x56) . . . . . . . . . . . . . . . . . . . 31
6.5.1. Graceful Shutdown Procedure . . . . . . . . . . . . . 32
6.6. Yield and Claim Check Extensions (Layer 2) . . . . . . . 33
6.6.1. Extension Header . . . . . . . . . . . . . . . . . . 33
6.6.2. Yield Extension (Stat = 0x8) . . . . . . . . . . . . 33
6.6.3. Claim Check Extension (Stat = 0x9) . . . . . . . . . 34
Rickert Expires 24 January 2027 [Page 3]
Internet-Draft PipeStream July 2026
6.7. Serialized Message Frames (0x80-0xFF) . . . . . . . . . . 34
6.8. Entity Frames . . . . . . . . . . . . . . . . . . . . . . 35
6.8.1. Entity Frame Structure . . . . . . . . . . . . . . . 35
6.8.2. Message Schema (CDDL) . . . . . . . . . . . . . . . . 35
6.8.3. Checksum Algorithm . . . . . . . . . . . . . . . . . 37
6.8.4. Chunked Transfer . . . . . . . . . . . . . . . . . . 37
7. Entity Payload Model . . . . . . . . . . . . . . . . . . . . 38
7.1. Entity Header . . . . . . . . . . . . . . . . . . . . . . 38
7.2. Data Layers . . . . . . . . . . . . . . . . . . . . . . . 38
7.3. Application-Level Envelope . . . . . . . . . . . . . . . 39
7.4. Common Storage References . . . . . . . . . . . . . . . . 39
8. Protocol Operations . . . . . . . . . . . . . . . . . . . . . 40
8.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 40
8.2. CONNECT Action . . . . . . . . . . . . . . . . . . . . . 42
8.2.1. ALPN Identifier . . . . . . . . . . . . . . . . . . . 42
8.2.2. Capability Exchange . . . . . . . . . . . . . . . . . 42
8.3. PARSE Action . . . . . . . . . . . . . . . . . . . . . . 42
8.4. PROCESS Action . . . . . . . . . . . . . . . . . . . . . 43
8.5. SINK Action . . . . . . . . . . . . . . . . . . . . . . . 44
9. Rehydration Semantics . . . . . . . . . . . . . . . . . . . . 44
9.1. Entity ID Lifecycle and Cursor . . . . . . . . . . . . . 44
9.2. Assembly Manifest . . . . . . . . . . . . . . . . . . . . 46
9.3. Checkpoint Blocking . . . . . . . . . . . . . . . . . . . 46
9.4. Scope ID Allocation (Layer 1) . . . . . . . . . . . . . . 47
9.5. Scope Digest Propagation (Layer 1) . . . . . . . . . . . 48
9.6. Rehydration Readiness Tracking . . . . . . . . . . . . . 49
9.7. Stopping Point Validation (Layer 2) . . . . . . . . . . . 49
10. Security Considerations . . . . . . . . . . . . . . . . . . . 49
10.1. Transport Security . . . . . . . . . . . . . . . . . . . 49
10.2. Entity Payload Integrity . . . . . . . . . . . . . . . . 49
10.2.1. Algorithm Agility . . . . . . . . . . . . . . . . . 50
10.3. Resource Exhaustion . . . . . . . . . . . . . . . . . . 51
10.4. Amplification Attacks . . . . . . . . . . . . . . . . . 52
10.5. Privacy Considerations . . . . . . . . . . . . . . . . . 52
10.6. Replay and Token Reuse . . . . . . . . . . . . . . . . . 53
10.6.1. Yield Token Replay . . . . . . . . . . . . . . . . . 53
10.6.2. Claim Check Replay . . . . . . . . . . . . . . . . . 54
10.7. Encryption Key Management . . . . . . . . . . . . . . . 54
11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 55
11.1. ALPN Identifier Registration . . . . . . . . . . . . . . 55
11.2. PipeStream Frame Type Registry . . . . . . . . . . . . . 55
11.2.1. Unknown Frame Handling . . . . . . . . . . . . . . . 56
11.3. PipeStream Status Code Registry . . . . . . . . . . . . 56
11.4. PipeStream Error Code Registry . . . . . . . . . . . . . 57
11.5. PipeStream Serialization Format Registry . . . . . . . . 58
11.6. URI Scheme Registration . . . . . . . . . . . . . . . . 59
11.6.1. URI Syntax . . . . . . . . . . . . . . . . . . . . . 60
11.7. Guidance for Designated Experts . . . . . . . . . . . . 60
Rickert Expires 24 January 2027 [Page 4]
Internet-Draft PipeStream July 2026
12. References . . . . . . . . . . . . . . . . . . . . . . . . . 61
12.1. Normative References . . . . . . . . . . . . . . . . . . 61
12.2. Informative References . . . . . . . . . . . . . . . . . 62
Appendix A. Protocol Layer Capability Matrix . . . . . . . . . . 63
Appendix B. Relationship to Existing Protocols . . . . . . . . . 64
B.1. QUIC as Application Transport . . . . . . . . . . . . . . 64
B.2. Efficiency Considerations . . . . . . . . . . . . . . . . 65
B.2.1. Framing Overhead . . . . . . . . . . . . . . . . . . 65
B.2.2. Stateless Control Plane . . . . . . . . . . . . . . . 65
B.2.3. Stream Independence . . . . . . . . . . . . . . . . . 65
B.3. HTTP/3 . . . . . . . . . . . . . . . . . . . . . . . . . 66
B.4. gRPC . . . . . . . . . . . . . . . . . . . . . . . . . . 66
B.5. WebTransport . . . . . . . . . . . . . . . . . . . . . . 67
B.6. Media over QUIC Transport . . . . . . . . . . . . . . . . 67
B.7. Broker-Based Pipeline Architectures . . . . . . . . . . . 68
B.8. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 68
Appendix C. Schema Reference (CDDL) . . . . . . . . . . . . . . 69
Appendix D. Implementation Status . . . . . . . . . . . . . . . 73
D.1. Reference Implementation . . . . . . . . . . . . . . . . 74
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 74
1. Introduction
1.1. Problem Statement
Distributed processing pipelines face significant challenges when
handling large, complex inputs that require multiple stages of
transformation, analysis, and enrichment. Traditional batch
processing approaches require entire inputs to be loaded into memory,
processed sequentially, and transmitted in their entirety between
processing stages. This methodology introduces substantial latency,
excessive memory consumption, and poor utilization of distributed
computing resources.
Modern distributed processing workflows increasingly demand the
ability to:
* Process inputs incrementally as data becomes available
* Distribute processing load across heterogeneous worker nodes
* Maintain consistency guarantees across parallel processing paths
* Handle inputs of arbitrary size without memory constraints
* Support recursive decomposition where constituent parts may
themselves be decomposed
Rickert Expires 24 January 2027 [Page 5]
Internet-Draft PipeStream July 2026
* Scale from single inputs to collections of millions of entries
Current approaches based on batch processing and store-and-forward
architectures are inefficient for large inputs and fail to exploit
the inherent parallelism available in distributed processing
environments. Furthermore, existing streaming protocols do not
provide the consistency semantics required for hierarchical
processing where the integrity of the rehydrated output depends on
the successful processing of all constituent parts.
These pipelines also increasingly span organizational and vendor
boundaries: a processing stage operated by one party must
interoperate with stages operated by others. Today that
interoperability is achieved through per-vendor SDKs and bespoke
coordination logic, because no standardized wire protocol expresses
the decomposition, tracking, and reassembly semantics these systems
share. PipeStream specifies that wire protocol.
1.1.1. The Limits of Existing Transport and RPC Mechanisms
Existing multiplexed RPC frameworks (for example, gRPC over HTTP/2 or
HTTP/3) and raw transport APIs (for example, WebTransport) excel at
point-to-point data streaming. However, they lack native mechanisms
for distributed consistency when workloads are hierarchically
decomposed.
In a standard streaming RPC model, an application must manually track
state if a primary task spawns tens of thousands of distributed sub-
tasks. PipeStream solves this by pushing the scatter-gather state
machine down to the protocol level. Through the use of cryptographic
Scope Digests (Section 6.3) and Barrier Frames (Section 6.4),
PipeStream provides native barrier synchronization semantics,
ensuring a parent stream cannot logically terminate until all
scattered child entities across distributed nodes have reached a
terminal status.
Broker-based pipeline architectures (distributed commit logs and
message queues) address a related problem by decoupling stages
through intermediary persistence, but they externalize exactly the
semantics PipeStream makes native: determining whether a
hierarchically decomposed job has fully completed requires an
external coordination store, and end-to-end integrity verification is
left to the application. Appendix B compares PipeStream to these and
other existing protocols in detail.
Rickert Expires 24 January 2027 [Page 6]
Internet-Draft PipeStream July 2026
1.2. Applicability
PipeStream is a domain-neutral recursive streaming protocol. Its
design has been validated against distributed document processing,
but it is equally applicable to any workload that can be modeled as
hierarchical decomposition of a root entity into sub-entities,
parallel processing of those sub-entities, and deterministic
reassembly of results. Example domains include:
* Distributed document processing and content enrichment pipelines
* Hierarchical video transcoding and rendering (scene decomposition)
* Federated computation pipelines such as distributed machine
learning
* Genomic sequencing and assembly workflows
PipeStream uniquely bridges true streaming and request-response
processing models. Layer 0 streaming is optimal when processing
nodes have low-latency interconnects and can process entities
incrementally. Layer 2 yield/resume and claim checks handle
workflows that require external service calls, rate limiting, human
approval gates, or cross-session resumption. This dual nature allows
a single protocol to serve both real-time streaming pipelines and
complex multi-stage workflows with heterogeneous latency
characteristics.
1.3. PipeStream Overview
PipeStream addresses these challenges by defining a streaming
protocol that enables incremental processing with strong consistency
guarantees. The protocol is built upon QUIC [RFC9000] transport,
leveraging its native support for multiplexed streams, low-latency
connection establishment, and reliable delivery semantics.
The fundamental innovation of PipeStream is its treatment of inputs
as recursive compositions of entities. A root entity MAY be
decomposed into multiple sub-entities, each of which MAY itself be
further decomposed, creating a tree structure of processing tasks.
This recursive decomposition enables fine-grained parallelism while
the protocol's control stream mechanism ensures that all branches of
the decomposition tree are tracked and synchronized.
PipeStream employs a dual-stream design:
Rickert Expires 24 January 2027 [Page 7]
Internet-Draft PipeStream July 2026
1. *Data Stream*: Carries entity payloads through the processing
pipeline. Entities flow through this stream with minimal
buffering, enabling low-latency incremental processing.
2. *Control Stream*: Carries control information tracking the status
of entity decomposition and rehydration. The control stream
ensures that all parts of a dehydrated entity are accounted for
before rehydration proceeds.
1.4. Design Philosophy
PipeStream implements a recursive scatter-gather pattern
[scatter-gather] over QUIC streams. An input entity is "dehydrated"
(scattered) at the source into constituent sub-entities, these sub-
entities are transmitted and processed in parallel across distributed
pipeline stages, and finally the results are "rehydrated" (gathered)
at the destination to reconstitute the complete processed output.
The checkpoint blocking mechanism (Section 9.3) provides barrier
synchronization semantics analogous to the barrier pattern in
parallel computing.
This approach provides several advantages:
* *Incremental Processing*: Processing nodes MAY begin work on early
entities before the complete input has been transmitted.
* *Parallelism*: Independent entities MAY be processed concurrently
across multiple worker nodes.
* *Memory Efficiency*: No single node is required to hold the
complete input in memory.
* *Fault Isolation*: Failures in processing individual entities can
be detected, reported, and potentially retried without affecting
other entities.
* *Consistency*: The checkpoint blocking mechanism ensures that
rehydration operations proceed only when all constituent parts
have been successfully processed.
1.5. Protocol Layering
PipeStream is organized into three protocol layers to accommodate
varying deployment requirements:
Rickert Expires 24 January 2027 [Page 8]
Internet-Draft PipeStream July 2026
+================+============+=============================+
| Protocol Layer | Name | Description |
+================+============+=============================+
| Layer 0 | Core | Basic streaming, dehydrate/ |
| | | rehydrate, checkpoint |
+----------------+------------+-----------------------------+
| Layer 1 | Recursive | Hierarchical scopes, digest |
| | | propagation, barriers |
+----------------+------------+-----------------------------+
| Layer 2 | Resilience | Yield/resume, claim checks, |
| | | completion policies |
+----------------+------------+-----------------------------+
Table 1
Implementations MUST support Layer 0. Support for Layers 1 and 2 is
OPTIONAL and negotiated during connection establishment.
1.6. Scope
This document specifies the PipeStream protocol including message
formats, state machines, error handling, and the interaction between
data and control streams. The document defines the transport-level
layer field and recursive processing semantics but leaves concrete
payload meaning to application profile specifications.
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
2.1. Protocol Entities
*Entity* The fundamental unit of data flowing through a PipeStream
pipeline. An Entity represents either a complete input or a
constituent part of a decomposed input. Each Entity possesses a
unique identifier within its processing scope and carries payload
data tagged with a transport-level layer value. Entities are
immutable once created; transformations produce new Entities
rather than modifying existing ones. An input enters the pipeline
as a single root Entity and MAY be decomposed into multiple
Entities during processing; it is considered complete when its
root Entity (or the rehydrated result of its decomposition) exits
the pipeline.
Rickert Expires 24 January 2027 [Page 9]
Internet-Draft PipeStream July 2026
*Scope* A hierarchical namespace for Entity IDs. Each scope
maintains its own Entity ID space, cursor, and Assembly Manifest.
Scopes enable root entities to contain parts, and parts to contain
sub-tasks, each with independent ID management. (Protocol Layer
1)
2.2. Dehydration and Rehydration
*Scatter-Gather* The distributed processing pattern implemented by
PipeStream. A single input is "scattered" (dehydrated) into
multiple parts for parallel processing, and the results are
"gathered" (rehydrated) back into a single output. PipeStream
extends classical scatter-gather with recursive nesting: any
scattered part may itself be scattered further.
*Dehydrate (Scatter)* The operation of decomposing an Entity into
multiple constituent Entities for parallel or distributed
processing. When an Entity is dehydrated, the originating node
MUST create an Assembly Manifest entry recording the identifiers
of all resulting sub-entities. The dehydration operation is
recursive; a sub-entity produced by dehydration MAY itself be
dehydrated, creating a tree of decomposition. Dehydration
transitions data from a solid state (a single complete unit) to a
fluid state (multiple in-flight entities).
*Rehydrate (Gather)* The operation of reassembling multiple Entities
back into a single composite Entity. A rehydrate operation MUST
NOT proceed until all constituent Entities listed in the
corresponding Assembly Manifest entry have been received and
processed (or handled according to the Completion Policy).
Rehydration transitions data from a fluid state back to a solid
state.
*Solid State* An Entity that exists as a complete, stored unit or as
a single root Entity entering or exiting a pipeline. Contrast
with "fluid state".
*Fluid State* An input that has been decomposed into multiple in-
flight Entities being processed in parallel across distributed
nodes. An input is in the fluid state between dehydration and
rehydration. Contrast with "solid state".
2.3. Consistency Mechanisms
*Checkpoint* A synchronization point in the processing pipeline
Rickert Expires 24 January 2027 [Page 10]
Internet-Draft PipeStream July 2026
where all in-flight Entities MUST reach a consistent state before
processing may continue. A checkpoint is considered "satisfied"
when all Assembly Manifest entries created before the checkpoint
have been resolved.
*Barrier* A synchronization point scoped to a specific subtree.
Unlike checkpoints which are global, barriers block only entities
dependent on a specific parent's descendants. (Protocol Layer 1)
*Control Stream* The control stream that tracks Entity completion
status throughout the processing pipeline. The Control Stream is
transmitted on a dedicated QUIC stream parallel to the data
streams.
*Assembly Manifest* A data structure within the Control Stream that
tracks the relationship between a composite Entity and its
constituent sub-entities produced by dehydration.
*Cursor* A pointer to the lowest unresolved Entity ID within a
scope. Entity IDs behind the cursor are considered resolved and
MAY be recycled. The cursor enables efficient ID space management
without global coordination.
2.4. Resilience Mechanisms (Protocol Layer 2)
*Yield* A temporary pause in Entity processing, typically due to
external dependencies (API calls, rate limiting, human approval).
A yielded Entity carries a continuation token enabling resumption
without reprocessing.
*Claim Check* A detached reference to a deferred Entity that can be
queried or resumed independently, potentially in a different
session. Claim checks enable asynchronous processing patterns and
retry queues.
*Completion Policy* A configuration specifying how to handle partial
failures during dehydration. Policies include STRICT (all must
succeed), LENIENT (continue with partial results), BEST_EFFORT
(complete with whatever succeeds), and QUORUM (require minimum
success ratio).
2.5. Data Representation
*Data Layer* A transport-level 2-bit field carried in the Entity
Header that identifies the payload representation. The concrete
meaning of each layer value is defined by an application profile
rather than by the core protocol.
Rickert Expires 24 January 2027 [Page 11]
Internet-Draft PipeStream July 2026
2.6. Additional Terms
*Pipeline* A configured sequence of processing stages through which
Entities flow.
*Processor* A node in the mesh that performs operations on entities
(e.g., transformation, dehydration, or rehydration).
*Sink* A terminal stage in a pipeline where rehydrated entities or
terminal entities are persisted or delivered to an external
system.
*Stage* A single processing step within a Pipeline.
*Scope Digest* A cryptographic summary (Merkle root) of all Entity
statuses within a completed scope, propagated to parent scopes for
efficient verification. (Protocol Layer 1)
3. Protocol Layers
PipeStream defines three protocol layers that build upon each other.
This layered approach allows simple deployments to use only the core
protocol while complex deployments can leverage advanced features.
3.1. Layer 0: Core Protocol
Layer 0 provides the fundamental streaming capabilities:
* Unified Control Frame (UCF) header (1-octet type + 4-octet length
prefix)
* Status frame (16-octet base bit-packed frame)
* Entity frame (header + payload)
* Status codes: PENDING, PROCESSING, COMPLETE, FAILED, CHECKPOINT
* Assembly Manifest for parent-child tracking
* Cursor-based Entity ID recycling
* Single-level dehydrate/rehydrate
* Checkpoint blocking
All implementations MUST support Layer 0.
Rickert Expires 24 January 2027 [Page 12]
Internet-Draft PipeStream July 2026
3.2. Layer 1: Recursive Extension
Layer 1 adds hierarchical processing capabilities:
* Scoped Entity ID namespaces (root -> component -> sub-task ->
leaf)
* Explicit Depth tracking in status frames
* SCOPE_DIGEST for Merkle-based subtree completion
* BARRIER for subtree-scoped synchronization
* Nested dehydration with depth tracking
Layer 1 is OPTIONAL. Implementations advertise Layer 1 support
during capability negotiation.
3.3. Layer 2: Resilience Extension
Layer 2 adds fault tolerance and async processing:
* YIELDED status with continuation tokens
* DEFERRED status with claim checks
* RETRYING, SKIPPED, ABANDONED statuses
* Completion policies (STRICT, LENIENT, BEST_EFFORT, QUORUM)
* Claim check extensions and deferred processing tokens
* Stopping point validation
Layer 2 is OPTIONAL and requires Layer 1. Implementations advertise
Layer 2 support during capability negotiation.
3.4. Capability Negotiation
PipeStream uses a two-tier negotiation model. The ALPN identifier
(Section 11.1) identifies the base PipeStream transport mapping,
while the capabilities structure handles dynamic resource limits and
optional layer support that may vary based on endpoint configuration
or real-time load.
Rickert Expires 24 January 2027 [Page 13]
Internet-Draft PipeStream July 2026
During CONNECT, endpoints exchange supported capabilities using the
capabilities structure. This message MUST be encoded using the
default CBOR format for both the client's initiation and the server's
response (Section 3.4.2).
serialization-format = uint .le 255
; Value from the PipeStream Serialization
; Formats registry (Section 11.5).
; This document defines only CBOR (0).
capabilities = {
layer0-core: bool, ; MUST be true
layer1-recursive: bool, ; Scoped IDs, digests
layer2-resilience: bool, ; Yield, claim checks
? max-scope-depth: uint .le 7, ; Default: 7 (8 levels, 0-7)
? max-entities-per-scope: uint, ; Default: 4,294,967,292
; (Section 9.1)
? max-window-size: uint, ; Default: 2,147,483,646
; (Max in-flight entities;
; see Section 9.1)
? serialization-format: serialization-format, ; Default: CBOR
? keepalive-timeout-ms: uint, ; Default: 30000 (30s)
}
Peers negotiate down to common capabilities. If Layer 2 is requested
but Layer 1 is not supported, Layer 2 MUST be disabled.
The layer0-core field MUST be true; an endpoint that receives a
Capabilities message with layer0-core set to false MUST close the
connection with PIPESTREAM_LAYER_UNSUPPORTED (0x0C). The max-
entities-per-scope and max-window-size values are bounded by the
Entity ID space constraints defined in Section 9.1; an endpoint MUST
NOT advertise values exceeding those bounds.
3.4.1. Version Negotiation
PipeStream protocol versioning is carried in two places: the ALPN
identifier and the Ver field in the STATUS frame (Section 6.2.1).
The ALPN identifier pipestream/1 identifies the major protocol
version and the QUIC transport mapping defined in this document. The
4-bit Ver field in STATUS frames carries the value 0x1 for this
specification.
A future major version of PipeStream (e.g., pipestream/2) would
register a new ALPN identifier. QUIC's native ALPN negotiation
during the TLS handshake provides version selection: if a client
offers both pipestream/2 and pipestream/1 and the server supports
only pipestream/1, the TLS handshake selects pipestream/1 without
Rickert Expires 24 January 2027 [Page 14]
Internet-Draft PipeStream July 2026
additional round trips. This mechanism is consistent with the
versioning approach used by HTTP/3 [RFC9114] and DNS over QUIC
[RFC9250].
Minor, backward-compatible extensions (such as new optional
capability fields or new status codes within the reserved ranges) do
not require a new ALPN identifier. Such extensions are negotiated
through the capabilities structure or the IANA registries defined in
Section 11.
3.4.2. Serialization Format Negotiation
The serialization_format field determines the encoding used for all
variable-length control messages (frame types 0x80-0xFF) and entity
headers.
This document defines a single serialization format: CBOR [RFC8949],
value 0 in the PipeStream Serialization Formats registry
(Section 11.5). All PipeStream implementations MUST support CBOR.
The serialization-format capability field and its registry exist to
permit future specifications to define additional formats; any such
specification MUST normatively define the encoding of every
serialized message in this document.
Negotiation proceeds as follows:
1. CBOR [RFC8949] is the default and MUST be supported by all
endpoints.
2. Each peer advertises its preferred serialization-format in its
Capabilities message.
3. If both peers advertise the same format and both support it, that
format is used.
4. If preferences differ, if either peer omits the field, or if a
peer advertises a value the other does not recognize, the peers
MUST fall back to CBOR [RFC8949].
The initial Capabilities exchange on a new connection MUST use the
default CBOR format for both the client's initiation and the server's
response. The negotiated serialization format and resource limits
take effect immediately following the successful completion of this
initial Capabilities exchange (one request and one response). If a
peer cannot decode the initial Capabilities exchange, it MUST close
the connection with PIPESTREAM_FRAME_ERROR (0x0D).
Rickert Expires 24 January 2027 [Page 15]
Internet-Draft PipeStream July 2026
4. Protocol Overview
This section provides a high-level overview of the PipeStream
protocol architecture, design principles, and operational model.
4.1. Design Goals
This section is descriptive; the normative requirements that realize
these goals appear in Sections 5 through 9.
4.1.1. True Streaming Processing
PipeStream enables entities to be transmitted and processed
incrementally as they become available. The framing and status
mechanisms operate on individual entities rather than complete
inputs, so a sender never needs to buffer a complete input before
initiating transmission.
4.1.2. Recursive Decomposition
The protocol supports recursive decomposition of entities, wherein a
single input entity may produce zero, one, or many output entities.
4.1.3. Checkpoint Consistency
PipeStream provides checkpoint blocking semantics (Section 9.3) to
maintain processing consistency across distributed workers.
4.1.4. Control and Data Plane Separation
The protocol maintains strict separation between the control plane
(the Control Stream) and the data plane (Entity Streams).
4.1.5. QUIC Foundation
PipeStream is defined directly over QUIC [RFC9000] to leverage:
* Native stream multiplexing without head-of-line blocking
* Built-in flow control at both connection and stream levels
* TLS 1.3 security by default
* Connection migration capabilities
Rickert Expires 24 January 2027 [Page 16]
Internet-Draft PipeStream July 2026
4.1.6. Multi-Layer Data Representation
The protocol carries a 2-bit data layer field supporting four
distinct data representation layers whose concrete semantics are
defined by application profiles:
+=======+===================+=====================================+
| Layer | Conventional Role | Description |
+=======+===================+=====================================+
| 0 | Raw input | Binary or originating payload bytes |
+-------+-------------------+-------------------------------------+
| 1 | Enriched | Annotated or partially processed |
| | intermediate | payload |
+-------+-------------------+-------------------------------------+
| 2 | Structured result | Normalized or extracted output |
+-------+-------------------+-------------------------------------+
| 3 | Extension | Application-specific payload |
| | | semantics |
+-------+-------------------+-------------------------------------+
Table 2
4.2. Architecture Summary
PipeStream uses a dual-plane architecture within a single QUIC
connection. The endpoint that initiates the QUIC connection is the
client. Either endpoint MAY originate entities (Section 5.2); in
many deployments the client acts as producer and the server as
consumer, but the protocol does not require this asymmetry after
connection establishment.
+=========+====================+=========+==========================+
| Stream | Type | Plane | Content |
+=========+====================+=========+==========================+
| Stream | Bidirectional | Control | STATUS, SCOPE_DIGEST, |
| 0 | (client-initiated) | | BARRIER, GOAWAY, |
| | | | CAPABILITIES, |
| | | | CHECKPOINT |
+---------+--------------------+---------+--------------------------+
| Entity | Unidirectional | Data | Entity frames (Header |
| Streams | (either endpoint) | | + Payload) |
+---------+--------------------+---------+--------------------------+
Table 3
Rickert Expires 24 January 2027 [Page 17]
Internet-Draft PipeStream July 2026
4.3. Connection Lifecycle
A PipeStream connection follows this lifecycle:
1. *Establishment:* Client initiates QUIC connection with ALPN
identifier "pipestream/1"
2. *Capability Exchange:* Client and server exchange supported
protocol layers and limits
3. *Control Stream Initialization:* Client opens Stream 0 as
bidirectional Control Stream
4. *Entity Streaming:* Entities are transmitted per Sections 5 and 6
5. *Termination:* Connection closes via GOAWAY-initiated graceful
shutdown (Section 6.5) or QUIC CONNECTION_CLOSE
5. QUIC Stream Mapping
PipeStream leverages the native multiplexing capabilities of QUIC
[RFC9000] to provide a clean separation between control coordination
and data transmission.
5.1. Control Stream (Stream 0)
The Control Stream provides the control plane for PipeStream
operations.
5.1.1. Stream Identification
The Control Stream MUST use QUIC Stream ID 0, which per RFC 9000 is a
bidirectional, client-initiated stream.
5.1.2. Usage Rules
1. The Control Stream MUST be opened immediately upon connection
establishment.
2. Capability negotiation (Section 3.4) MUST occur on Stream 0
before any Entity Streams are opened.
3. Stream 0 MUST NOT carry entity payload data.
Rickert Expires 24 January 2027 [Page 18]
Internet-Draft PipeStream July 2026
4. Implementations SHOULD assign the Control Stream a high priority
to ensure timely delivery of status updates. An implementation
MAY choose a different priority policy when operating in
constrained environments where QUIC stream scheduling overhead
must be minimized.
5.1.3. Flow Control Considerations
The Control Stream carries small, bit-packed control frames. STATUS
frame payloads are 16 octets base (21 octets on wire including the
5-octet UCF header). Implementations MUST ensure adequate flow
control credits:
* The initial MAX_STREAM_DATA for Stream 0 SHOULD be at least 8192
octets. A lower value is permissible for extremely constrained
devices but risks stalling status delivery.
* Implementations SHOULD NOT block Entity Stream transmission due to
Control Stream flow control exhaustion. In rare cases where
strict ordering between control and data planes is required by the
application, an implementation MAY temporarily pause entity
transmission until control stream credits are replenished.
5.1.4. Heartbeat Mechanism
QUIC already provides native transport liveness signals (for example,
PING and idle timeout handling). Implementations SHOULD rely on
those transport mechanisms for connection liveness.
PipeStream heartbeat frames are OPTIONAL and are intended for
application-level responsiveness checks (for example, detecting
stalled processing logic even when the transport remains healthy).
When used, an endpoint sends a STATUS frame with all fields set to
their heartbeat values:
Rickert Expires 24 January 2027 [Page 19]
Internet-Draft PipeStream July 2026
+==========+===============+============================+
| Field | Value | Description |
+==========+===============+============================+
| Type | 0x50 (STATUS) | |
+----------+---------------+----------------------------+
| Length | 16 | STATUS payload length with |
| | | no cursor or extension |
+----------+---------------+----------------------------+
| Stat | 0x0 | Heartbeat signal |
| | (UNSPECIFIED) | |
+----------+---------------+----------------------------+
| Entity | 0xFFFFFFFF | CONNECTION_LEVEL |
| ID | | |
+----------+---------------+----------------------------+
| Scope ID | 0x00000000 | Root scope |
+----------+---------------+----------------------------+
| Reserved | 0x00000000 | MUST be zero |
+----------+---------------+----------------------------+
Table 4
The KEEPALIVE_TIMEOUT defaults to 30 seconds. Endpoints MAY
negotiate a different value by including a keepalive-timeout-ms field
(in milliseconds) in the capabilities exchange (Section 3.4); the
effective timeout is the minimum of the two peers' advertised values.
When no status updates have been transmitted for KEEPALIVE_TIMEOUT,
an endpoint MAY send a heartbeat frame. If no data is received on
Stream 0 for 3 * KEEPALIVE_TIMEOUT, the endpoint SHOULD first apply
transport-native liveness policy (e.g., QUIC PING); it MAY close the
connection with PIPESTREAM_IDLE_TIMEOUT (0x02) when application-level
inactivity policy requires it.
5.1.5. Transport Session vs. Application Session Context
The session-id segment of the pipestream URI scheme (Section 11.6)
identifies application context for detached or resumable resources
(for example, Layer 2 yield/claim-check flows). PipeStream Layer 0
streaming semantics do not depend on this URI scheme.
5.1.6. Interaction with QUIC Flow Control and Congestion Control
PipeStream relies on the flow control and congestion control
mechanisms provided by QUIC [RFC9000] and does not define its own
transport-layer congestion control. QUIC provides flow control at
both the stream level (MAX_STREAM_DATA) and the connection level
(MAX_DATA). PipeStream's cursor-based backpressure (Section 9.1)
operates at the application layer and is complementary to QUIC flow
Rickert Expires 24 January 2027 [Page 20]
Internet-Draft PipeStream July 2026
control:
* QUIC flow control limits the number of bytes in flight on any
given stream or connection.
* PipeStream backpressure limits the number of entities in flight
(i.e., the number of Entity IDs between cursor and last_assigned).
When the QUIC connection-level flow control window is exhausted, new
Entity Streams cannot transmit data regardless of whether
PipeStream's entity window has capacity. Conversely, when
PipeStream's entity window is full, no new Entity Streams are opened
even if QUIC flow control credits are available. Implementations
MUST respect both limits. An implementation SHOULD monitor QUIC-
level flow control credit availability and avoid opening new Entity
Streams when connection-level credits are below the expected entity
size, to prevent head-of-line blocking across streams sharing the
connection budget.
5.2. Entity Streams
Entity Streams carry entity payload data using the Entity Frame
format defined in Section 6.8.
5.2.1. Stream Identification and Direction
Each Entity Stream is a QUIC unidirectional stream. Either endpoint
MAY open Entity Streams once the initial Capabilities exchange
(Section 3.4) has completed. Per [RFC9000], client-initiated
unidirectional streams carry Stream IDs 2, 6, 10, and so on, and
server-initiated unidirectional streams carry Stream IDs 3, 7, 11,
and so on.
Endpoints MUST NOT open bidirectional streams other than Stream 0.
An endpoint that receives a bidirectional stream other than Stream 0
MUST treat this as a connection error of type PIPESTREAM_FRAME_ERROR
(0x0D).
5.2.2. One Entity per Stream
Each Entity Stream carries exactly one Entity Frame (Section 6.8): a
Header Length prefix, a serialized EntityHeader, and the payload
octets. The following rules apply:
1. The sender MUST close the stream (QUIC FIN) immediately after the
final payload octet. The end of the stream delimits the payload.
Rickert Expires 24 January 2027 [Page 21]
Internet-Draft PipeStream July 2026
2. When the EntityHeader includes a payload-length field, the
receiver MUST verify that the number of payload octets received
before the FIN equals that value. A mismatch MUST be treated as
a stream error of type PIPESTREAM_ENTITY_INVALID (0x05).
3. A sender that abandons transmission of an entity SHOULD abort the
stream with RESET_STREAM carrying an appropriate PipeStream error
code; the receiver MUST discard any partial payload data (see
Section 5.5).
4. A receiver that no longer requires an entity MAY send
STOP_SENDING on the corresponding Entity Stream.
5. Additional data received on an Entity Stream after a complete
Entity Frame MUST be treated as a stream error of type
PIPESTREAM_FRAME_ERROR (0x0D).
5.3. Prohibition of 0-RTT Early Data
QUIC permits applications to send data in 0-RTT early data before the
TLS handshake completes; such data is replayable by an attacker.
PipeStream capability negotiation establishes per-connection state
whose replay could alter negotiated limits or serialization formats.
Endpoints MUST NOT send PipeStream frames in 0-RTT early data, and a
server MUST NOT process PipeStream frames received in early data
before the QUIC handshake is confirmed. A future extension may
define 0-RTT semantics along with the required replay protections.
5.4. Performance Considerations
5.4.1. Entity Granularity and Stream Overhead
PipeStream's "one entity per stream" model (Section 5.2) provides
clean multiplexing and flow control but introduces per-stream
overhead (QUIC STREAM frame headers and internal stack state).
1. *Small Payloads:* For workloads consisting of many small entities
(e.g., <1 KiB), the per-stream overhead may become significant.
Implementations SHOULD avoid excessive fragmentation and prefer
coarser entity granularity where possible.
2. *Aggregation:* Application profiles MAY define mechanisms for
bundling multiple small logical units into a single PipeStream
entity to reduce transport-layer overhead.
Rickert Expires 24 January 2027 [Page 22]
Internet-Draft PipeStream July 2026
3. *Stream Limits:* Senders MUST respect the peer's QUIC
initial_max_streams_uni and MAX_STREAMS limits. High-frequency
entity producers SHOULD monitor stream credit availability and
adjust dehydration rates accordingly to avoid stalling the
pipeline.
5.4.2. Control Stream Priority
Implementations SHOULD assign the Control Stream (Stream 0) a higher
priority than Entity Streams. Timely delivery of STATUS and
SCOPE_DIGEST frames is critical for advancing the cursor and
releasing backpressure (Section 9.1).
5.5. Transport Error Mapping
PipeStream error signaling on Stream 0 and QUIC transport signals are
complementary. Endpoints SHOULD bridge them so peers receive both
transport-level and protocol-level context. An implementation MAY
omit bridging when operating as a simple pass-through proxy that does
not inspect entity status.
1. If an Entity Stream is aborted with RESET_STREAM or STOP_SENDING,
the endpoint SHOULD emit a corresponding status (FAILED when
terminal, ABANDONED, or policy-driven equivalent) for that entity
on Stream 0. The endpoint MAY omit the status frame only if the
connection itself is being closed immediately.
2. If PipeStream determines a terminal entity error first (for
example, checksum failure or invalid frame), the endpoint SHOULD
abort the affected Entity Stream with an appropriate QUIC error
and emit the corresponding PipeStream status/error context on
Stream 0. Aborting the stream MAY be deferred if the entity
payload is still needed for diagnostic purposes.
3. If Stream 0 is reset or becomes unusable, endpoints SHOULD treat
this as a control-plane failure and close the connection with
PIPESTREAM_CONTROL_RESET (0x03). An endpoint that can recover
control-plane state through an application-layer mechanism MAY
attempt reconnection before closing.
4. On QUIC connection termination (CONNECTION_CLOSE), entities
without a previously observed terminal status MUST be handled by
local failure policy, typically by marking them FAILED or
ABANDONED.
Rickert Expires 24 January 2027 [Page 23]
Internet-Draft PipeStream July 2026
6. Frame Formats
This section defines the wire formats for PipeStream frames. All
multi-octet integer fields are encoded in network byte order (big-
endian).
*Forward Compatibility:* All Reserved and Flags fields defined in
this section MUST be set to zero when sent and MUST be ignored by
receivers. This convention enables future specifications to assign
meaning to currently-reserved bits without breaking deployed
implementations. Receivers that encounter non-zero values in
reserved fields MUST NOT treat this as an error.
6.1. Control Stream Framing (Stream 0)
To ensure forward compatibility and consistent parsing, all messages
on the Control Stream use a Unified Control Frame (UCF) structure
with an explicit length prefix.
6.1.1. UCF Header
Every message on Stream 0 MUST begin with a 1-octet Frame Type and a
4-octet Length field.
+=========+==========+=================================+
| Field | Type | Description |
+=========+==========+=================================+
| Type | 1 octet | Frame Type identifier |
+---------+----------+---------------------------------+
| Length | 4 octets | Total length of the frame |
| | | payload (excluding Type/Length) |
+---------+----------+---------------------------------+
| Payload | variable | Frame-specific data |
+---------+----------+---------------------------------+
Table 5
This structure allows any implementation to skip an unknown frame
type by reading its length and discarding the payload.
For frame types with a fixed or computable payload layout defined by
this specification, the Length field MUST equal the length implied by
that layout (for STATUS frames, the base payload plus any cursor
update and extension; see Section 6.2.2). A receiver that detects a
Length value inconsistent with the frame's type-specific layout MUST
close the connection with PIPESTREAM_FRAME_ERROR (0x0D).
Rickert Expires 24 January 2027 [Page 24]
Internet-Draft PipeStream July 2026
Although the Length field can express values up to 2^32 - 1,
implementations SHOULD enforce a locally configured maximum control
frame size and MAY treat frames exceeding it as a connection error of
type PIPESTREAM_FRAME_ERROR (0x0D). See also the incremental
allocation requirement in Section 10.3.
For compactness, the fixed-frame diagrams below show the frame
payload that follows the common UCF header. On the wire, each Stream
0 frame is encoded as Type (1 octet) | Length (4 octets) | Payload.
6.1.2. Frame Types
The following frame types are defined by this document. All frames
use the common UCF header:
+======+==============+=======+=========================+
| Type | Name | Layer | Payload Format |
+======+==============+=======+=========================+
| 0x50 | STATUS | 0 | Fixed 16-octet payload, |
| | | | with optional cursor |
| | | | and extension payloads |
+------+--------------+-------+-------------------------+
| 0x54 | SCOPE_DIGEST | 1 | Fixed 72-octet payload |
+------+--------------+-------+-------------------------+
| 0x55 | BARRIER | 1 | Fixed 12-octet payload |
+------+--------------+-------+-------------------------+
| 0x56 | GOAWAY | 0 | Fixed 8-octet payload |
+------+--------------+-------+-------------------------+
| 0x80 | CAPABILITIES | 0 | Serialized message |
| | | | payload (negotiated |
| | | | format) |
+------+--------------+-------+-------------------------+
| 0x81 | CHECKPOINT | 0 | Serialized message |
| | | | payload (negotiated |
| | | | format) |
+------+--------------+-------+-------------------------+
Table 6
6.2. Status Frames (Layer 0)
6.2.1. Status Frame Format (0x50)
The Status Frame payload reports lifecycle transitions for entities.
The payload is 128-bit aligned for efficient parsing on 64-bit
architectures.
Rickert Expires 24 January 2027 [Page 25]
Internet-Draft PipeStream July 2026
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver(4) |Stat(4)|E|C|D(3) | Flags (19 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Entity ID (32 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope ID (32 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (32 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Ver (4 bits): Protocol version. MUST be set to 0x1 for this
specification. Receivers that encounter an unsupported version
MUST close the connection with PIPESTREAM_LAYER_UNSUPPORTED
(0x0C).
Stat (4 bits): Status code (see Section 6.2.2).
E (1 bit): Extended frame flag. If set, an Extension Header
(Section 6.6.1) MUST follow the base frame (and any cursor
update).
C (1 bit): Cursor update flag. A 4-octet cursor value follows
(Section 6.2.4).
D (3 bits): Explicit scope nesting depth (0-7). 0=Root. Layer 1.
Flags (19 bits): Reserved for future use. MUST be zero when sent
and MUST be ignored by receivers.
Entity ID (32 bits): Unsigned integer identifying the entity.
Scope ID (32 bits): Identifier for the scope to which this entity
belongs. Expanding to 32 bits ensures uniqueness across high-
frequency workloads.
Reserved (32 bits): Reserved for future use. MUST be zero when sent
and MUST be ignored by receivers.
6.2.2. Status Codes
+=======+=============+=======+============================+
| Value | Name | Layer | Description |
+=======+=============+=======+============================+
| 0x0 | UNSPECIFIED | - | Default / heartbeat signal |
+-------+-------------+-------+----------------------------+
| 0x1 | PENDING | 0 | Entity announced, not yet |
Rickert Expires 24 January 2027 [Page 26]
Internet-Draft PipeStream July 2026
| | | | transmitting |
+-------+-------------+-------+----------------------------+
| 0x2 | PROCESSING | 0 | Entity transmission in |
| | | | progress |
+-------+-------------+-------+----------------------------+
| 0x3 | COMPLETE | 0 | Entity successfully |
| | | | processed |
+-------+-------------+-------+----------------------------+
| 0x4 | FAILED | 0 | Entity processing failed |
+-------+-------------+-------+----------------------------+
| 0x5 | CHECKPOINT | 0 | Synchronization barrier |
+-------+-------------+-------+----------------------------+
| 0x6 | DEHYDRATING | 0 | Dehydrating into children |
+-------+-------------+-------+----------------------------+
| 0x7 | REHYDRATING | 0 | Rehydrating children |
+-------+-------------+-------+----------------------------+
| 0x8 | YIELDED | 2 | Paused with continuation |
| | | | token |
+-------+-------------+-------+----------------------------+
| 0x9 | DEFERRED | 2 | Detached with claim check |
+-------+-------------+-------+----------------------------+
| 0xA | RETRYING | 2 | Retry in progress |
+-------+-------------+-------+----------------------------+
| 0xB | SKIPPED | 2 | Intentionally skipped |
+-------+-------------+-------+----------------------------+
| 0xC | ABANDONED | 2 | Timed out |
+-------+-------------+-------+----------------------------+
Table 7
The base STATUS payload is 16 octets (21 octets on the wire including
the UCF header). When C=1, a 4-octet cursor value follows (20-octet
payload, 25 octets on the wire). When E=1, an Extension Header
follows all other STATUS fields.
6.2.3. Entity Status State Machine
The following table defines the complete set of valid status
transitions. A receiver that observes a transition not listed in
this table MUST treat the status frame as a protocol error and close
the connection with PIPESTREAM_ENTITY_INVALID (0x05) as the QUIC
Application Error Code.
Rickert Expires 24 January 2027 [Page 27]
Internet-Draft PipeStream July 2026
+=============+=====================================================+
| From State | Valid Transitions (To) |
+=============+=====================================================+
| PENDING | PROCESSING, DEHYDRATING, FAILED, |
| | SKIPPED, ABANDONED |
+-------------+-----------------------------------------------------+
| PROCESSING | COMPLETE, FAILED, DEHYDRATING, |
| | CHECKPOINT, YIELDED, DEFERRED, ABANDONED |
+-------------+-----------------------------------------------------+
| DEHYDRATING | REHYDRATING, FAILED, ABANDONED |
+-------------+-----------------------------------------------------+
| REHYDRATING | COMPLETE, FAILED, ABANDONED |
+-------------+-----------------------------------------------------+
| CHECKPOINT | PROCESSING |
+-------------+-----------------------------------------------------+
| YIELDED | PROCESSING, FAILED, DEFERRED, ABANDONED |
+-------------+-----------------------------------------------------+
| DEFERRED | PROCESSING, FAILED, SKIPPED, ABANDONED |
+-------------+-----------------------------------------------------+
| FAILED | RETRYING, ABANDONED |
+-------------+-----------------------------------------------------+
| RETRYING | PROCESSING, FAILED, ABANDONED |
+-------------+-----------------------------------------------------+
| COMPLETE | (terminal -- no transitions) |
+-------------+-----------------------------------------------------+
| SKIPPED | (terminal -- no transitions) |
+-------------+-----------------------------------------------------+
| ABANDONED | (terminal -- no transitions) |
+-------------+-----------------------------------------------------+
Table 8
Notes:
1. PENDING is the implicit initial state for every entity upon ID
assignment.
2. The FAILED -> RETRYING transition is valid only when the entity's
completion policy permits retries (max-retries > 0) and the retry
count has not been exhausted. If retries are not permitted or
are exhausted, FAILED MUST be treated as a terminal state.
3. Layer 2 states (YIELDED, DEFERRED, RETRYING, SKIPPED, ABANDONED)
MUST NOT appear when Layer 2 has not been negotiated. A receiver
operating at Layer 0 or Layer 1 that observes a Layer 2 status
code MUST treat it as PIPESTREAM_LAYER_UNSUPPORTED (0x0C).
Rickert Expires 24 January 2027 [Page 28]
Internet-Draft PipeStream July 2026
4. The UNSPECIFIED (0x0) status is used only for heartbeat frames
(Section 5.1.4) and connection-level signals. It is not a valid
entity lifecycle state and MUST NOT appear in transitions for
entity IDs other than 0xFFFFFFFF.
6.2.4. Cursor Update Extension
When C=1, a 4-octet cursor update follows the status frame:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| New Cursor Value (32 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
New Cursor Value (32 bits): The numeric value of the new cursor.
Entities with IDs lower than this value (modulo circular ID rules)
are considered resolved and their IDs MAY be recycled.
6.2.5. Status Reporting Direction
STATUS frames flow in both directions on Stream 0. For each entity,
responsibility for status reporting is divided between the
originating endpoint (the endpoint that assigned the Entity ID and
transmits the entity) and the processing endpoint (the endpoint that
receives the corresponding Entity Stream):
1. The originating endpoint MAY announce an entity with a PENDING
status frame before opening its Entity Stream.
2. Once the Entity Stream has been opened, the processing endpoint
is authoritative for the entity's lifecycle and emits the STATUS
frames for subsequent transitions (PROCESSING, DEHYDRATING,
REHYDRATING, COMPLETE, FAILED, and the Layer 2 statuses).
3. The originating endpoint applies terminal statuses (typically
FAILED or ABANDONED) on its own authority only as part of
transport error handling (Section 5.5) or local failure policy,
for example when the Entity Stream is reset or the connection is
lost before a terminal report arrives.
4. The originating endpoint advances its cursor based on observed
terminal statuses and announces cursor advancement using the C
flag (Section 6.2.4).
Rickert Expires 24 January 2027 [Page 29]
Internet-Draft PipeStream July 2026
If an endpoint receives a STATUS frame for an entity from a peer that
is not authoritative for that transition under these rules, the frame
MUST be validated against the state machine of Section 6.2.3;
conflicting reports are resolved in favor of the processing endpoint.
6.3. Scope Digest Frame (0x54)
When Protocol Layer 1 is negotiated, a scope completion is
summarized:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Flags (8) | Reserved (24) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope ID (32 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Entities Processed (64 bits) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Entities Succeeded (64 bits) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Entities Failed (64 bits) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Entities Deferred (64 bits) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Merkle Root (256 bits) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Flags (8 bits): Reserved for future use. MUST be zero when sent and
MUST be ignored by receivers.
Scope ID (32 bits): Identifier of the scope being summarized.
Entities Processed (64 bits): The total number of entities that were
processed within the scope.
Entities Succeeded (64 bits): The number of entities that reached a
terminal success state.
Rickert Expires 24 January 2027 [Page 30]
Internet-Draft PipeStream July 2026
Entities Failed (64 bits): The number of entities that reached a
terminal failure state.
Entities Deferred (64 bits): The number of entities that were
deferred via claim checks.
Merkle Root (256 bits): The SHA-256 Merkle root covering all entity
statuses in the scope (see Section 9.5).
The SCOPE_DIGEST payload is 72 octets (77 octets on the wire
including the UCF header). The Scope ID MUST match the 32-bit
identifier defined in Section 6.2.1.
6.4. Barrier Frame (0x55)
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| Reserved (31 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope ID (32 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Parent Entity ID (32 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
S (1 bit): Status (0 = waiting, 1 = released).
Reserved (31 bits): Reserved for future use. MUST be zero when sent
and MUST be ignored by receivers.
Scope ID (32 bits): Identifier for the scope to which this barrier
applies.
Parent Entity ID (32 bits): The identifier of the parent entity
whose sub-tree is blocked by this barrier.
The BARRIER payload is 12 octets (17 octets on the wire including the
UCF header).
6.5. GOAWAY Frame (0x56)
The GOAWAY frame signals that the sender will not accept new entities
beyond a specified Entity ID. It enables graceful shutdown: in-
flight entities with IDs at or below the Last Entity ID are processed
to completion, while the peer refrains from opening new Entity
Streams.
Rickert Expires 24 January 2027 [Page 31]
Internet-Draft PipeStream July 2026
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (32 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Last Entity ID (32 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Reserved (32 bits): Reserved for future use. MUST be zero when sent
and MUST be ignored by receivers.
Last Entity ID (32 bits): The highest Entity ID that the sender will
process. Entities with IDs greater than this value (per the
circular ordering defined in Section 9.1) MUST NOT be sent by the
peer after receiving this frame.
The GOAWAY payload is 8 octets (13 octets on the wire including the
UCF header).
6.5.1. Graceful Shutdown Procedure
1. An endpoint wishing to shut down sends a GOAWAY frame on Stream 0
with Last Entity ID set to the highest entity it is willing to
process.
2. Upon receiving GOAWAY, the peer MUST NOT open new Entity Streams
for entities with IDs above Last Entity ID. Entity Streams
already open for IDs at or below Last Entity ID continue to
completion.
3. Both peers continue processing status updates on Stream 0 until
all in-flight entities reach terminal state.
4. Once all entities are resolved, either peer MAY close the QUIC
connection with PIPESTREAM_NO_ERROR (0x00).
5. If an endpoint receives an entity with an ID above the Last
Entity ID after sending GOAWAY, it MUST reject it with
PIPESTREAM_ENTITY_INVALID (0x05).
An endpoint MAY send multiple GOAWAY frames to progressively lower
the Last Entity ID. The Last Entity ID MUST NOT increase across
successive GOAWAY frames within the same connection, where "increase"
is evaluated using the circular ordering function is_before defined
in Section 9.3.
Rickert Expires 24 January 2027 [Page 32]
Internet-Draft PipeStream July 2026
6.6. Yield and Claim Check Extensions (Layer 2)
When E=1 in a status frame, an Extension Header MUST follow.
6.6.1. Extension Header
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Extension Length (32 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Extension Length (32 bits): The total length of the extension data
that follows this header, in octets. This allows receivers that
do not support specific status extensions to skip the extension
data and continue parsing the control stream.
If E=1 is set for a Status code that does not define an extension
layout in this specification (or a negotiated extension), the
receiver MUST use the Extension Length to skip the data. If the
Extension Length is zero, extends beyond the end of the frame as
delimited by the UCF Length field, or is missing, the receiver MUST
treat the frame as malformed and close the connection with
PIPESTREAM_FRAME_ERROR (0x0D).
6.6.2. Yield Extension (Stat = 0x8)
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Yield Reason | Token Length (24 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Yield Token (variable) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Yield Reason (8 bits): The reason for yielding (see
Section 6.6.2.1).
Token Length (24 bits): The length of the Yield Token in bytes
(maximum 16,777,215).
Yield Token (variable): The opaque continuation state.
For a Yield Extension, the Extension Length (Section 6.6.1) MUST
equal 4 + Token Length. A mismatch MUST be treated as
PIPESTREAM_FRAME_ERROR (0x0D).
Rickert Expires 24 January 2027 [Page 33]
Internet-Draft PipeStream July 2026
6.6.2.1. Yield Reason Codes
+================+===================+==============================+
| Value | Name | Description |
+================+===================+==============================+
| 0x1 | EXTERNAL_CALL | Waiting on external service |
+----------------+-------------------+------------------------------+
| 0x2 | RATE_LIMITED | Voluntary throttle |
+----------------+-------------------+------------------------------+
| 0x3 | AWAITING_SIBLING | Waiting for specific sibling |
+----------------+-------------------+------------------------------+
| 0x4 | AWAITING_APPROVAL | Human/workflow gate |
+----------------+-------------------+------------------------------+
| 0x5 | RESOURCE_BUSY | Semaphore/lock |
+----------------+-------------------+------------------------------+
| 0x0, | Reserved | Reserved for future use |
| 0x06-0xFF | | |
+----------------+-------------------+------------------------------+
Table 9
6.6.3. Claim Check Extension (Stat = 0x9)
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Claim Check ID (64 bits) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Expiry Timestamp (64 bits, Unix micros) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Claim Check ID (64 bits): A cryptographically secure random
identifier for the claim.
Expiry Timestamp (64 bits): Unix epoch timestamp in microseconds
when the claim expires.
6.7. Serialized Message Frames (0x80-0xFF)
These frame types use the common UCF header defined in Section 6.1.
The payload body is encoded using the serialization format negotiated
during capability exchange (Section 3.4.2). If no format was
negotiated, CBOR [RFC8949] is the default.
Rickert Expires 24 January 2027 [Page 34]
Internet-Draft PipeStream July 2026
+======+==============+=============+
| Type | Message Name | Reference |
+======+==============+=============+
| 0x80 | Capabilities | Section 3.4 |
+------+--------------+-------------+
| 0x81 | Checkpoint | Section 9.3 |
+------+--------------+-------------+
Table 10
6.8. Entity Frames
Entity frames carry the actual entity payload data on Entity Streams.
6.8.1. Entity Frame Structure
+---------------------------+
| Header Length (4) | 4 octets, big-endian uint32
+---------------------------+
| |
| Header (serialized) | Variable length
| |
+---------------------------+
| |
| Payload | Variable length (per header)
| |
+---------------------------+
Header Length (4 octets): The length of the serialized EntityHeader
in bytes.
Header (serialized): The EntityHeader message encoded in the
negotiated serialization format (see Section 6.8.2).
Payload (variable): The raw entity data. The payload extends to the
end of the Entity Stream (QUIC FIN); see Section 5.2. When the
payload-length field is present in the EntityHeader, it MUST equal
the number of payload octets.
6.8.2. Message Schema (CDDL)
Normative definitions for serialized PipeStream messages use CDDL
[RFC8610] notation; Appendix C consolidates the complete schema.
6.8.2.1. Entity Header
Rickert Expires 24 January 2027 [Page 35]
Internet-Draft PipeStream July 2026
entity-header = {
entity-id: uint, ; 32-bit on wire (STATUS frame)
? parent-id: uint, ; 32-bit scope-local
? scope-id: uint, ; 32-bit (Section 6.2.1)
layer: uint .le 3, ; Data layer 0-3
? content-type: tstr, ; MIME type
? payload-length: uint, ; Octet count of the payload
; carried in this frame; SHOULD be
; present. Omitted only when the
; final length is unknown at
; header-emission time (the stream
; FIN delimits the payload).
? checksum: bstr .size 32, ; SHA-256; SHOULD be present
? metadata: { * tstr => tstr },
? chunk-info: chunk-info,
? completion-policy: completion-policy, ; Layer 2
}
6.8.2.2. Chunk Info
chunk-info = {
total-chunks: uint,
chunk-index: uint,
chunk-offset: uint,
}
6.8.2.3. Yield and Deferral
yield-token = {
reason: yield-reason, ; See Appendix C for enum values
? continuation-state: bstr,
? validation: stopping-point-validation,
}
claim-check = {
claim-id: uint,
entity-id: uint,
? scope-id: uint,
expiry-timestamp: uint, ; Unix epoch microseconds
? validation: stopping-point-validation,
}
6.8.2.4. Support Types
Rickert Expires 24 January 2027 [Page 36]
Internet-Draft PipeStream July 2026
; entity-status enumerates the status codes of Section 6.2.2;
; the full definition appears in Appendix C.
stopping-point-validation = {
? state-checksum: bstr, ; Hash of processing state
? bytes-processed: uint, ; Progress marker
? children-complete: uint,
? children-total: uint,
? is-resumable: bool,
? checkpoint-ref: tstr,
}
6.8.3. Checksum Algorithm
PipeStream uses SHA-256 [FIPS-180-4] for payload integrity
verification. The checksum MUST be exactly 32 octets.
6.8.4. Chunked Transfer
An entity whose payload is too large to transmit conveniently as a
single frame MAY be split into chunks. Each chunk is transmitted as
its own Entity Frame on its own Entity Stream, subject to the
following rules:
1. Every chunk of an entity MUST carry the same entity-id, parent-id
(if present), scope-id (if present), and layer values, and MUST
include a chunk-info structure.
2. The payload-length and checksum fields in a chunk's EntityHeader,
when present, cover only that chunk's payload octets (see
Section 10.2 for whole-entity integrity requirements).
3. chunk-offset is the octet offset of the chunk's first payload
octet within the complete entity payload. total-chunks MUST be
identical across all chunks of an entity, and the chunk-index
values MUST cover the range 0 to total-chunks - 1 with no
duplicates.
4. Chunks MAY be transmitted concurrently on separate Entity Streams
and MAY arrive in any order. The receiver reassembles the
payload by chunk-offset.
5. The entity payload is complete when all total-chunks chunks have
been received and the reassembled ranges are contiguous and non-
overlapping. Overlapping or duplicated ranges MUST be treated as
PIPESTREAM_ENTITY_INVALID (0x05).
Rickert Expires 24 January 2027 [Page 37]
Internet-Draft PipeStream July 2026
6. Lifecycle status for a chunked entity is reported once for the
entity as a whole (Section 6.2), not per chunk.
7. Entity Payload Model
This section describes the payload model visible to the core
protocol. PipeStream distinguishes between the wire-level Entity
Header carried on Entity Streams and any application-level envelope
that may be embedded within the payload bytes.
7.1. Entity Header
The normative wire-level Entity Header is defined in Section 6.8.2.
It provides transport-level identification and routing metadata,
including entity-id, parent-id, scope-id, layer, payload-length, and
optional integrity metadata. The core protocol interprets these
fields for stream processing, flow control, and recursive
coordination, but it does not define application-specific payload
structure.
7.2. Data Layers
PipeStream defines a 2-bit Data Layer field (values 0-3) in the
Entity Header that identifies the payload encoding class. The
semantics of each layer value are defined by Application Profile
specifications. The core protocol does not assign meaning to
specific layer values; it ensures that the layer field is faithfully
transmitted and that receivers can route entities to appropriate
handlers based on layer.
Application Profiles SHOULD define at most four data layers. Common
patterns include:
Rickert Expires 24 January 2027 [Page 38]
Internet-Draft PipeStream July 2026
+=======+===============================+======================+
| Layer | Conventional Meaning | Purpose |
+=======+===============================+======================+
| 0 | Raw or binary payload | Unprocessed input |
+-------+-------------------------------+----------------------+
| 1 | Annotated or enriched payload | Intermediate |
| | | processing artifacts |
+-------+-------------------------------+----------------------+
| 2 | Structured or normalized | Final extracted or |
| | payload | transformed output |
+-------+-------------------------------+----------------------+
| 3 | Application-specific | Domain extension |
| | extension | point |
+-------+-------------------------------+----------------------+
Table 11
This layering enables pipeline stages to progressively transform
entities from raw input to structured output while maintaining type
distinction through the layer field.
7.3. Application-Level Envelope
The Entity Header (Section 6.8.2) provides transport-level
identification (entity-id, scope-id, and layer). Applications that
require additional identification, such as a stable input identifier
that persists across pipeline runs, multi-tenancy context, or domain-
specific metadata, SHOULD define an application-level envelope
message carried within the entity payload.
If an application-level envelope carries its own entity identifier,
that identifier MUST match the entity-id in the enclosing Entity
Header. The definition of application-level envelopes is outside the
scope of this specification. See [PIPESTREAM-DOCPROC] for an example
companion profile that defines such an envelope for document-
processing workloads.
7.4. Common Storage References
PipeStream defines a common storage reference type for entities that
refer to externally stored data. Application Profiles are not
required to use this type but SHOULD prefer it over ad hoc storage
reference formats to improve interoperability between pipeline stages
from different implementations.
Rickert Expires 24 January 2027 [Page 39]
Internet-Draft PipeStream July 2026
file-storage-reference = {
provider: tstr, ; Storage provider identifier
bucket: tstr, ; Bucket/container name
key: tstr, ; Object key/path
? region: tstr, ; Optional region hint
? attrs: { * tstr => tstr }, ; Provider-specific attributes
? encryption: encryption-metadata,
}
encryption-metadata = {
algorithm: tstr, ; "AES-256-GCM", "AES-256-CBC"
? key-provider: tstr, ; "aws-kms", "azure-keyvault",
; "gcp-kms", "vault"
? key-id: tstr, ; Key ARN/URI/ID
? wrapped-key: bstr, ; Client-side encrypted DEK
? iv: bstr, ; Initialization vector
? context: { * tstr => tstr }, ; Encryption context
}
8. Protocol Operations
This section defines the protocol-level operations that PipeStream
endpoints perform during a session. These operations describe the
phases of a PipeStream session lifecycle, from connection
establishment through entity processing to terminal consumption.
8.1. Overview
A PipeStream session proceeds through four sequential actions:
Rickert Expires 24 January 2027 [Page 40]
Internet-Draft PipeStream July 2026
+---------------------------------------------+
| PipeStream Action Flow |
+---------------------------------------------+
|
v
+---------------------------------------------+
| CONNECT |
| (Session + Capability Negotiation) |
+---------------------------------------------+
|
v
+---------------------------------------------+
| PARSE |
| (Dehydration: 1:N possible) |
+---------------------------------------------+
|
+-------------+-------------+
v v v
+-----------+ +-----------+ +-----------+
| PROCESS | | PROCESS | | PROCESS |
| (1:1) | | (1:1) | | (N:1) |
+-----------+ +-----------+ +-----------+
| | |
+-------------+-------------+
|
v
+---------------------------------------------+
| SINK |
| (Terminal Consumption) |
+---------------------------------------------+
Rickert Expires 24 January 2027 [Page 41]
Internet-Draft PipeStream July 2026
+=======+=========+=============+===============================+
| Phase | Action | Cardinality | Description |
+=======+=========+=============+===============================+
| 1 | CONNECT | 1:1 | Session establishment and |
| | | | capability negotiation |
+-------+---------+-------------+-------------------------------+
| 2 | PARSE | 1:N | Decomposition: dehydrate a |
| | | | root entity into sub-entities |
+-------+---------+-------------+-------------------------------+
| 3 | PROCESS | 1:1 or N:1 | Transform, rehydrate, |
| | | | aggregate, or pass through |
| | | | entities (parallel) |
+-------+---------+-------------+-------------------------------+
| 4 | SINK | N:1 | Terminal consumption by an |
| | | | application-defined sink |
+-------+---------+-------------+-------------------------------+
Table 12
8.2. CONNECT Action
The CONNECT action establishes the session with capability
negotiation.
8.2.1. ALPN Identifier
ALPN Protocol ID: pipestream/1
8.2.2. Capability Exchange
Immediately after QUIC handshake, peers exchange Capabilities
messages on Stream 0.
The Capabilities exchange includes serialization format negotiation
(Section 3.4.2). The agreed-upon format applies to all subsequent
variable-length serialized messages on Stream 0 and to all entity
headers on Entity Streams.
8.3. PARSE Action
The PARSE action performs decomposition by dehydrating an input
entity into one or more sub-entities. When Layer 2 is negotiated,
the sender MAY attach a completion policy that governs how partial
success, timeouts, or retries are handled during recursive
processing.
Rickert Expires 24 January 2027 [Page 42]
Internet-Draft PipeStream July 2026
completion-policy = {
? mode: completion-mode,
? max-retries: uint, ; Default: 3
? retry-delay-ms: uint, ; Default: 1000
? timeout-ms: uint, ; Default: 300000 (5 min)
? min-success-ratio: float32, ; For QUORUM mode
? on-timeout: failure-action,
? on-failure: failure-action,
}
completion-mode = &(
unspecified: 0, ; Default; treat as STRICT
strict: 1, ; All children MUST complete
lenient: 2, ; Continue with partial results
best-effort: 3, ; Complete with whatever succeeds
quorum: 4, ; Need min-success-ratio
)
failure-action = &(
unspecified: 0, ; Default; treat as FAIL
fail: 1, ; Propagate failure up
skip: 2, ; Skip, continue with siblings
retry: 3, ; Retry up to max-retries
defer: 4, ; Create claim check, continue
)
8.4. PROCESS Action
+=============+========================================+
| Mode | Description |
+=============+========================================+
| TRANSFORM | 1:1 entity transformation |
+-------------+----------------------------------------+
| REHYDRATE | N:1 merge of siblings from dehydration |
+-------------+----------------------------------------+
| AGGREGATE | N:1 with reduction function |
+-------------+----------------------------------------+
| PASSTHROUGH | Metadata-only modification |
+-------------+----------------------------------------+
Table 13
Rickert Expires 24 January 2027 [Page 43]
Internet-Draft PipeStream July 2026
8.5. SINK Action
The SINK action represents terminal consumption of processed
entities. Sink implementations are application-specific and are
defined by Application Profile specifications. Common sink patterns
include persistent storage, search engine indexing, event
notification, and downstream service delivery.
9. Rehydration Semantics
9.1. Entity ID Lifecycle and Cursor
Entity IDs are managed using a cursor-based circular recycling scheme
within the 32-bit ID space. All circular arithmetic in this document
is performed modulo 0xFFFFFFFD; this modulus excludes the reserved
values at the top of the 32-bit space from circulation. The
following Entity ID values are reserved and MUST NOT be assigned to
entities:
+=======================+==================+===================+
| Entity ID | Name | Purpose |
+=======================+==================+===================+
| 0x00000000 | NULL_ENTITY | Reserved; skipped |
| | | during assignment |
+-----------------------+------------------+-------------------+
| 0xFFFFFFFD-0xFFFFFFFE | Reserved | Outside the |
| | | circular ID space |
+-----------------------+------------------+-------------------+
| 0xFFFFFFFF | CONNECTION_LEVEL | Connection-scoped |
| | | signals such as |
| | | heartbeats |
| | | (Section 5.1.4) |
+-----------------------+------------------+-------------------+
Table 14
Assignable Entity IDs therefore range from 0x00000001 to 0xFFFFFFFC,
yielding 4,294,967,292 usable identifiers per scope.
The ID space is divided into three logical regions relative to the
current cursor and last_assigned pointers:
Rickert Expires 24 January 2027 [Page 44]
Internet-Draft PipeStream July 2026
+============+===================+===========================+
| Region | ID Range | Description |
+============+===================+===========================+
| Recyclable | IDs behind cursor | Resolved entities; IDs |
| | | may be reused |
+------------+-------------------+---------------------------+
| In-flight | cursor to | Active entities (PENDING, |
| | last_assigned | PROCESSING, etc.) |
+------------+-------------------+---------------------------+
| Free | Beyond | Available for new entity |
| | last_assigned | assignment |
+------------+-------------------+---------------------------+
Table 15
The window size is computed as (last_assigned - cursor) mod
0xFFFFFFFD. If window_size >= max_window, the sender MUST apply
backpressure and stop assigning new IDs until the cursor advances.
The max_window value MUST NOT exceed 2,147,483,646 (the largest value
strictly less than 0xFFFFFFFD / 2). This bound guarantees that the
circular comparison function is_before (Section 9.3) is unambiguous
for any pair of in-flight Entity IDs. The default max-window-size
advertised in the capabilities exchange (Section 3.4) is
2,147,483,646.
*Rules:*
1. new_id = (last_assigned + 1) % 0xFFFFFFFD
2. If new_id == 0, new_id = 1 (skip reserved NULL_ENTITY)
3. If (new_id - cursor) % 0xFFFFFFFD >= max_window -> STOP, apply
backpressure
4. On reaching a terminal state (COMPLETE, SKIPPED, ABANDONED, or
FAILED with no remaining retries): mark resolved; if entity_id ==
cursor, advance cursor past all contiguous resolved IDs
5. IDs behind cursor are implicitly recyclable
An entity in the FAILED state that may still transition to RETRYING
(see the state transition table in Section 6.2) MUST NOT be marked
resolved. Only when retries are exhausted or no completion policy
permits retries does FAILED become terminal for cursor purposes.
Rickert Expires 24 January 2027 [Page 45]
Internet-Draft PipeStream July 2026
9.2. Assembly Manifest
The Assembly Manifest is a local data structure maintained by each
endpoint to track the parent-child relationships created during
dehydration. It is not transmitted on the wire; rather, each
endpoint constructs its own manifest from the parent-id fields in
received EntityHeaders and from status updates observed on the
Control Stream. The CDDL below defines the logical structure of each
entry; implementations MAY use any internal representation that
preserves the required semantics.
Each Assembly Manifest entry tracks:
assembly-manifest-entry = {
parent-id: uint,
? scope-id: uint, ; Layer 1
children-ids: [* uint],
? children-status: [* entity-status],
? policy: completion-policy, ; Layer 2
? created-at: uint,
? state: resolution-state,
}
resolution-state = &(
unspecified: 0,
active: 1,
resolved: 2,
partial: 3, ; Some children failed/skipped
failed: 4,
)
9.3. Checkpoint Blocking
A checkpoint is satisfied when:
1. All entities in the checkpoint scope with IDs less than
checkpoint_entity_id (considering circular wrap) have reached
terminal state.
2. All Assembly Manifest entries within the checkpoint scope have
been resolved.
3. All nested checkpoints within the checkpoint scope have been
satisfied.
CheckpointFrame (Section 6.7 / Appendix C) carries both:
Rickert Expires 24 January 2027 [Page 46]
Internet-Draft PipeStream July 2026
checkpoint-frame = {
checkpoint-id: tstr,
sequence-number: uint,
checkpoint-entity-id: uint,
? scope-id: uint,
? flags: uint,
? timeout-ms: uint,
}
* checkpoint_id: an opaque identifier for logging and correlation.
* checkpoint_entity_id: the numeric ordering key used for barrier
evaluation.
Implementations MUST use checkpoint_entity_id (not checkpoint_id)
when evaluating Condition 1.
For circular comparison in Condition 1, implementations MUST use the
same modulo ordering as cursor management. Define MAX = 0xFFFFFFFD
and:
is_before(a, b) = ((b - a + MAX) % MAX) < (MAX / 2)
An entity ID a is considered "less than checkpoint_entity_id b" iff
is_before(a, b) is true.
9.4. Scope ID Allocation (Layer 1)
When Layer 1 is negotiated, Scope IDs are 32-bit unsigned integers
assigned by the endpoint that initiates the dehydration. The
allocation scheme is as follows:
1. Scope ID 0 is the root scope and MUST NOT be used for child
scopes.
2. The dehydrating endpoint assigns a unique Scope ID to each new
child scope created during dehydration. The Scope ID MUST be
unique within the connection for the lifetime of that scope
(i.e., until the scope's SCOPE_DIGEST frame has been sent and all
entities in the scope have reached terminal state).
3. Scope IDs MAY be allocated sequentially or randomly; the protocol
does not require any particular ordering. Sequential allocation
is RECOMMENDED for simplicity and debuggability.
Rickert Expires 24 January 2027 [Page 47]
Internet-Draft PipeStream July 2026
4. Once a scope has been closed (its SCOPE_DIGEST has been sent and
all entities within the scope have reached terminal state), the
Scope ID MAY be reused for a new scope. Implementations MUST
ensure that no in-flight status frames reference a recycled Scope
ID before reuse.
9.5. Scope Digest Propagation (Layer 1)
When a scope completes, the endpoint MUST compute a Scope Digest and
propagate it to the parent scope via a SCOPE_DIGEST frame
(Section 6.3).
The Merkle root in the Scope Digest is computed as follows:
1. For each entity in the scope, ordered by ascending numeric Entity
ID value, construct a 5-octet leaf value by concatenating:
* The 4-octet big-endian Entity ID.
* A 1-octet status field where the lower 4 bits contain the
entity's terminal Stat code (Section 6.2.2) and the upper 4
bits are zero.
2. Compute each leaf hash as SHA-256(0x00 || leaf), where leaf is
the 5-octet value from step 1.
3. Build a binary Merkle tree by repeatedly hashing pairs of sibling
nodes: SHA-256(0x01 || left || right). If the number of nodes at
any level is odd, the last node is promoted to the next level
without hashing.
4. The root of this tree is the merkle_root value in the
SCOPE_DIGEST frame.
The 0x00 and 0x01 domain-separation prefixes distinguish leaf hashes
from interior-node hashes, preventing second-preimage attacks in
which an interior node is presented as a leaf (or vice versa). This
construction follows the approach used for Merkle Tree Hashes in
Certificate Transparency.
This construction is deterministic: any two implementations
processing the same set of entity statuses MUST produce the same
Merkle root.
Because each Entity ID contributes exactly one leaf, an
implementation MUST NOT recycle an Entity ID within a scope whose
SCOPE_DIGEST has not yet been computed. Cursor-based recycling
(Section 9.1) already guarantees this when scopes complete before the
Rickert Expires 24 January 2027 [Page 48]
Internet-Draft PipeStream July 2026
ID space wraps; implementations whose scopes approach the ID-space
capacity MUST close and digest the scope before reusing any of its
Entity IDs.
9.6. Rehydration Readiness Tracking
Implementations MUST track Assembly Manifest resolution order using a
local data structure that can efficiently identify the next parent
entity eligible for rehydration as child statuses arrive out of
order.
The specific algorithm and internal representation are implementation
choices and are outside the scope of this specification.
9.7. Stopping Point Validation (Layer 2)
When yielding or deferring, include validation:
stopping-point-validation = {
? state-checksum: bstr, ; Hash of processing state
? bytes-processed: uint, ; Progress marker
? children-complete: uint,
? children-total: uint,
? is-resumable: bool,
? checkpoint-ref: tstr,
}
10. Security Considerations
10.1. Transport Security
PipeStream inherits security from QUIC [RFC9000] and TLS 1.3
[RFC8446]. All connections MUST use TLS 1.3 or later.
Implementations MUST NOT provide mechanisms to disable encryption.
PipeStream frames MUST NOT be sent or processed in 0-RTT early data
(Section 5.3), which removes the replay exposure that early data
would otherwise introduce for capability negotiation and status
frames.
10.2. Entity Payload Integrity
Each Entity SHOULD include a SHA-256 [FIPS-180-4] checksum in its
EntityHeader (the checksum field defined in Section 6.8.2). The
checksum is OPTIONAL in the wire format to accommodate zero-length
entities, streamed entities whose final length is unknown at header-
emission time, and scenarios where application-layer integrity
mechanisms provide equivalent guarantees. When a checksum is
Rickert Expires 24 January 2027 [Page 49]
Internet-Draft PipeStream July 2026
present, it MUST be exactly 32 octets containing the SHA-256 digest
computed over the raw payload bytes (the octet sequence following the
EntityHeader on the Entity Stream). The checksum does not cover the
EntityHeader itself.
For chunked entities (where chunk-info is present in the
EntityHeader), each chunk MAY carry its own per-chunk checksum. The
checksum in the first chunk's EntityHeader, if present, MUST cover
only that chunk's payload bytes. An implementation that requires
whole-entity integrity verification MUST either compute a rolling
digest across all chunks or require the sender to transmit a final
summary entity containing the whole-payload checksum.
To support true streaming of large entities, implementations MAY
begin processing an entity payload before the complete payload has
been received and verified. However, the final rehydration or
terminal SINK operation MUST NOT be committed until the complete
payload checksum has been verified.
If a checksum verification fails, the implementation MUST:
1. Reject the entity with PIPESTREAM_INTEGRITY_ERROR (0x04).
2. Discard any partial results or temporary state associated with
the entity.
3. Propagate the failure according to the Completion Policy
(Section 8.3).
Implementations that require immediate consistency SHOULD buffer the
entire entity and verify the checksum before initiating processing.
10.2.1. Algorithm Agility
This specification mandates SHA-256 [FIPS-180-4] as the sole checksum
algorithm for both payload integrity (this section) and Merkle tree
construction (Section 9.5). SHA-256 is well-studied and widely
deployed; however, future developments may necessitate migration to a
different algorithm.
Rickert Expires 24 January 2027 [Page 50]
Internet-Draft PipeStream July 2026
PipeStream supports algorithm migration through the capability
negotiation mechanism (Section 3.4). A future specification MAY
define additional fields in the capabilities structure to advertise
supported checksum algorithms, following the general principles
outlined in [RFC7696]. Until such negotiation is defined, all
implementations MUST use SHA-256 when producing or verifying
checksums. An implementation that receives a checksum of a length
other than 32 octets MUST reject the entity with
PIPESTREAM_INTEGRITY_ERROR (0x04).
The checksum field in the EntityHeader is typed as bstr .size 32 in
the CDDL schema (Appendix C). A future algorithm negotiation
extension would need to update this constraint, the SCOPE_DIGEST
Merkle root size, and the corresponding IANA registry entries.
10.3. Resource Exhaustion
+=================+===============+========================+
| Limit | Default | Description |
+=================+===============+========================+
| Max scope depth | 7 | Prevents recursive |
| | | bombs (8 levels: 0-7) |
+-----------------+---------------+------------------------+
| Max entities | 4,294,967,292 | Memory bounds |
| per scope | | (Section 9.1) |
+-----------------+---------------+------------------------+
| Max window size | 2,147,483,646 | Max in-flight entities |
| | | (Section 9.1) |
+-----------------+---------------+------------------------+
| Checkpoint | 30s | Prevents stuck state |
| timeout | | |
+-----------------+---------------+------------------------+
| Claim check | 86400s | Garbage collection |
| expiry | | |
+-----------------+---------------+------------------------+
Table 16
Implementations MUST enforce all resource limits listed above.
Exceeding any limit MUST result in the corresponding error code (see
Section 11.4). Implementations SHOULD allow operators to configure
stricter limits than the defaults shown here.
Rickert Expires 24 January 2027 [Page 51]
Internet-Draft PipeStream July 2026
To prevent memory-exhaustion attacks, implementations MUST NOT pre-
allocate memory for variable-length payloads based solely on the
32-bit Length field in the UCF header (Section 6.1.1). Memory MUST
be allocated incrementally as octets are received, or capped at a
smaller initial buffer until the message type and context are
verified.
10.4. Amplification Attacks
A single dehydration operation can produce an arbitrary number of
child entities from a small input, creating a potential amplification
vector. To mitigate this:
1. Implementations MUST enforce the max_entities_per_scope limit
negotiated during capability exchange (Section 3.4). Any
dehydration that would exceed this limit MUST be rejected.
2. Implementations MUST enforce the max_scope_depth limit. A
dehydration chain deeper than this limit MUST be rejected with
PIPESTREAM_DEPTH_EXCEEDED (0x07).
3. Implementations SHOULD enforce a configurable ratio between input
entity size and total child entity count. A recommended default
is no more than 1,000 children per megabyte of parent payload.
4. The backpressure mechanism (Section 9.1) provides a natural
throttle: when the in-flight window fills, no new Entity IDs can
be assigned until existing entities complete and the cursor
advances. Implementations MUST NOT bypass backpressure for
dehydration-generated entities.
10.5. Privacy Considerations
PipeStream entity headers and control stream frames carry metadata
that may reveal information about the entities being processed, even
when payloads are encrypted at the application layer:
1. *Entity structure leakage*: The number of child entities produced
by dehydration, the scope depth, and the Entity ID assignment
pattern may reveal the structure of the input being processed
(e.g., an entity that dehydrates into 50 children is likely a
multi-part input). Implementations that require structural
privacy SHOULD pad dehydration counts or use fixed decomposition
granularity. Deployments that do not handle privacy-sensitive
data MAY omit this padding.
Rickert Expires 24 January 2027 [Page 52]
Internet-Draft PipeStream July 2026
2. *Metadata in headers*: The content_type, metadata map, and
payload_length fields in EntityHeader (Section 6.8.2) are
transmitted in cleartext within the QUIC-encrypted stream.
Implementations that require metadata confidentiality beyond
transport encryption SHOULD encrypt EntityHeader fields at the
application layer and use an opaque content_type such as
application/octet-stream. This overhead is unnecessary when the
deployment operates within a trusted network.
3. *Traffic analysis*: The timing and size of status frames on the
Control Stream may correlate with processing patterns.
Implementations operating in privacy-sensitive environments
SHOULD send status frames at fixed intervals with padding to
obscure processing timing. Deployments in trusted environments
MAY omit traffic padding to reduce bandwidth overhead.
4. *Identifiers*: Application-level input identifiers and filenames
carried inside profile-defined payload envelopes are not
interpreted by PipeStream Core but may still be logged by
intermediate processing nodes. Implementations SHOULD provide
mechanisms to redact or pseudonymize such identifiers at pipeline
boundaries. This recommendation may be relaxed when all nodes in
the pipeline are operated by the same administrative entity.
10.6. Replay and Token Reuse
10.6.1. Yield Token Replay
Yield tokens (Section 6.6.2) contain opaque continuation state that
enables resumption of paused entity processing. A replayed yield
token could cause an entity to be processed multiple times or to
resume from a stale state. To prevent this:
1. Implementations MUST associate each yield token with a stable
application context identifier (for example, a session
identifier) and Entity ID. In Layer 0-only operation, this
context MAY be implicit in the active transport connection. For
Layer 2 resumptions that can occur across reconnects or different
nodes, the context identifier MUST remain stable across transport
connections. A yield token MUST be rejected if presented in a
different context than the one that issued it, unless the token
was explicitly transferred via a claim check.
2. Implementations MUST invalidate a yield token after it has been
consumed for resumption. A second resumption attempt with the
same token MUST be rejected.
Rickert Expires 24 January 2027 [Page 53]
Internet-Draft PipeStream July 2026
3. The StoppingPointValidation (Section 9.7) provides integrity
checking at resume time. Implementations MUST verify the
state_checksum field before accepting a resumed entity. If the
checksum does not match the current state, the resumption MUST be
rejected and the entity MUST be reprocessed from the beginning.
10.6.2. Claim Check Replay
Claim checks (Section 6.6.3) are long-lived references that can be
redeemed in different sessions. To prevent misuse:
1. Each claim check carries an expiry_timestamp (Unix epoch
microseconds). Implementations MUST reject expired claim checks.
2. Implementations MUST track redeemed claim check IDs and reject
duplicate redemptions. The tracking state MUST persist for at
least the claim check expiry duration.
3. Claim check IDs MUST be generated using a cryptographically
secure random number generator to prevent guessing.
4. Because the claim check identifier space is 64 bits, an online
attacker with sufficient query volume could attempt to enumerate
valid identifiers. Implementations SHOULD rate-limit claim
redemption attempts per peer and SHOULD treat repeated redemption
failures as a signal of probing.
10.7. Encryption Key Management
When using FileStorageReference with encryption:
1. Key IDs MUST reference keys in approved providers.
2. Wrapped keys MUST use approved envelope encryption.
3. Key rotation MUST be supported via key_id versioning.
4. Implementations MUST NOT log key material.
5. Implementations MUST NOT include unwrapped data encryption keys
in EntityHeader metadata or Control Stream frames.
Rickert Expires 24 January 2027 [Page 54]
Internet-Draft PipeStream July 2026
11. IANA Considerations
This document requests the creation of several new registries and one
ALPN identifier registration. All registries defined in this section
use the "Expert Review" policy [RFC8126] for new assignments unless
otherwise stated; guidance for the designated experts appears in
Section 11.7.
11.1. ALPN Identifier Registration
This document registers the following ALPN [RFC7301] protocol
identifier:
Protocol: PipeStream Version 1
Identification Sequence: 0x70 0x69 0x70 0x65 0x73 0x74 0x72 0x65
0x61 0x6D 0x2F 0x31 ("pipestream/1")
Specification: This document
11.2. PipeStream Frame Type Registry
IANA is requested to create the "PipeStream Frame Types" registry.
All frames on Stream 0 MUST use a 4-octet length prefix following the
1-octet Type. The registry is partitioned into three ranges that
determine both the registration policy and the framing class of the
payload:
+===========+=================================+=====================+
| Range | Framing Class | Registration |
| | | Policy |
+===========+=================================+=====================+
| 0x00-0x7F | Fixed bit-packed payload | Expert |
| | (Section 6) | Review |
+-----------+---------------------------------+---------------------+
| 0x80-0xBF | Serialized payload in the | Expert |
| | negotiated format (Section 6.7) | Review |
+-----------+---------------------------------+---------------------+
| 0xC0-0xFF | Private use | Not |
| | | applicable |
+-----------+---------------------------------+---------------------+
Table 17
Each registration consists of a value, a frame type name, the minimum
protocol layer, a short description, and a reference to a
specification. The initial contents are:
Rickert Expires 24 January 2027 [Page 55]
Internet-Draft PipeStream July 2026
+=======+=================+=======+===================+===========+
| Value | Frame Type Name | Layer | Description | Reference |
+=======+=================+=======+===================+===========+
| 0x50 | STATUS | 0 | Entity lifecycle | Section |
| | | | status | 6.2 |
+-------+-----------------+-------+-------------------+-----------+
| 0x54 | SCOPE_DIGEST | 1 | Merkle completion | Section |
| | | | summary | 6.3 |
+-------+-----------------+-------+-------------------+-----------+
| 0x55 | BARRIER | 1 | Subtree | Section |
| | | | synchronization | 6.4 |
+-------+-----------------+-------+-------------------+-----------+
| 0x56 | GOAWAY | 0 | Graceful shutdown | Section |
| | | | signal | 6.5 |
+-------+-----------------+-------+-------------------+-----------+
| 0x80 | CAPABILITIES | 0 | Negotiated | Section |
| | | | limits/layers | 3.4 |
+-------+-----------------+-------+-------------------+-----------+
| 0x81 | CHECKPOINT | 0 | Global | Section |
| | | | synchronization | 9.3 |
+-------+-----------------+-------+-------------------+-----------+
Table 18
11.2.1. Unknown Frame Handling
Receivers that encounter a frame type that they do not recognize MUST
skip the frame by reading and discarding the number of octets
indicated by the 4-octet length prefix. This mechanism ensures that
future protocol extensions can be introduced without breaking
backward compatibility for older implementations.
11.3. PipeStream Status Code Registry
IANA is requested to create the "PipeStream Status Codes" registry.
Status codes are 4-bit values (0x0-0xF). Values 0xD-0xE are reserved
for Expert Review. Value 0xF is reserved for private use.
Rickert Expires 24 January 2027 [Page 56]
Internet-Draft PipeStream July 2026
+=======+=============+=======+===========================+
| Value | Name | Layer | Description |
+=======+=============+=======+===========================+
| 0x0 | UNSPECIFIED | - | Default / heartbeat |
+-------+-------------+-------+---------------------------+
| 0x1 | PENDING | 0 | Entity announced |
+-------+-------------+-------+---------------------------+
| 0x2 | PROCESSING | 0 | In progress |
+-------+-------------+-------+---------------------------+
| 0x3 | COMPLETE | 0 | Success |
+-------+-------------+-------+---------------------------+
| 0x4 | FAILED | 0 | Failed |
+-------+-------------+-------+---------------------------+
| 0x5 | CHECKPOINT | 0 | Barrier |
+-------+-------------+-------+---------------------------+
| 0x6 | DEHYDRATING | 0 | Dehydrating into children |
+-------+-------------+-------+---------------------------+
| 0x7 | REHYDRATING | 0 | Rehydrating children |
+-------+-------------+-------+---------------------------+
| 0x8 | YIELDED | 2 | Paused |
+-------+-------------+-------+---------------------------+
| 0x9 | DEFERRED | 2 | Claim check issued |
+-------+-------------+-------+---------------------------+
| 0xA | RETRYING | 2 | Retry in progress |
+-------+-------------+-------+---------------------------+
| 0xB | SKIPPED | 2 | Intentionally skipped |
+-------+-------------+-------+---------------------------+
| 0xC | ABANDONED | 2 | Timed out |
+-------+-------------+-------+---------------------------+
Table 19
11.4. PipeStream Error Code Registry
IANA is requested to create the "PipeStream Error Codes" registry.
Values in the range 0x00-0x3F are assigned by Expert Review. Values
in the range 0x40-0xFF are reserved for private use.
PipeStream error codes are used as QUIC application error codes in
CONNECTION_CLOSE and RESET_STREAM frames. When terminating a
connection or aborting a stream due to a protocol-level error, the
endpoint MUST use the corresponding PipeStream error code value as
the QUIC Application Error Code.
Rickert Expires 24 January 2027 [Page 57]
Internet-Draft PipeStream July 2026
+=======+==============================+===========================+
| Value | Name | Description |
+=======+==============================+===========================+
| 0x00 | PIPESTREAM_NO_ERROR | Graceful shutdown |
+-------+------------------------------+---------------------------+
| 0x01 | PIPESTREAM_INTERNAL_ERROR | Implementation error |
+-------+------------------------------+---------------------------+
| 0x02 | PIPESTREAM_IDLE_TIMEOUT | Idle timeout |
+-------+------------------------------+---------------------------+
| 0x03 | PIPESTREAM_CONTROL_RESET | Control stream must reset |
+-------+------------------------------+---------------------------+
| 0x04 | PIPESTREAM_INTEGRITY_ERROR | Checksum failed |
+-------+------------------------------+---------------------------+
| 0x05 | PIPESTREAM_ENTITY_INVALID | Invalid format or state |
+-------+------------------------------+---------------------------+
| 0x06 | PIPESTREAM_ENTITY_TOO_LARGE | Size exceeded |
+-------+------------------------------+---------------------------+
| 0x07 | PIPESTREAM_DEPTH_EXCEEDED | Scope depth exceeded |
+-------+------------------------------+---------------------------+
| 0x08 | PIPESTREAM_WINDOW_EXCEEDED | Window full |
+-------+------------------------------+---------------------------+
| 0x09 | PIPESTREAM_SCOPE_INVALID | Invalid scope |
+-------+------------------------------+---------------------------+
| 0x0A | PIPESTREAM_CLAIM_EXPIRED | Claim check expired |
+-------+------------------------------+---------------------------+
| 0x0B | PIPESTREAM_CLAIM_NOT_FOUND | Claim check not found |
+-------+------------------------------+---------------------------+
| 0x0C | PIPESTREAM_LAYER_UNSUPPORTED | Protocol layer not |
| | | supported |
+-------+------------------------------+---------------------------+
| 0x0D | PIPESTREAM_FRAME_ERROR | Malformed frame or |
| | | improper stream usage |
+-------+------------------------------+---------------------------+
Table 20
11.5. PipeStream Serialization Format Registry
IANA is requested to create the "PipeStream Serialization Formats"
registry. New entries require Expert Review [RFC8126]. A
registration request MUST include a reference to a specification that
normatively defines the encoding of every serialized PipeStream
message (Section 3.4.2).
Rickert Expires 24 January 2027 [Page 58]
Internet-Draft PipeStream July 2026
+=======+============+=======================+===============+
| Value | Name | Description | Reference |
+=======+============+=======================+===============+
| 0 | CBOR | Concise Binary Object | RFC 8949, |
| | | Representation | this document |
+-------+------------+-----------------------+---------------+
| 1-255 | Unassigned | Available via Expert | |
| | | Review | |
+-------+------------+-----------------------+---------------+
Table 21
11.6. URI Scheme Registration
This section registers the "pipestream" URI scheme per [RFC7595].
The URI scheme identifies application context for detached or
resumable resources (for example, Layer 2 yield/claim-check flows).
PipeStream Layer 0 streaming semantics do not depend on this URI
scheme.
Scheme name: pipestream
Status: Permanent
Applications/protocols that use this scheme: PipeStream protocol
(this document)
Scheme syntax: See Section 11.6.1.
Scheme semantics: A pipestream URI identifies an application-level
session context within a PipeStream deployment and, optionally, a
scope path and an entity reference within that context. It is
used to locate detached or resumable resources, such as Layer 2
claim checks, potentially across transport connections. The
authority component identifies the endpoint that issued the
resource.
Encoding considerations: All components other than the authority are
restricted to a subset of unreserved ASCII characters by the ABNF
in Section 11.6.1. Percent-encoding is neither required nor
permitted in those components. The authority component follows
the syntax and encoding rules of [RFC3986], Section 3.2.
Interoperability considerations: None beyond those described in this
document.
Security considerations: See Section 10 of this document. A
Rickert Expires 24 January 2027 [Page 59]
Internet-Draft PipeStream July 2026
pipestream URI may embed session and claim identifiers that
function as bearer capabilities; such URIs SHOULD be treated as
sensitive and MUST NOT be exposed in logs or referral contexts
where they could enable claim redemption by unauthorized parties.
Contact: Kristian Rickert (kristian.rickert@pipestream.ai)
Change controller: IETF
References: This document
11.6.1. URI Syntax
The URI syntax is defined using ABNF [RFC5234]:
pipestream-URI = "pipestream://" authority "/" session-id
[ "/" scope-path ] [ "/" entity-ref ]
session-id = 1*( ALPHA / DIGIT / "-" )
scope-path = scope-id *( "." scope-id )
scope-id = 1*DIGIT
entity-ref = 1*( ALPHA / DIGIT )
authority = <authority, see RFC 3986, Section 3.2>
Examples:
* pipestream://processor.example.com/a1b2c3d4
* pipestream://processor.example.com:8443/a1b2c3d4/1.42/e5f6
11.7. Guidance for Designated Experts
For all registries defined in this section, the designated experts
are advised to apply the following criteria when evaluating requests:
1. A permanent, readily available public specification of the
proposed value is required and needs to define the value's
semantics precisely enough for independent interoperable
implementations.
2. The proposed value must not conflict with, duplicate, or create
ambiguity with existing registrations, including the framing
class of frame types (fixed bit-packed versus serialized payload;
Section 11.2).
Rickert Expires 24 January 2027 [Page 60]
Internet-Draft PipeStream July 2026
3. Requests should be evaluated for their effect on existing
deployments; extensions that require behavior changes from
unmodified endpoints are inappropriate for these registries and
instead require a new protocol version (Section 3.4.1).
4. Assignments from scarce code spaces (in particular the 4-bit
status code space) should be granted conservatively; the expert
may require demonstrated deployment interest before assigning
values from a scarce space.
12. References
12.1. Normative References
[FIPS-180-4]
National Institute of Standards and Technology, "Secure
Hash Standard (SHS)", FIPS PUB 180-4, August 2015.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/rfc/rfc2119>.
[RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
Resource Identifier (URI): Generic Syntax", STD 66,
RFC 3986, DOI 10.17487/RFC3986, January 2005,
<https://www.rfc-editor.org/rfc/rfc3986>.
[RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", STD 68, RFC 5234,
DOI 10.17487/RFC5234, January 2008,
<https://www.rfc-editor.org/rfc/rfc5234>.
[RFC7301] Friedl, S., Popov, A., Langley, A., and E. Stephan,
"Transport Layer Security (TLS) Application-Layer Protocol
Negotiation Extension", RFC 7301, DOI 10.17487/RFC7301,
July 2014, <https://www.rfc-editor.org/rfc/rfc7301>.
[RFC7595] Thaler, D., Ed., Hansen, T., and T. Hardie, "Guidelines
and Registration Procedures for URI Schemes", BCP 35,
RFC 7595, DOI 10.17487/RFC7595, June 2015,
<https://www.rfc-editor.org/rfc/rfc7595>.
[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for
Writing an IANA Considerations Section in RFCs", BCP 26,
RFC 8126, DOI 10.17487/RFC8126, June 2017,
<https://www.rfc-editor.org/rfc/rfc8126>.
Rickert Expires 24 January 2027 [Page 61]
Internet-Draft PipeStream July 2026
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.
[RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol
Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
<https://www.rfc-editor.org/rfc/rfc8446>.
[RFC8610] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data
Definition Language (CDDL): A Notational Convention to
Express Concise Binary Object Representation (CBOR) and
JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610,
June 2019, <https://www.rfc-editor.org/rfc/rfc8610>.
[RFC8949] Bormann, C. and P. Hoffman, "Concise Binary Object
Representation (CBOR)", STD 94, RFC 8949,
DOI 10.17487/RFC8949, December 2020,
<https://www.rfc-editor.org/rfc/rfc8949>.
[RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
Multiplexed and Secure Transport", RFC 9000,
DOI 10.17487/RFC9000, May 2021,
<https://www.rfc-editor.org/rfc/rfc9000>.
12.2. Informative References
[MOQT] Nandakumar, S., Vasiliev, V., Swett, I., and A. Frindell,
"Media over QUIC Transport", Work in Progress, Internet-
Draft, draft-ietf-moq-transport-19, 6 July 2026,
<https://datatracker.ietf.org/doc/html/draft-ietf-moq-
transport-19>.
[PIPESTREAM-DOCPROC]
Rickert, K., "PipeStream Document Processing Profile",
March 2026, <https://github.com/ai-pipestream/pipestream-
quic-protocol-rfc/tree/main/docproc>.
[RFC7574] Bakker, A., Petrocco, R., and V. Grishchenko, "Peer-to-
Peer Streaming Peer Protocol (PPSPP)", RFC 7574,
DOI 10.17487/RFC7574, July 2015,
<https://www.rfc-editor.org/rfc/rfc7574>.
[RFC7696] Housley, R., "Guidelines for Cryptographic Algorithm
Agility and Selecting Mandatory-to-Implement Algorithms",
BCP 201, RFC 7696, DOI 10.17487/RFC7696, November 2015,
<https://www.rfc-editor.org/rfc/rfc7696>.
Rickert Expires 24 January 2027 [Page 62]
Internet-Draft PipeStream July 2026
[RFC7942] Sheffer, Y. and A. Farrel, "Improving Awareness of Running
Code: The Implementation Status Section", BCP 205,
RFC 7942, DOI 10.17487/RFC7942, July 2016,
<https://www.rfc-editor.org/rfc/rfc7942>.
[RFC9114] Bishop, M., Ed., "HTTP/3", RFC 9114, DOI 10.17487/RFC9114,
June 2022, <https://www.rfc-editor.org/rfc/rfc9114>.
[RFC9250] Huitema, C., Dickinson, S., and A. Mankin, "DNS over
Dedicated QUIC Connections", RFC 9250,
DOI 10.17487/RFC9250, May 2022,
<https://www.rfc-editor.org/rfc/rfc9250>.
[RFC9260] Stewart, R., Tuexen, M., and K. Nielsen, "Stream Control
Transmission Protocol", RFC 9260, June 2022,
<https://www.rfc-editor.org/rfc/rfc9260>.
[RFC9297] Schinazi, D. and L. Pardue, "HTTP Datagrams and the
Capsule Protocol", RFC 9297, DOI 10.17487/RFC9297, August
2022, <https://www.rfc-editor.org/rfc/rfc9297>.
[RFC9308] Kuehlewind, M. and B. Trammell, "Applicability of the QUIC
Transport Protocol", RFC 9308, September 2022,
<https://www.rfc-editor.org/rfc/rfc9308>.
[scatter-gather]
Lea, D., "The Scatter-Gather Design Pattern",
DOI 10.1007/978-1-4612-1260-6, 1996,
<https://doi.org/10.1007/978-1-4612-1260-6>.
Appendix A. Protocol Layer Capability Matrix
+=====================================+=========+=========+=========+
| Feature | Layer 0 | Layer 1 | Layer 2 |
+=====================================+=========+=========+=========+
| Unified status frame | X | X | X |
| (128-bit base) | | | |
+-------------------------------------+---------+---------+---------+
| Entity streaming | X | X | X |
+-------------------------------------+---------+---------+---------+
| PENDING/PROCESSING/COMPLETE/ | X | X | X |
| FAILED | | | |
+-------------------------------------+---------+---------+---------+
| DEHYDRATING/REHYDRATING | X | X | X |
+-------------------------------------+---------+---------+---------+
| Checkpoint blocking | X | X | X |
+-------------------------------------+---------+---------+---------+
| Assembly Manifest | X | X | X |
Rickert Expires 24 January 2027 [Page 63]
Internet-Draft PipeStream July 2026
+-------------------------------------+---------+---------+---------+
| Cursor-based ID recycling | X | X | X |
+-------------------------------------+---------+---------+---------+
| Scoped status fields (Scope | | X | X |
| ID, depth) | | | |
+-------------------------------------+---------+---------+---------+
| Hierarchical scopes | | X | X |
+-------------------------------------+---------+---------+---------+
| Scope digest (Merkle) | | X | X |
+-------------------------------------+---------+---------+---------+
| Barrier (subtree sync) | | X | X |
+-------------------------------------+---------+---------+---------+
| YIELDED status | | | X |
+-------------------------------------+---------+---------+---------+
| DEFERRED status | | | X |
+-------------------------------------+---------+---------+---------+
| Claim checks | | | X |
+-------------------------------------+---------+---------+---------+
| Completion policies | | | X |
+-------------------------------------+---------+---------+---------+
| SKIPPED/ABANDONED statuses | | | X |
+-------------------------------------+---------+---------+---------+
Table 22
Appendix B. Relationship to Existing Protocols
This appendix discusses the relationship between PipeStream and
existing transport and application protocols. The intent is to
clarify the design rationale for specifying a new application
protocol directly over QUIC [RFC9000] rather than layering on HTTP/3
[RFC9114], gRPC, or WebTransport [RFC9297].
B.1. QUIC as Application Transport
RFC 9308 [RFC9308] provides guidance for designers of application
protocol mappings to QUIC. PipeStream follows this guidance: stream
semantics are mapped to the protocol's data model (Section 4), flow
control operates at both the stream and connection level, and an ALPN
token is registered for protocol identification (Section 11).
Rickert Expires 24 January 2027 [Page 64]
Internet-Draft PipeStream July 2026
The precedent for application protocols that bypass HTTP and map
directly onto QUIC is well established. DNS over Dedicated QUIC
Connections [RFC9250] adopts a direct mapping on the grounds that
HTTP framing introduces unnecessary overhead when the application has
its own message semantics. The Media over QUIC Transport protocol
[MOQT] similarly defines its own framing and control messages over
QUIC streams, with HTTP/3 as an optional encapsulation rather than a
requirement.
PipeStream's requirements align with these precedents. The
protocol's dual-stream architecture (Section 4), bit-packed control
frames (Section 6), and recursive entity lifecycle (Section 9) have
no counterpart in HTTP semantics, and mapping them onto request-
response pairs would add complexity without benefit.
B.2. Efficiency Considerations
The decision to implement PipeStream as a standalone protocol over
QUIC is driven by the following efficiency requirements:
B.2.1. Framing Overhead
HTTP/3 and gRPC introduce multiple layers of framing. A gRPC message
over HTTP/3 incurs QUIC stream overhead, HTTP/3 DATA frame overhead
(minimum 2 octets), and gRPC envelope overhead (5 octets). For small
status updates like PipeStream's 21-octet on-wire STATUS frame, this
encapsulation would still materially increase bandwidth requirements.
PipeStream's bit-packed frames minimize coordination overhead for
high-frequency control traffic.
B.2.2. Stateless Control Plane
HTTP/3 requires maintaining state for QPACK header compression. In
high-concurrency environments where a single node manages thousands
of sessions, the memory overhead of these contexts is significant.
PipeStream status frames are stateless and self-describing; a
receiver can parse any frame without reference to previous frames,
enabling low-latency processing and easier load balancing across
redundant nodes.
B.2.3. Stream Independence
gRPC bidirectional streaming is constrained by the lifecycle of a
single RPC call. While multiple entities can be multiplexed into one
RPC, this reintroduces head-of-line blocking at the application layer
if one entity requires retransmission. Alternatively, opening a new
RPC for every entity (which could number in the millions for large
input sets) incurs substantial setup and teardown overhead.
Rickert Expires 24 January 2027 [Page 65]
Internet-Draft PipeStream July 2026
PipeStream treats each entity as an independent QUIC stream, ensuring
maximum parallelism and optimal use of QUIC's loss recovery
mechanisms.
B.3. HTTP/3
HTTP/3 [RFC9114] provides multiplexed request-response exchanges over
QUIC. Its stream model binds each client-initiated request to a
server response on the same stream. PipeStream requires
bidirectional, peer-initiated entity streams where either endpoint
may open new streams to transmit sub-entities arising from recursive
decomposition. The request-response constraint precludes this.
PipeStream also requires a persistent control stream carrying
compact, fixed-size status frames at high frequency. HTTP/3 does
define unidirectional control streams, but their framing is specific
to HTTP semantics (SETTINGS, GOAWAY, MAX_PUSH_ID) and cannot be
repurposed for application-level status coordination without
introducing a parallel signaling mechanism that duplicates much of
what QUIC already provides.
B.4. gRPC
gRPC defines a remote procedure call framework over HTTP/2, with
experimental support for HTTP/3. Bidirectional streaming in gRPC is
scoped to a single RPC method: one request stream and one response
stream per call. PipeStream requires an arbitrary number of
concurrent entity streams with independent flow control, plus a
dedicated control stream, all within a single connection. Achieving
this over gRPC would require either multiplexing all entities onto a
single bidirectional RPC (sacrificing per-stream flow control and
head-of-line independence) or opening a separate RPC per entity
(sacrificing session-level coordination and incurring per-call
overhead).
Beyond the stream model mismatch, gRPC does not provide protocol-
level primitives for recursive entity decomposition, checkpoint-based
consistency across parallel processing paths, or hierarchical scope
management with digest propagation. Building equivalent semantics on
top of gRPC requires application-layer coordination logic that
PipeStream provides natively as part of the protocol specification.
gRPC further mandates a 5-octet length-prefixed framing envelope for
every message. PipeStream's fixed-size control frames are bit-packed
at the wire level with zero serialization overhead, which is material
at the status update frequencies the protocol is designed to sustain.
Rickert Expires 24 January 2027 [Page 66]
Internet-Draft PipeStream July 2026
PipeStream complements rather than replaces gRPC: a PipeStream
pipeline stage MAY internally use gRPC to communicate with processing
services while relying on PipeStream for inter-stage coordination and
consistency.
B.5. WebTransport
WebTransport [RFC9297] provides bidirectional streams and unreliable
datagrams over HTTP/3, and is the closest existing protocol to the
transport abstraction PipeStream requires. However, several
properties make it unsuitable as a substrate:
WebTransport sessions are established via an HTTP/3 CONNECT request,
inheriting the client-server asymmetry of HTTP. In PipeStream, both
endpoints participate symmetrically in capability negotiation and may
initiate entity streams; the protocol does not distinguish a "client"
role from a "server" role after the handshake.
WebTransport is designed for environments constrained by the web
security model (origin-based isolation, CORS). PipeStream targets
server-to-server processing pipelines where these constraints are
inapplicable.
WebTransport provides raw byte streams with no built-in coordination
semantics. PipeStream would need to implement its own framing,
status state machine, checkpoint barriers, and scope hierarchy on top
of WebTransport streams. At that point, the HTTP/3 session layer
introduces an additional round trip during establishment and per-
stream framing overhead, with no corresponding benefit.
B.6. Media over QUIC Transport
The Media over QUIC Transport protocol [MOQT] is architecturally the
closest existing QUIC-native protocol to PipeStream: both use a
bidirectional control stream for session coordination alongside
unidirectional streams carrying independent data objects, following
the application-mapping guidance of [RFC9308].
The two protocols nevertheless solve different problems. MOQT is a
publish/subscribe media delivery protocol optimized for fan-out
through relays: objects within a track are delivered to a dynamic set
of subscribers, delivery of any individual object may be abandoned
when it ceases to be useful (for example, after a latency deadline
passes), and no participant tracks whether the totality of published
objects reached any subscriber. PipeStream is a scatter-gather
processing protocol with the opposite requirements: the set of child
entities produced by a decomposition is determinate, every entity
MUST be driven to a terminal status, and reassembly is gated on
Rickert Expires 24 January 2027 [Page 67]
Internet-Draft PipeStream July 2026
cryptographically verifiable completion of the entire set
(Section 9). MOQT has no counterpart to hierarchical scopes, scope
digests, barrier synchronization, or completion policies, and
PipeStream has no counterpart to tracks, subscriptions, or relay-
based fan-out.
Extending MOQT with completion tracking would require adding
precisely the control-plane machinery this document defines, while
inheriting subscription semantics that have no role in pipeline
processing.
B.7. Broker-Based Pipeline Architectures
Many production processing pipelines are built not on RPC frameworks
but on message brokers and distributed commit logs, with stages
communicating through persistent intermediary queues or topics.
These architectures provide durable decoupling and replay, and
PipeStream is not a replacement for them where store-and-forward
durability between organizationally separate stages is the primary
requirement.
However, broker-based architectures externalize the semantics that
PipeStream provides natively. Completion of a hierarchically
decomposed job must be tracked in a coordination store outside the
message path; end-to-end integrity of a reassembled output must be
verified by application logic; and backpressure operates per queue
rather than end-to-end across a pipeline. Each stage boundary
additionally imposes a full persist-and-forward cycle, which defeats
incremental streaming of large entities. The broker itself is also
an interoperability boundary: no standardized wire protocol expresses
scatter-gather semantics across brokers from different vendors, so
multi-organization pipelines are coupled through vendor-specific
client libraries.
A PipeStream deployment MAY use a broker behind an individual
processing stage (as it may use gRPC; see above) while relying on
PipeStream for inter-stage transport, completion tracking, and
integrity verification.
B.8. Summary
PipeStream occupies a design point not addressed by existing
protocols: a QUIC-native application protocol combining multiplexed
entity streaming, recursive decomposition with hierarchical scopes,
Merkle-based integrity propagation, and barrier-synchronized
reassembly. While existing protocols like SCTP [RFC9260] and PPSPP
[RFC7574] address subsets of these requirements, none provide the
integrated lifecycle and coordination semantics that PipeStream
Rickert Expires 24 January 2027 [Page 68]
Internet-Draft PipeStream July 2026
defines.
Appendix C. Schema Reference (CDDL)
This appendix consolidates the normative CDDL [RFC8610] schema
definitions for PipeStream Core messages that use negotiated
serialization. These definitions are authoritative for the wire
format when CBOR [RFC8949] is the negotiated serialization format
(the default). Fixed control frames on Stream 0, such as STATUS and
SCOPE_DIGEST, use the bit-packed wire formats defined in Section 6
and are not serialized as CDDL messages.
Application-specific payload envelopes and profile-specific schemas
are outside the scope of this appendix; see [PIPESTREAM-DOCPROC] for
an example companion profile that defines such messages.
; -----------------------------------------------------------
; Integer size convention
; -----------------------------------------------------------
; CDDL "uint" is an unbounded unsigned integer. When
; encoded in CBOR, the encoder MUST use the smallest
; CBOR major-type-0 encoding that fits the value.
; The following aliases document the wire-format field
; widths used in fixed-size frames; they do not constrain
; the CBOR encoding but record the maximum value each
; field may carry.
;
; uint32 values 0..4294967295 (Entity ID, Scope ID)
; uint64 values 0..2^64-1 (counters, timestamps)
;
; For variable-length serialized messages (CBOR), the
; natural uint encoding applies and receivers MUST accept
; any valid CBOR unsigned integer.
; -----------------------------------------------------------
; -----------------------------------------------------------
; Serialization format negotiation
; -----------------------------------------------------------
serialization-format = uint .le 255
; Value from the PipeStream Serialization
; Formats registry (Section 11.5).
; This document defines only CBOR (0).
; -----------------------------------------------------------
; Capabilities (exchanged during CONNECT on Stream 0)
; -----------------------------------------------------------
Rickert Expires 24 January 2027 [Page 69]
Internet-Draft PipeStream July 2026
capabilities = {
layer0-core: bool, ; MUST be true
layer1-recursive: bool,
layer2-resilience: bool,
? max-scope-depth: uint .le 7, ; Default: 7
? max-entities-per-scope: uint, ; Default: 4,294,967,292
? max-window-size: uint, ; Default: 2,147,483,646
; (Section 9.1)
? serialization-format: serialization-format,
? keepalive-timeout-ms: uint, ; Default: 30000 (30s)
}
; -----------------------------------------------------------
; Entity header (prefixes each entity on Entity Streams)
; -----------------------------------------------------------
entity-header = {
entity-id: uint, ; 32-bit on wire (STATUS frame)
? parent-id: uint, ; 32-bit scope-local
? scope-id: uint, ; 32-bit (Section 6.2.1)
layer: uint .le 3, ; Data layer 0-3
? content-type: tstr,
? payload-length: uint, ; Octet count of this frame's
; payload; SHOULD be present
; (Section 6.8.1)
? checksum: bstr .size 32, ; SHA-256; SHOULD be present
? metadata: { * tstr => tstr },
? chunk-info: chunk-info,
? completion-policy: completion-policy, ; Layer 2
}
chunk-info = {
total-chunks: uint,
chunk-index: uint,
chunk-offset: uint,
}
; -----------------------------------------------------------
; Completion policy (Layer 2)
; -----------------------------------------------------------
completion-policy = {
? mode: completion-mode,
? max-retries: uint,
? retry-delay-ms: uint,
? timeout-ms: uint,
? min-success-ratio: float32,
? on-timeout: failure-action,
Rickert Expires 24 January 2027 [Page 70]
Internet-Draft PipeStream July 2026
? on-failure: failure-action,
}
completion-mode = &(
unspecified: 0,
strict: 1,
lenient: 2,
best-effort: 3,
quorum: 4,
)
failure-action = &(
unspecified: 0,
fail: 1,
skip: 2,
retry: 3,
defer: 4,
)
; -----------------------------------------------------------
; Checkpoint frame (Type 0x81)
; -----------------------------------------------------------
checkpoint-frame = {
checkpoint-id: tstr,
sequence-number: uint,
checkpoint-entity-id: uint,
? scope-id: uint,
? flags: uint,
? timeout-ms: uint,
}
; -----------------------------------------------------------
; Entity status codes
; -----------------------------------------------------------
entity-status = &(
unspecified: 0,
pending: 1,
processing: 2,
complete: 3,
failed: 4,
checkpoint: 5,
dehydrating: 6,
rehydrating: 7,
yielded: 8,
deferred: 9,
retrying: 10,
Rickert Expires 24 January 2027 [Page 71]
Internet-Draft PipeStream July 2026
skipped: 11,
abandoned: 12,
)
; -----------------------------------------------------------
; Assembly Manifest entry
; -----------------------------------------------------------
assembly-manifest-entry = {
parent-id: uint,
? scope-id: uint,
children-ids: [* uint],
? children-status: [* entity-status],
? policy: completion-policy,
? created-at: uint,
? state: resolution-state,
}
resolution-state = &(
unspecified: 0,
active: 1,
resolved: 2,
partial: 3,
failed: 4,
)
; -----------------------------------------------------------
; Yield token (Layer 2)
; -----------------------------------------------------------
yield-token = {
reason: yield-reason,
? continuation-state: bstr,
? validation: stopping-point-validation,
}
yield-reason = &(
unspecified: 0,
external-call: 1,
rate-limited: 2,
awaiting-sibling: 3,
awaiting-approval: 4,
resource-busy: 5,
)
; -----------------------------------------------------------
; Claim check (Layer 2)
; -----------------------------------------------------------
Rickert Expires 24 January 2027 [Page 72]
Internet-Draft PipeStream July 2026
claim-check = {
claim-id: uint,
entity-id: uint,
? scope-id: uint,
expiry-timestamp: uint,
? validation: stopping-point-validation,
}
; -----------------------------------------------------------
; Stopping point validation (Layer 2)
; -----------------------------------------------------------
stopping-point-validation = {
? state-checksum: bstr,
? bytes-processed: uint,
? children-complete: uint,
? children-total: uint,
? is-resumable: bool,
? checkpoint-ref: tstr,
}
; -----------------------------------------------------------
; File storage reference
; -----------------------------------------------------------
file-storage-reference = {
provider: tstr,
bucket: tstr,
key: tstr,
? region: tstr,
? attrs: { * tstr => tstr },
? encryption: encryption-metadata,
}
encryption-metadata = {
algorithm: tstr,
? key-provider: tstr,
? key-id: tstr,
? wrapped-key: bstr,
? iv: bstr,
? context: { * tstr => tstr },
}
Appendix D. Implementation Status
[[RFC Editor: please remove this entire appendix, and the reference
to [RFC7942], before publication.]]
Rickert Expires 24 January 2027 [Page 73]
Internet-Draft PipeStream July 2026
This appendix records the status of known implementations of the
protocol defined by this specification at the time of posting of this
Internet-Draft, following the process described in [RFC7942]. The
description of implementations in this appendix is intended to assist
the IETF in its decision processes in progressing drafts to RFCs.
Please note that the listing of any individual implementation here
does not imply endorsement by the IETF. Furthermore, no effort has
been spent to verify the information presented here that was supplied
by IETF contributors. This is not intended as, and must not be
construed to be, a catalog of available implementations or their
features. Readers are advised to note that other implementations may
exist.
D.1. Reference Implementation
Organization: PipeStream AI
Description: Reference implementation of the PipeStream protocol.
Maturity: Under development; not yet publicly released.
Coverage: To be updated as development progresses.
Licensing: To be determined.
Contact: Kristian Rickert (kristian.rickert@pipestream.ai)
The authors welcome reports of additional implementations for
inclusion in future revisions of this appendix.
Author's Address
Kristian Rickert
PipeStream AI
Email: kristian.rickert@pipestream.ai
Rickert Expires 24 January 2027 [Page 74]