|Schinazi & Pardue
|Expires 13 October 2022
- Intended Status:
- Standards Track
HTTP Datagrams and the Capsule Protocol
This document describes HTTP Datagrams, a convention for conveying multiplexed, potentially unreliable datagrams inside an HTTP connection.¶
In HTTP/3, HTTP Datagrams can be conveyed natively using the QUIC DATAGRAM extension. When the QUIC DATAGRAM frame is unavailable or undesirable, they can be sent using the Capsule Protocol, a more general convention for conveying data in HTTP connections.¶
HTTP Datagrams and the Capsule Protocol are intended for use by HTTP extensions, not applications.¶
This note is to be removed before publishing as an RFC.¶
The latest revision of this draft can be found at https://ietf-wg-masque.github.io/draft-ietf-masque-h3-datagram/draft-ietf-masque-h3-datagram.html. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-ietf-masque-h3-datagram/.¶
Source for this draft and an issue tracker can be found at https://github.com/ietf-wg-masque/draft-ietf-masque-h3-datagram.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 13 October 2022.¶
Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
HTTP extensions sometimes need to access underlying transport protocol features such as unreliable delivery (as offered by [DGRAM]) to enable desirable features. For example, this could allow introducing an unreliable version of the CONNECT method, or adding unreliable delivery to WebSockets [RFC6455].¶
In Section 2, this document describes HTTP Datagrams, a convention that supports the bidirectional and optionally multiplexed exchange of data inside an HTTP connection. While HTTP datagrams are associated with HTTP requests, they are not part of message content; instead, they are intended for use by HTTP extensions (such as the CONNECT method), and are compatible with all versions of HTTP.¶
When HTTP is running over a transport protocol that supports unreliable delivery (such as when the QUIC DATAGRAM extension is available to HTTP/3), HTTP Datagrams can use that capability.¶
This document also describes the HTTP Capsule Protocol in Section 3, to allow conveyance of HTTP Datagrams using reliable delivery. This addresses HTTP/3 cases where use of the QUIC DATAGRAM frame is unavailable or undesirable, or where the transport protocol only provides reliable delivery, such as with HTTP/1 or HTTP/2 over TCP.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
HTTP Datagrams are a convention for conveying bidirectional and potentially unreliable datagrams inside an HTTP connection, with multiplexing when possible. All HTTP Datagrams are associated with an HTTP request.¶
When HTTP Datagrams are conveyed on an HTTP/3 connection, the QUIC DATAGRAM frame can be used to achieve these goals, including unreliable delivery; see Section 2.1. Negotiating the use of QUIC DATAGRAM frames for HTTP Datagrams is achieved via the exchange of HTTP/3 settings; see Section 2.1.1.¶
When running over HTTP/2, demultiplexing is provided by the HTTP/2 framing layer, but unreliable delivery is unavailable. HTTP Datagrams are negotiated and conveyed using the Capsule Protocol; see Section 3.5.¶
When running over HTTP/1, requests are strictly serialized in the connection, and therefore demultiplexing is not available. Unreliable delivery is likewise not available. HTTP Datagrams are negotiated and conveyed using the Capsule Protocol; see Section 3.5.¶
HTTP Datagrams MUST only be sent with an association to an HTTP request that explicitly supports them. For example, existing HTTP methods GET and POST do not define semantics for associated HTTP Datagrams; therefore, HTTP Datagrams cannot be sent associated with GET or POST request streams.¶
If an HTTP Datagram is received and it is associated with a request that has no known semantics for HTTP Datagrams, the receiver MUST terminate the request; if HTTP/3 is in use, the request stream MUST be aborted with H3_DATAGRAM_ERROR (0x33). HTTP extensions can override these requirements by defining a negotiation mechanism and semantics for HTTP Datagrams.¶
- Quarter Stream ID:
A variable-length integer that contains the value of the client-initiated bidirectional stream that this datagram is associated with, divided by four (the division by four stems from the fact that HTTP requests are sent on client-initiated bidirectional streams, and those have stream IDs that are divisible by four). The largest legal QUIC stream ID value is 262-1, so the largest legal value of Quarter Stream ID is 260-1. Receipt of an HTTP/3 Datagram that includes a larger value MUST be treated as an HTTP/3 connection error of type H3_DATAGRAM_ERROR (0x33).¶
- HTTP Datagram Payload:
The payload of the datagram, whose semantics are defined by the extension that is using HTTP Datagrams. Note that this field can be empty.¶
Receipt of a QUIC DATAGRAM frame whose payload is too short to allow parsing the Quarter Stream ID field MUST be treated as an HTTP/3 connection error of type H3_DATAGRAM_ERROR (0x33).¶
HTTP/3 Datagrams MUST NOT be sent unless the corresponding stream's send side is open. If a datagram is received after the corresponding stream's receive side is closed, the received datagrams MUST be silently dropped.¶
If an HTTP/3 datagram is received and its Quarter Stream ID maps to a stream that has not yet been created, the receiver SHALL either drop that datagram silently or buffer it temporarily (on the order of a round trip) while awaiting the creation of the corresponding stream.¶
If an HTTP/3 datagram is received and its Quarter Stream ID maps to a stream that cannot be created due to client-initiated bidirectional stream limits, it SHOULD be treated as an HTTP/3 connection error of type H3_ID_ERROR. Generating an error is not mandatory in this case because HTTP/3 implementations might have practical barriers to determining the active stream concurrency limit that is applied by the QUIC layer.¶
Prioritization of HTTP/3 datagrams is not defined in this document. Future extensions MAY define how to prioritize datagrams, and MAY define signaling to allow communicating prioritization preferences.¶
Endpoints can indicate to their peer that they are willing to receive HTTP/3 Datagrams by sending the SETTINGS_H3_DATAGRAM (0x33) setting with a value of 1.¶
The value of the SETTINGS_H3_DATAGRAM setting MUST be either 0 or 1. A value of 0 indicates that the implementation is not willing to receive HTTP Datagrams. If the SETTINGS_H3_DATAGRAM setting is received with a value that is neither 0 or 1, the receiver MUST terminate the connection with error H3_SETTINGS_ERROR.¶
QUIC DATAGRAM frames MUST NOT be sent until the SETTINGS_H3_DATAGRAM setting has been both sent and received with a value of 1.¶
When clients use 0-RTT, they MAY store the value of the server's SETTINGS_H3_DATAGRAM setting. Doing so allows the client to send QUIC DATAGRAM frames in 0-RTT packets. When servers decide to accept 0-RTT data, they MUST send a SETTINGS_H3_DATAGRAM setting greater than or equal to the value they sent to the client in the connection where they sent them the NewSessionTicket message. If a client stores the value of the SETTINGS_H3_DATAGRAM setting with their 0-RTT state, they MUST validate that the new value of the SETTINGS_H3_DATAGRAM setting sent by the server in the handshake is greater than or equal to the stored value; if not, the client MUST terminate the connection with error H3_SETTINGS_ERROR. In all cases, the maximum permitted value of the SETTINGS_H3_DATAGRAM setting parameter is 1.¶
It is RECOMMENDED that implementations that support receiving HTTP/3 Datagrams always send the SETTINGS_H3_DATAGRAM setting with a value of 1, even if the application does not intend to use HTTP/3 Datagrams. This helps to avoid "sticking out"; see Section 4.¶
[[RFC editor: please remove this section before publication.]]¶
Some revisions of this draft specification use a different value (the Identifier field of a Setting in the HTTP/3 SETTINGS frame) for the SETTINGS_H3_DATAGRAM setting. This allows new draft revisions to make incompatible changes. Multiple draft versions MAY be supported by sending multiple values for SETTINGS_H3_DATAGRAM. Once SETTINGS have been sent and received, an implementation that supports multiple drafts MUST compute the intersection of the values it has sent and received, and then it MUST select and use the most recent draft version from the intersection set. This ensures that both peers negotiate the same draft version.¶
One mechanism to extend HTTP is to introduce new HTTP Upgrade Tokens (see Section 16.7 of [HTTP]). In HTTP/1.x, these tokens are used via the Upgrade mechanism (see Section 7.8 of [HTTP]). In HTTP/2 and HTTP/3, these tokens are used via the Extended CONNECT mechanism (see [EXT-CONNECT2] and [EXT-CONNECT3]).¶
This specification introduces the Capsule Protocol. The Capsule Protocol is a sequence of type-length-value tuples that definitions of new HTTP Upgrade Tokens can choose to use. It allows endpoints to reliably communicate request-related information end-to-end on HTTP request streams, even in the presence of HTTP intermediaries. The Capsule Protocol can be used to exchange HTTP Datagrams, which is necessary when HTTP is running over a transport that does not support the QUIC DATAGRAM frame. The Capsule Protocol can also be used to communicate reliable and bidirectional control messages associated with a datagram-based protocol even when HTTP/3 Datagrams are in use.¶
This specification defines the "data stream" of an HTTP request as the bidirectional stream of bytes that follows the header section of the request message and the final, successful (i.e., 2xx) response message.¶
In HTTP/1.x, the data stream consists of all bytes on the connection that follow the blank line that concludes either the request header section, or the response header section. As a result, only a single HTTP request starting the capsule protocol can be sent on HTTP/1.x connections.¶
In HTTP/2 and HTTP/3, the data stream of a given HTTP request consists of all bytes sent in DATA frames with the corresponding stream ID.¶
The concept of a data stream is particularly relevant for methods such as CONNECT where there is no HTTP message content after the headers.¶
Definitions of new HTTP Upgrade Tokens can state that their associated request's data stream uses the Capsule Protocol. If they do so, that means that the contents of the associated request's data stream uses the following format (using the notation from the "Notational Conventions" section of [QUIC]):¶
- Capsule Type:
A variable-length integer indicating the Type of the capsule.¶
- Capsule Length:
The length of the Capsule Value field following this field, encoded as a variable-length integer. Note that this field can have a value of zero.¶
- Capsule Value:
The payload of this capsule. Its semantics are determined by the value of the Capsule Type field.¶
Because new protocols or extensions might define new capsule types, intermediaries that wish to allow for future extensibility SHOULD forward capsules without modification, unless the definition of the Capsule Type in use specifies additional intermediary processing. One such Capsule Type is the DATAGRAM capsule; see Section 3.5. In particular, intermediaries SHOULD forward Capsules with an unknown Capsule Type without modification.¶
Endpoints which receive a Capsule with an unknown Capsule Type MUST silently drop that Capsule and skip over it to parse the next Capsule.¶
By virtue of the definition of the data stream:¶
- The Capsule Protocol is not in use unless the response includes a 2xx (Successful) status code.¶
- When the Capsule Protocol is in use, the associated HTTP request and response do not carry HTTP content. A future extension MAY define a new capsule type to carry HTTP content.¶
The Capsule Protocol MUST NOT be used with messages that contain Content-Length, Content-Type, or Transfer-Encoding header fields. Additionally, HTTP status codes 204 (No Content), 205 (Reset Content), and 206 (Partial Content) MUST NOT be sent on responses that use the Capsule Protocol. A receiver that observes a violation of these requirements MUST treat the HTTP message as malformed.¶
When an error occurs in processing the Capsule Protocol, the receiver MUST treat the message as malformed or incomplete, according to the underlying transport protocol. For HTTP/3, the handling of malformed messages is described in Section 4.1.3 of [HTTP/3]. For HTTP/2, the handling of malformed messages is described in Section 8.1.1 of [HTTP/2]. For HTTP/1.1, the handling of incomplete messages is described in Section 8 of [HTTP/1.1].¶
Each capsule's payload MUST contain exactly the fields identified in its description. A capsule payload that contains additional bytes after the identified fields or a capsule payload that terminates before the end of the identified fields MUST be treated as a malformed or incomplete message. In particular, redundant length encodings MUST be verified to be self-consistent.¶
If the receive side of a stream carrying capsules is terminated cleanly (for example, in HTTP/3 this is defined as receiving a QUIC STREAM frame with the FIN bit set) and the last capsule on the stream was truncated, this MUST be treated as a malformed or incomplete message.¶
The "Capsule-Protocol" header field is an Item Structured Field, see Section 3.3 of [STRUCT-FIELD]; its value MUST be a Boolean; any other value type MUST be handled as if the field were not present by recipients (for example, if this field is included multiple times, its type will become a List and the field will therefore be ignored). This document does not define any parameters for the Capsule-Protocol header field value, but future documents might define parameters. Receivers MUST ignore unknown parameters.¶
Endpoints indicate that the Capsule Protocol is in use on a data stream by sending a Capsule-Protocol header field with a true value. A Capsule-Protocol header field with a false value has the same semantics as when the header is not present.¶
Intermediaries MAY use this header field to allow processing of HTTP Datagrams for unknown HTTP Upgrade Tokens; note that this is only possible for HTTP Upgrade or Extended CONNECT.¶
The Capsule-Protocol header field MUST NOT be used on HTTP responses with a status code outside the 2xx range.¶
When using the Capsule Protocol, HTTP endpoints SHOULD send the Capsule-Protocol header field to simplify intermediary processing. Definitions of new HTTP Upgrade Tokens that use the Capsule Protocol MAY alter this recommendation.¶
This document defines the DATAGRAM (0x00) capsule type. This capsule allows HTTP Datagrams to be sent on a stream using the Capsule Protocol. This is particularly useful when HTTP is running over a transport that does not support the QUIC DATAGRAM frame.¶
- HTTP Datagram Payload:
The payload of the datagram, whose semantics are defined by the extension that is using HTTP Datagrams. Note that this field can be empty.¶
HTTP Datagrams sent using the DATAGRAM capsule have the same semantics as those sent in QUIC DATAGRAM frames. In particular, the restrictions on when it is allowed to send an HTTP Datagram and how to process them from Section 2.1 also apply to HTTP Datagrams sent and received using the DATAGRAM capsule.¶
An intermediary can reencode HTTP Datagrams as it forwards them. In other words, an intermediary MAY send a DATAGRAM capsule to forward an HTTP Datagram that was received in a QUIC DATAGRAM frame, and vice versa. Intermediaries MUST NOT perform this reencoding unless they have identified the use of the Capsule Protocol on the corresponding request stream; see Section 3.2.¶
Note that while DATAGRAM capsules that are sent on a stream are reliably delivered in order, intermediaries can reencode DATAGRAM capsules into QUIC DATAGRAM frames when forwarding messages, which could result in loss or reordering.¶
If an intermediary receives an HTTP Datagram in a QUIC DATAGRAM frame and is forwarding it on a connection that supports QUIC DATAGRAM frames, the intermediary SHOULD NOT convert that HTTP Datagram to a DATAGRAM capsule. If the HTTP Datagram is too large to fit in a DATAGRAM frame (for example because the path MTU of that QUIC connection is too low or if the maximum UDP payload size advertised on that connection is too low), the intermediary SHOULD drop the HTTP Datagram instead of converting it to a DATAGRAM capsule. This preserves the end-to-end unreliability characteristic that methods such as Datagram Packetization Layer Path MTU Discovery (DPLPMTUD) depend on [DPLPMTUD]. An intermediary that converts QUIC DATAGRAM frames to DATAGRAM capsules allows HTTP Datagrams to be arbitrarily large without suffering any loss; this can misrepresent the true path properties, defeating methods such as DPLPMTUD.¶
While DATAGRAM capsules can theoretically carry a payload of length 262-1, most HTTP extensions that use HTTP Datagrams will have their own limits on what datagram payload sizes are practical. Implementations SHOULD take those limits into account when parsing DATAGRAM capsules: if an incoming DATAGRAM capsule has a length that is known to be so large as to not be usable, the implementation SHOULD discard the capsule without buffering its contents into memory.¶
Note that it is possible for an HTTP extension to use HTTP Datagrams without using the Capsule Protocol. For example, if an HTTP extension that uses HTTP Datagrams is only defined over transports that support QUIC DATAGRAM frames, it might not need a stream encoding. Additionally, HTTP extensions can use HTTP Datagrams with their own data stream protocol. However, new HTTP extensions that wish to use HTTP Datagrams SHOULD use the Capsule Protocol as failing to do so will make it harder for the HTTP extension to support versions of HTTP other than HTTP/3 and will prevent interoperability with intermediaries that only support the Capsule Protocol.¶
Since transmitting HTTP Datagrams using QUIC DATAGRAM frames requires sending the HTTP/3 SETTINGS_H3_DATAGRAM setting, it "sticks out". In other words, probing clients can learn whether a server supports HTTP Datagrams over QUIC DATAGRAM frames. As some servers might wish to obfuscate the fact that they offer application services that use HTTP datagrams, it's best for all implementations that support this feature to always send this setting, see Section 2.1.1.¶
This document will request IANA to register the following entry in the "HTTP/3 Settings" registry:¶
This document will request IANA to register the following entry in the "HTTP/3 Error Codes" registry:¶
This document will request IANA to register the following entry in the "HTTP Field Name" registry:¶
This document establishes a registry for HTTP capsule type codes. The "HTTP Capsule Types" registry governs a 62-bit space, and operates under the QUIC registration policy documented in Section 22.1 of [QUIC]. This new registry includes the common set of fields listed in Section 22.1.1 of [QUIC]. In addition to those common fields, all registrations in this registry MUST include a "Capsule Type" field which contains a short name or label for the capsule type.¶
Permanent registrations in this registry are assigned using the Specification Required policy (Section 4.6 of [IANA-POLICY]), except for values between 0x00 and 0x3f (in hexadecimal; inclusive), which are assigned using Standards Action or IESG Approval as defined in Sections 4.9 and 4.10 of [IANA-POLICY].¶
Capsule types with a value of the form 0x29 * N + 0x17 for integer values of N are reserved to exercise the requirement that unknown capsule types be ignored. These capsules have no semantics and can carry arbitrary values. These values MUST NOT be assigned by IANA and MUST NOT appear in the listing of assigned values.¶
This registry initially contains the following entry:¶
- Pauly, T., Kinnear, E., and D. Schinazi, "An Unreliable Datagram Extension to QUIC", RFC 9221, DOI 10.17487/RFC9221, , <https://www.rfc-editor.org/rfc/rfc9221>.
- Fielding, R. T., Nottingham, M., and J. Reschke, "HTTP Semantics", Work in Progress, Internet-Draft, draft-ietf-httpbis-semantics-19, , <https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-semantics-19>.
- Fielding, R. T., Nottingham, M., and J. Reschke, "HTTP/1.1", Work in Progress, Internet-Draft, draft-ietf-httpbis-messaging-19, , <https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-messaging-19>.
- Thomson, M. and C. Benfield, "HTTP/2", Work in Progress, Internet-Draft, draft-ietf-httpbis-http2bis-07, , <https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-http2bis-07>.
- Bishop, M., "Hypertext Transfer Protocol Version 3 (HTTP/3)", Work in Progress, Internet-Draft, draft-ietf-quic-http-34, , <https://datatracker.ietf.org/doc/html/draft-ietf-quic-http-34>.
- Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, , <https://www.rfc-editor.org/rfc/rfc8126>.
- Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, DOI 10.17487/RFC9000, , <https://www.rfc-editor.org/rfc/rfc9000>.
- Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/rfc/rfc2119>.
- Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/rfc/rfc8174>.
- Nottingham, M. and P-H. Kamp, "Structured Field Values for HTTP", RFC 8941, DOI 10.17487/RFC8941, , <https://www.rfc-editor.org/rfc/rfc8941>.
- Fairhurst, G., Jones, T., Tüxen, M., Rüngeler, I., and T. Völker, "Packetization Layer Path MTU Discovery for Datagram Transports", RFC 8899, DOI 10.17487/RFC8899, , <https://www.rfc-editor.org/rfc/rfc8899>.
- McManus, P., "Bootstrapping WebSockets with HTTP/2", RFC 8441, DOI 10.17487/RFC8441, , <https://www.rfc-editor.org/rfc/rfc8441>.
- Hamilton, R., "Bootstrapping WebSockets with HTTP/3", Work in Progress, Internet-Draft, draft-ietf-httpbis-h3-websockets-04, , <https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-h3-websockets-04>.
- Oku, K. and L. Pardue, "Extensible Prioritization Scheme for HTTP", Work in Progress, Internet-Draft, draft-ietf-httpbis-priority-12, , <https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-priority-12>.
- Fette, I. and A. Melnikov, "The WebSocket Protocol", RFC 6455, DOI 10.17487/RFC6455, , <https://www.rfc-editor.org/rfc/rfc6455>.
Portions of this document were previously part of the QUIC DATAGRAM frame definition itself, the authors would like to acknowledge the authors of that document and the members of the IETF MASQUE working group for their suggestions. Additionally, the authors would like to thank Martin Thomson for suggesting the use of an HTTP/3 setting. Furthermore, the authors would like to thank Ben Schwartz for writing the first proposal that used two layers of indirection. The final design in this document came out of the HTTP Datagrams Design Team, whose members were Alan Frindell, Alex Chernyakhovsky, Ben Schwartz, Eric Rescorla, Marcus Ihlar, Martin Thomson, Mike Bishop, Tommy Pauly, Victor Vasiliev, and the authors of this document. The authors thank Mark Nottingham and Philipp Tiesel for their helpful comments.¶