Internet-Draft CHIC November 2026
Thomson Expires 5 May 2027 [Page]
Workgroup:
HTTP
Internet-Draft:
draft-thomson-httpbis-chic-01
Published:
Intended Status:
Standards Track
Expires:
Author:
M. Thomson
Mozilla

Chained Hash Integrity Coding for HTTP

Abstract

This document defines a content coding for HTTP that provides strong integrity protection while maintaining the ability to perform some progressive processing.

About This Document

This note is to be removed before publishing as an RFC.

The latest revision of this draft can be found at https://martinthomson.github.io/chic/draft-thomson-httpbis-chic.html. Status information for this document may be found at https://datatracker.ietf.org/doc/draft-thomson-httpbis-chic/.

Discussion of this document takes place on the HTTP Group mailing list (mailto:ietf-http-wg@w3.org), which is archived at https://lists.w3.org/Archives/Public/ietf-http-wg/.

Source for this draft and an issue tracker can be found at https://github.com/martinthomson/chic.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 5 May 2027.

1. Introduction

Hash-based integrity is a powerful tool for ensuring that content matches expectations. Hashes are often the basis of digital signature schemes that can authenticate content, or at least provide verifiable proof of authorization.

The main challenge with hashes is that validating integrity requires every byte in the sequence. This document describes a method of hash generation that can be verified incrementally. This enables systems that depend on progressive processing, such as streaming processing of a long sequence of records, or performance-sensitive applications like HTML rendering.

The way this works is by splitting a document into chunks, called chiclets. Each chiclet comprises a length, some number of bytes of content, and a hash for the next chiclet. This is shown in Figure 1.

... H() H() Len Content Hash Len Content Hash ... First byte Last byte
Figure 1: Overview of Format

The hash of the entirety of the content can thereby be represented by the hash of the first chiclet, or chi digest.

This enables progressive processing of content, maintaining strong integrity for all content bytes, provided that each chiclet is only released after being validated; see Section 3.2. That can reduce latency -- or at least perceived latency -- for applications that are sensitive to delays by allowing processing to start before the entire sequence of bytes is received.

The cost of this scheme is that there is a small amount of overhead. Additional bytes for the hash and length fields are interspersed with content bytes. The relative cost of that overhead depends on the size of chiclets; a 4 KiB chiclet size adds overhead of 0.83%, whereas a 65535-byte chiclet adds overhead of 0.055%. Larger chiclets reduce overhead, but add latency; smaller chiclets can improve latency with higher overheads.

Though it is possible to process data incrementally, constructing this format requires two passes over the entire sequence of bytes; see Section 3.3.

2. Conventions and Definitions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

The term "chi digest" refers to the hash of the first chiclet in the format defined in Section 3.

This document uses the varint encoding and presentation language developed for QUIC Sections 16 and 1.3 of [QUIC].

3. Chained Hash Integrity Format

Any continuous stream of bytes can be turned into a stream of variable-length chiclets. This format allows for progressive validation of the integrity of content.

3.1. Format Definition

The format comprises zero or more non-final chiclets and a final chiclet. Content bytes are therefore interspersed with framing and integrity bytes.

A non-final chiclet comprises a non-zero length (for framing), between 1 and 261-41 bytes (inclusive) of content (see Section 3.4), and a 32-byte hash (that is, integrity bytes).

Lengths are encoded using the variable-length integer format from Section 16 of [QUIC]. Lengths do not need to be encoded in the minimal form; the chosen encoding will affect the resulting digest.

The hash is the output of the SHA-256 algorithm [SHA2] that covers the entirety of the subsequent chiclet.

The final chiclet includes a zero-valued length field, up to 261-2 bytes of content (less if the length is encoded in more than one byte), and no hash value. The final chiclet ends when content ends, which means that this format can only be used where there is a distinct end to the protected content. That might mean the end of a file, the end of a stream, or the end of an HTTP message; see Section 4.

The hash value that is computed from the very first chiclet -- the chi digest -- is not written to the output. The chi digest commits to the entire sequence of chiclets and the content that it protects. A chi digest needs to be carried outside the content; see Section 4.2 for one way to convey a chi digest.

The format is illustrated in Figure 2 using the notation from Section 1.3 of [QUIC].

Chained Hash Integrity Format {
  Chiclet (..) ...,
  Final Chiclet (..),
}

Chiclet {
  Length (i),
  Content (8..),
  Hash (256),
}

Final Chiclet {
  Length (i) = 0,
  Content (..),
}
Figure 2: Chained Hash Integrity Format

The format uses SHA-256 for integrity [SHA2]; see Section 5.1 for details.

3.2. Consumption and Validation

An entity that receives a stream of bytes in this format can use the chi digest to validate the content. That value might be provided via some alternative communication channel; one method for carrying this hash value in HTTP fields is defined in Section 4.2.

Without the chi digest, it is not possible to validate the integrity of the content. Content can still be processed by removing the length and hash bytes.

The process for progressively validating and releasing content bytes proceeds as follows, starting with an expected hash value set to the chi digest:

  1. Initialize a new instance of SHA-256 [SHA2].

  2. Read a variable-length integer as a length. Update the hash instance with each byte as it is read. Abort with an error if the stream ends before reading a complete value.

  3. Read the indicated number of bytes from the stream. If the length value is 0, read to the end of the stream instead. Update the hash instance with each byte as it is read. Abort with an error if the stream ends before reading this amount.

  4. If the length value is non-zero, read 32 bytes from the stream and assign those bytes to the next hash value. Update the hash instance with each byte as it is read. Abort with an error if the stream ends before reading this amount.

  5. Finalize the hash and generate hash output. If the hash output is not identical to the expected hash value, abort with an error.

  6. Release the content bytes.

  7. If the length value is non-zero, set the expected hash value to the next hash value and return to the start. Otherwise, end the process successfully.

This process ensures that released bytes are all integrity-protected, according to the value that was provided.

The Chi-Digest field (Section 4.2) provides only a single hash, but other mechanisms could be defined, such as a future extension to [SRI].

When hashes are obtained via means other than the Chi-Digest field, there is a risk that chiclet layout has changed. A different layout invalidates the hash, so resources should not change the chiclet layout.

To account for variations in how chiclets are constructed (see Section 5.4), content could be matched against one of a set of chi digests. An entity can validate against a set of chi digests by matching the hash of the first chiclet against any value from the provided set. Once the first chiclet is processed, subsequent chiclets are validated against the single hash that was included in that first chiclet.

3.3. Construction

In order to produce hashes, content needs to be processed in reverse order. Content is split into multiple pieces, which are turned into chiclets. These chiclets are hashed, starting from the last, placing the resulting hash from each chiclet at the end of the chiclet that immediately precedes it; see Figure 2.

When constructing content, if the content is processed in a forward direction, chiclets might be written with placeholder values for each hash. Hashes can then be added in reverse order once the final chiclet is hashed.

The construction of chiclets can operate on fixed-sized pieces, or application needs can drive how content is split. For instance, an image that can be displayed at progressively-higher levels of quality can have certain points at which significant quality improvements can be realized. Aligning chiclets to those points might enable better incremental processing. Similarly, a video file might have chiclets aligned to frame boundaries or significant features within frames.

The need to produce the encoding in reverse order makes this format ill-suited to situations where content of arbitrary length is streamed. It is also poorly suited to use with dynamically-generated content, especially where content is latency sensitive, because of the need for two processing passes. It is therefore best suited to static resources where the format can be pregenerated and saved.

3.4. Length Constraints

An implementation that processes input needs to hold an entire chiclet in its memory before validating and releasing it. Implementations that accept untrusted inputs might refuse to support large chiclets to protect against memory exhaustion attacks.

Receivers MUST support chiclets with content lengths of up to 65535 bytes (216-1). The total size of chiclets will be larger as they also contain a length field of up to 8 bytes and non-final chiclets include a 32-byte hash. Encoders MUST NOT generate larger chiclets without knowing, by means outside of this specification, that the receiver supports the size it uses.

Receivers can support chiclets larger than the 65535-byte minimum. A chiclet larger than the supported maximum results in a decode error.

A receiver can validate that the length of a non-final chiclet is acceptable before reading further. In contrast, the final chiclet has a zero-valued length prefix, so receivers need to either rely on other length indicators (such as the HTTP Content-Length field) or abort when received bytes exceed their configured maximum.

The absolute maximum length of a chiclet is determined by the length encoding in SHA-2, which is expressed in bits as a 64-bit value. Consequently, the maximum length is 261-1 bytes, less overheads of 40 bytes for a non-final chiclet and up to 8 bytes for a final chiclet (though this can be a single byte). This is less than the 262-1 length that could otherwise be expressed in a QUIC variable-length integer. Of course, there is little value in having such large chiclets, which directly act against the goal of incremental processing.

An abusive implementation might also seek to generate wasteful work for recipients by constructing many small chiclets. While small chiclets might be necessary for some applications, very small lengths can add significant amounts of overhead. Recipients therefore need to look for abusive patterns, such as receiving a large number of chiclets below a given threshold, rather than have a strict lower bound on chiclet length.

4. HTTP Usage

The use of progressive validation is well-suited to HTTP applications [HTTP] that rely on the streaming and incremental processing of message content.

A content coding is defined to enable progressive decoding while maintaining integrity; see Section 4.1. A header field is defined to carry a chi digest; see Section 4.2.

4.1. Content Coding

The chi content coding indicates that the representation is encoded as a sequence of chiclets, following the construction defined in Section 3. Encoding follows the guidance in Section 3.3; decoding follows the procedure in Section 3.2.

A decoder can read a message that uses chi content coding without a target digest, though that provides no integrity guarantee.

A client can indicate that it will accept chi content coding in responses using the Accept-Encoding field. A client can attempt to use chi content coding on requests, which could result in an error response that carries Accept-Encoding if it is not understood by the server; see Section 12.5.3 of [HTTP] for details.

Intermediaries can check, add, or remove chi content coding at their discretion. Adding the coding requires buffering of much of the message before the hash for any non-final chiclet can be sent; see Section 3.3. Intermediaries MUST update any Chi-Digest field when adding or removing chi codings; see Section 4.2.

Like other content codings, the chi content coding applies to the entirety of a representation, so it is less useful in combination with range requests (Section 14 of [HTTP]). A client could still benefit from the use of the coding for a series of range requests if they are contiguous with the first byte of the representation.

Compression content coding, such as gzip or br, can be applied in combination with chi in any order. Applying integrity after compression ensures that invalid content does not need to be decompressed; whereas applying integrity before compression allows compression to be negotiated without needing fresh integrity values.

4.2. Chi-Digest Field

The Chi-Digest field is a sf-list header field; see Section 3.1 of [FIELDS]. Each list item is a byte sequence; see Section 3.3.5 of [FIELDS].

This field carries only chi digests, each value being 32 bytes. Values with other types or lengths MUST be treated as errors, causing the entire field value to be ignored, as recommended by Section 2.2 of [FIELDS]. No semantics are defined for parameters, which MUST be ignored on receipt, as recommended by Section 2.3 of [FIELDS].

Each digest in the Chi-Digest field corresponds to a chi content coding in the order that they are applied (this follows the same rule as content codings; see Section 8.4 of [HTTP]). That means that a recipient uses the last Chi-Digest value when removing the outermost chi coding.

Use of the Chi-Digest field is not mandatory when using the chi content coding. However, if the Chi-Digest field is present when adding an additional chi content coding, the field MUST either be removed or have a value added. This prevents existing values from being invalidated. An intermediary that removes a chi coding MUST remove the last Chi-Digest value if the field is present.

This field offers no means of negotiating different hash algorithms; see Section 5.1.

The Chi-Digest field can be carried in a trailer field, but this is strictly worse than a Repr-Digest field with a sha-256 digest, as defined in [DIGEST]. If the chi digest is only delivered in a trailer, progressive validation cannot be performed.

4.3. Example

This example shows a 47-byte sequence split into chiclets of 16 bytes. This chiclet size is far smaller than would be used in practice, but it allows the example to show multiple chiclets. The resulting 3 chiclets are shown in Figure 3, with the hash of each, starting from the last.

Chiclet 3:
  Length: 0
  Content: " the next one.\n"
  SHA-256 of Chiclet 3:
    1ea4150d 2454faec 9e8b2993 aac8ef95
    15477606 7c59005c 57f00583 81018b09

Chiclet 2:
  Length: 16
  Content: "ries the hash of"
  Hash of Chiclet 3:
    1ea4150d 2454faec 9e8b2993 aac8ef95
    15477606 7c59005c 57f00583 81018b09
  SHA-256 of Chiclet 2:
    c7156912 ad30c2a3 910cc3d8 06ed2318
    aa25474f ff11f6a8 9efb6e6d 4ff7fe84

Chiclet 1:
  Length: 16
  Content: "Each chiclet car"
  Hash of Chiclet 2:
    c7156912 ad30c2a3 910cc3d8 06ed2318
    aa25474f ff11f6a8 9efb6e6d 4ff7fe84
  SHA-256 of Chiclet 1:
    e3bb01bc b8baa8d0 9010fb28 66b73d27
    d8ff408a b74a7bc7 2af12be4 464f94d7
Figure 3: Example Chiclets

The chi digest can be carried in a Chi-Digest field, as shown in Figure 4.

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Encoding: chi
Content-Length: 114
Chi-Digest: :47sBvLi6qNCQEPsoZrc9J9j/QIq3SnvHKvEr5EZPlNc=:
Figure 4: Example Response Header

The 114 bytes of response content are shown in Figure 5 in hexadecimal.

10                                    # Length = 16
45616368 20636869 636c6574 20636172   # "Each chiclet car"
c7156912 ad30c2a3 910cc3d8 06ed2318   # Hash of Chiclet 2
aa25474f ff11f6a8 9efb6e6d 4ff7fe84
10                                    # Length = 16
72696573 20746865 20686173 68206f66   # "ries the hash of"
1ea4150d 2454faec 9e8b2993 aac8ef95   # Hash of Chiclet 3
15477606 7c59005c 57f00583 81018b09
00                                    # Length = 0
20746865 206e6578 74206f6e 652e0a     # " the next one.\n"
Figure 5: Example Encoded Content

Empty content is encoded as a single final chiclet, comprising a single byte with a value of 0. The chi digest of a zero-length sequence is therefore the SHA-256 hash of that one byte, as shown in Figure 6.

Content-Encoding: chi
Content-Length: 1
Chi-Digest: :bjQLnP+zepicpUTmu3gKLHiQHT+zNzh2hRGjBhevoB0=:
Figure 6: Fields for Empty Content

Note that these examples use the minimal encoding for the variable-length integers. Longer encodings for those integers would produce different digest values.

5. Security Considerations

This coding provides integrity protection equivalent to hashing the message content. However, there are subtle differences that need to be noted.

In general, integrity is only valuable if the chi digest comes from a source that also has integrity protection. A chi digest might be provided alongside a reference to a resource (such as with [SRI]) or it might be signed (for instance, an HTTP Message Signature [HTTPSIG] that covers a Chi-Digest field).

The integrity property depends on the hash function chosen (Section 5.1). Recipients are exposed to a risk of message truncation (Section 5.2). There is also a resource exhaustion exposure, particularly for recipients with limited resources (Section 5.3). Finally, the digest value produced is not canonical (Section 5.4).

5.1. Hash Choice

Integrity protection relies on the second pre-image resistance property of SHA-256 [SHA2]. If a recipient holds a chi digest, an attacker that wishes to substitute content needs to find a second sequence of chiclets that will produce the same chi digest.

If the digest is subsequently signed, the collision resistance of SHA-256 is also relied on to prevent message substitution.

This document does not define a way to negotiate an alternative hash function. If attacks on SHA-256 are considered an unacceptable risk, new codings and fields can be defined in a new specification. Whether those items reuse the construction in this document with a different hash function can be considered.

5.2. Truncation

The use of a streaming format means that data might be released to an application for processing, with no guarantee that subsequent chiclets will be delivered intact. Any application that uses this format therefore needs to be robust in the event that its input is truncated.

The final chiclet contains a zero-valued length prefix. The prefix of every chiclet is hashed, so a value of zero ensures that the final chiclet is easily distinguished from any non-final chiclets. This ensures that truncation can be reliably detected.

5.3. Chiclet Size Denial of Service

The requirement to hold an entire chiclet until it is validated creates a denial of service risk; see Section 3.4 for discussion and mitigations.

5.4. Non-Canonical

The same sequence of bytes can produce many different integrity values. As a result, the chi digest cannot act as a canonical identifier for content.

Each digest value commits to a specific sequence of chiclets, so cannot be used with a different arrangement of chiclet lengths. Accepting one of a set of values to safeguard against chiclet layout changes increases the chance of a pre-image attack by a negligible amount for each accepted chi digest.

6. IANA Considerations

This document registers a chi content coding and a header field for carrying chi digests.

6.1. Content Coding

IANA is requested to add the following entry to the "HTTP Content Coding Registry" established by Section 16.6.1 of [HTTP]:

Name:

chi

Description:

Chained Hash Integrity Coding

Reference:

Section 4.1 of this document

6.2. Chi-Digest Header Field

IANA is requested to add the following entry to the "Hypertext Transfer Protocol (HTTP) Field Name Registry" established by Section 16.3.1 of [HTTP]:

Field Name:

Chi-Digest

Status:

permanent

Structured Type:

List

Reference:

Section 4.2 of this document

Comments:

(none)

7. References

7.1. Normative References

[FIELDS]
Nottingham, M. and P. Kamp, "Structured Field Values for HTTP", RFC 9651, DOI 10.17487/RFC9651, , <https://www.rfc-editor.org/rfc/rfc9651>.
[HTTP]
Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Semantics", STD 97, RFC 9110, DOI 10.17487/RFC9110, , <https://www.rfc-editor.org/rfc/rfc9110>.
[QUIC]
Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, DOI 10.17487/RFC9000, , <https://www.rfc-editor.org/rfc/rfc9000>.
[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/rfc/rfc2119>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/rfc/rfc8174>.
[SHA2]
Eastlake 3rd, D. and T. Hansen, "US Secure Hash Algorithms (SHA and SHA-based HMAC and HKDF)", RFC 6234, DOI 10.17487/RFC6234, , <https://www.rfc-editor.org/rfc/rfc6234>.

7.2. Informative References

[DIGEST]
Polli, R. and L. Pardue, "Digest Fields", RFC 9530, DOI 10.17487/RFC9530, , <https://www.rfc-editor.org/rfc/rfc9530>.
[HTTPSIG]
Backman, A., Ed., Richer, J., Ed., and M. Sporny, "HTTP Message Signatures", RFC 9421, DOI 10.17487/RFC9421, , <https://www.rfc-editor.org/rfc/rfc9421>.
[SRI]
Braun, F., "Subresource Integrity", , <https://w3c.github.io/webappsec-subresource-integrity/>.

Acknowledgments

Jeffrey Yasskin was a co-author on a previous iteration of this work. Lucas Pardue suggested the Chi-Digest field, rather than an integration with RFC 9530.

Author's Address

Martin Thomson
Mozilla