Window Sizing for Zstandard Content Encoding
draft-jaju-httpbis-zstd-window-size-00

HTTPBIS                                                          N. Jaju
Internet-Draft                                                    Google
Intended status: Informational                           W. F. P. Handte
Expires: 5 September 2024                           Meta Platforms, Inc.
                                                            4 March 2024

              Window Sizing for Zstandard Content Encoding
                 draft-jaju-httpbis-zstd-window-size-00

Abstract

   Deployments of Zstandard, or "zstd", can use different window sizes
   to limit memory usage during compression and decompression.  Some
   browsers and user agents limit window sizes to mitigate memory usage
   concerns, causing interoperability issues.  This document updates the
   window size limit in RFC8878 from a recommendation to a requirement
   in HTTP contexts.

About This Document

   This note is to be removed before publishing as an RFC.

   The latest revision of this draft can be found at
   https://nidhijaju.github.io/draft-zstd-window-size/draft-jaju-
   httpbis-zstd-window-size.html.  Status information for this document
   may be found at https://datatracker.ietf.org/doc/draft-jaju-httpbis-
   zstd-window-size/.

   Discussion of this document takes place on the HTTPBIS Working Group
   mailing list (mailto:ietf-http-wg@w3.org), which is archived at
   https://lists.w3.org/Archives/Public/ietf-http-wg/.

   Source for this draft and an issue tracker can be found at
   https://github.com/nidhijaju/draft-zstd-window-size.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

Jaju & Handte           Expires 5 September 2024                [Page 1]
Internet-Draft              Zstd Window Size                  March 2024

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 5 September 2024.

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Conventions and Definitions . . . . . . . . . . . . . . . . .   3
   3.  Window Size . . . . . . . . . . . . . . . . . . . . . . . . .   3
   4.  Security Considerations . . . . . . . . . . . . . . . . . . .   4
   5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   4
     5.1.  Content Encoding  . . . . . . . . . . . . . . . . . . . .   4
   6.  Normative References  . . . . . . . . . . . . . . . . . . . .   4
   Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .   4
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   4

1.  Introduction

   Zstandard, or "zstd", specified in [RFC8878], is a lossless data
   compression mechanism similar to gzip.  When used with HTTP, the
   "zstd" Content Encoding token signals to the decoder that the content
   is Zstandard-compressed.

   Deployments of Zstandard can use different window sizes to configure
   the maximum memory a decoder requires to decompress a frame.  Larger
   window sizes tend to improve the compression ratio, but cause more
   memory to be used.  [RFC8878] provides a recommendation for decoders
   to support window sizes up to 8 MB, and for encoders to not generate
   frames requiring window sizes larger than 8 MB.  However, it is just
   a recommendation ([RFC8878], Section 3.1.1.1.2).

Jaju & Handte           Expires 5 September 2024                [Page 2]
Internet-Draft              Zstd Window Size                  March 2024

   To protect against unreasonable memory usage, some browsers and user
   agents limit the maximum window size allowed.  This causes
   incompatibilities if the content is compressed with a larger limit,
   leading to decreased interoperability.

   This document updates [RFC8878] to specify a window size limit
   associated with the "zstd" Content Encoding token.

2.  Conventions and Definitions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in
   BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

3.  Window Size

   Section 3.1.1.1.2 of [RFC8878] discusses window sizes in Zstandard.
   The window size provides guarantees about the minimum memory buffer
   required to decompress a frame.  This information is important for
   decoders to allocate enough memory.

   The minimum window size is 1 KB.  The maximum window size is (1<<41)
   + 7*(1<<38) bytes, which is 3.75 TB.

   In general, larger window size values tend to improve the compression
   ratio, but at the cost of increased memory usage.

   To properly decode compressed data, a decoder will need to allocate a
   buffer of at least the window size bytes.

   In order to protect decoders from unreasonable memory requirements, a
   decoder is allowed to reject a compressed frame that requests a
   memory size beyond the decoder's authorized range.

   To maintain interoperability of Zstandard in HTTP Content Encoding,
   decoders MUST support window sizes of up to and including 8 MB and
   encoders MUST NOT generate frames requiring a window size of larger
   than 8 MB, when using the "zstd" Content Encoding token (see
   Section 5.1).

   Decoders are free to support higher or lower limits, depending on
   local limitations, if negotiated out-of-band.  Many deployments of
   Zstandard operate in controlled, private environments and can
   directly communicate with their encoder and decoder to negotiate a
   higher or lower limit.

Jaju & Handte           Expires 5 September 2024                [Page 3]
Internet-Draft              Zstd Window Size                  March 2024

4.  Security Considerations

   This document introduces no new security considerations beyond those
   discussed in [RFC8878].

5.  IANA Considerations

5.1.  Content Encoding

   This document updates the entry added in [RFC8878] to the "HTTP
   Content Coding Registry" within the "Hypertext Transfer Protocol
   (HTTP) Parameters" registry:

   Name:  zstd

   Description:  A stream of bytes compressed using the Zstandard
      protocol with a window size of not more than 8 MB.

   Reference:  This document

6.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/rfc/rfc2119>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.

   [RFC8878]  Collet, Y. and M. Kucherawy, Ed., "Zstandard Compression
              and the 'application/zstd' Media Type", RFC 8878,
              DOI 10.17487/RFC8878, February 2021,
              <https://www.rfc-editor.org/rfc/rfc8878>.

Acknowledgments

   zstd was developed by Yann Collet.

   The authors would like to thank Yann Collet, Klaus Post, Adam Rice,
   and members of the Web Performance Working Group in the W3C for
   collaborating on the window size issue and helping to formulate a
   solution.  Also, thank you to Nick Terrell for providing feedback
   that went into RFC 8478 and RFC 8878.

Authors' Addresses

Jaju & Handte           Expires 5 September 2024                [Page 4]
Internet-Draft              Zstd Window Size                  March 2024

   Nidhi Jaju
   Google
   Shibuya Stream, 3 Chome-21-3 Shibuya, Shibuya City, Tokyo
   150-0002
   Japan
   Email: nidhijaju@google.com

   W. Felix P. Handte
   Meta Platforms, Inc.
   380 W 33rd St
   New York, NY 10001,
   United States of America
   Email: felixh@meta.com

Jaju & Handte           Expires 5 September 2024                [Page 5]