Zstandard Compression and the 'application/zstd' Media Type
RFC 8878

Document Type RFC - Informational (February 2021; Errata)
Obsoletes RFC 8478
Was draft-kucherawy-rfc8478bis (individual in art area)
Authors Yann Collet  , Murray Kucherawy 
Last updated 2021-02-26
Stream Internent Engineering Task Force (IETF)
Formats plain text html xml pdf htmlized (tools) htmlized bibtex
Reviews
Stream WG state (None)
Document shepherd Subodh Iyengar
Shepherd write-up Show (last changed 2019-12-19)
IESG IESG state RFC 8878 (Informational)
Action Holders
(None)
Consensus Boilerplate Yes
Telechat date
Responsible AD Barry Leiba
Send notices to Subodh Iyengar <subodh@fb.com>
IANA IANA review state IANA OK - Actions Needed
IANA action state RFC-Ed-Ack


Internet Engineering Task Force (IETF)                         Y. Collet
Request for Comments: 8878                             M. Kucherawy, Ed.
Obsoletes: 8478                                                 Facebook
Category: Informational                                    February 2021
ISSN: 2070-1721

      Zstandard Compression and the 'application/zstd' Media Type

Abstract

   Zstandard, or "zstd" (pronounced "zee standard"), is a lossless data
   compression mechanism.  This document describes the mechanism and
   registers a media type, content encoding, and a structured syntax
   suffix to be used when transporting zstd-compressed content via MIME.

   Despite use of the word "standard" as part of Zstandard, readers are
   advised that this document is not an Internet Standards Track
   specification; it is being published for informational purposes only.

   This document replaces and obsoletes RFC 8478.

Status of This Memo

   This document is not an Internet Standards Track specification; it is
   published for informational purposes.

   This document is a product of the Internet Engineering Task Force
   (IETF).  It represents the consensus of the IETF community.  It has
   received public review and has been approved for publication by the
   Internet Engineering Steering Group (IESG).  Not all documents
   approved by the IESG are candidates for any level of Internet
   Standard; see Section 2 of RFC 7841.

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at
   https://www.rfc-editor.org/info/rfc8878.

Copyright Notice

   Copyright (c) 2021 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction
   2.  Definitions
   3.  Compression Algorithm
     3.1.  Frames
       3.1.1.  Zstandard Frames
         3.1.1.1.  Frame Header
         3.1.1.2.  Blocks
         3.1.1.3.  Compressed Blocks
         3.1.1.4.  Sequence Execution
         3.1.1.5.  Repeat Offsets
       3.1.2.  Skippable Frames
   4.  Entropy Encoding
     4.1.  FSE
       4.1.1.  FSE Table Description
     4.2.  Huffman Coding
       4.2.1.  Huffman Tree Description
         4.2.1.1.  Huffman Tree Header
         4.2.1.2.  FSE Compression of Huffman Weights
         4.2.1.3.  Conversion from Weights to Huffman Prefix Codes
       4.2.2.  Huffman-Coded Streams
   5.  Dictionary Format
   6.  Use of Dictionaries
   7.  IANA Considerations
     7.1.  The 'application/zstd' Media Type
     7.2.  Content Encoding
     7.3.  Structured Syntax Suffix
     7.4.  Dictionaries
   8.  Security Considerations
   9.  References
     9.1.  Normative References
     9.2.  Informative References
   Appendix A.  Decoding Tables for Predefined Codes
     A.1.  Literals Length Code Table
     A.2.  Match Length Code Table
     A.3.  Offset Code Table
   Appendix B.  Changes since RFC 8478
   Acknowledgments
   Authors' Addresses

1.  Introduction

   Zstandard, or "zstd" (pronounced "zee standard"), is a data
   compression mechanism, akin to gzip [RFC1952].

   Despite use of the word "standard" as part of its name, readers are
   advised that this document is not an Internet Standards Track
   specification; it is being published for informational purposes only.

   This document describes the Zstandard format.  Also, to enable the
   transport of a data object compressed with Zstandard, this document
   registers a media type, content encoding, and structured syntax
   suffix that can be used to identify such content when it is used in a
   payload.

2.  Definitions

   Some terms used elsewhere in this document are defined here for
   clarity.

   uncompressed:  Describes an arbitrary set of bytes in their original
      form, prior to being subjected to compression.

   compressed:  Describes the result of passing a set of bytes through
      this mechanism.  The original input has thus been compressed.

   decompressed:  Describes the result of passing a set of bytes through
      the reverse of this mechanism.  When this is successful, the
      decompressed payload and the uncompressed payload are
      indistinguishable.

   encode:  The process of translating data from one form to another;
      this may include compression, or it may refer to other
      translations done as part of this specification.

   decode:  The reverse of "encode"; describes a process of reversing a
Show full document text