Zstandard Compression and the application/zstd Media Type
draft-kucherawy-rfc8478bis-05

Document Type Active Internet-Draft (individual in art area)
Authors Yann Collet  , Murray Kucherawy 
Last updated 2020-10-08 (latest revision 2020-04-23)
Stream IETF
Intended RFC status Informational
Formats plain text xml pdf htmlized (tools) htmlized bibtex
Reviews
Stream WG state (None)
Document shepherd Subodh Iyengar
Shepherd write-up Show (last changed 2019-12-19)
IESG IESG state RFC Ed Queue
Consensus Boilerplate Yes
Telechat date
Responsible AD Barry Leiba
Send notices to Subodh Iyengar <subodh@fb.com>
IANA IANA review state Version Changed - Review Needed
IANA action state RFC-Ed-Ack
RFC Editor RFC Editor state IESG
Details
Network Working Group                                          Y. Collet
Internet-Draft                                         M. Kucherawy, Ed.
Obsoletes: 8478 (if approved)                                   Facebook
Intended status: Informational                            April 23, 2020
Expires: October 25, 2020

       Zstandard Compression and the application/zstd Media Type
                     draft-kucherawy-rfc8478bis-05

Abstract

   Zstandard, or "zstd" (pronounced "zee standard"), is a data
   compression mechanism.  This document describes the mechanism and
   registers a media type and content encoding to be used when
   transporting zstd-compressed content via Multipurpose Internet Mail
   Extensions (MIME).  It also registers a corresponding media type,
   content encoding, and structured syntax suffix.

   Despite use of the word "standard" as part of its name, readers are
   advised that this document is not an Internet Standards Track
   specification; it is being published for informational purposes only.

   This document replaces and obsoletes RFC 8478.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on October 25, 2020.

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal

Collet & Kucherawy      Expires October 25, 2020                [Page 1]
Internet-Draft              application/zstd                  April 2020

   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Collet & Kucherawy      Expires October 25, 2020                [Page 2]
Internet-Draft              application/zstd                  April 2020

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
   2.  Definitions  . . . . . . . . . . . . . . . . . . . . . . . . .  4
   3.  Compression Algorithm  . . . . . . . . . . . . . . . . . . . .  5
     3.1.  Frames . . . . . . . . . . . . . . . . . . . . . . . . . .  6
       3.1.1.  Zstandard Frames . . . . . . . . . . . . . . . . . . .  6
         3.1.1.1.  Frame Header . . . . . . . . . . . . . . . . . . .  7
         3.1.1.2.  Blocks . . . . . . . . . . . . . . . . . . . . . . 12
         3.1.1.3.  Compressed Blocks  . . . . . . . . . . . . . . . . 14
         3.1.1.4.  Sequence Execution . . . . . . . . . . . . . . . . 27
         3.1.1.5.  Repeat Offsets . . . . . . . . . . . . . . . . . . 28
       3.1.2.  Skippable Frames . . . . . . . . . . . . . . . . . . . 28
   4.  Entropy Encoding . . . . . . . . . . . . . . . . . . . . . . . 29
     4.1.  FSE  . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
       4.1.1.  FSE Table Description  . . . . . . . . . . . . . . . . 30
     4.2.  Huffman Coding . . . . . . . . . . . . . . . . . . . . . . 33
       4.2.1.  Huffman Tree Description . . . . . . . . . . . . . . . 33
         4.2.1.1.  Huffman Tree Header  . . . . . . . . . . . . . . . 35
         4.2.1.2.  FSE Compression of Huffman Weights . . . . . . . . 36
         4.2.1.3.  Conversion from Weights to Huffman Prefix Codes  . 36
       4.2.2.  Huffman-Coded Streams  . . . . . . . . . . . . . . . . 37
   5.  Dictionary Format  . . . . . . . . . . . . . . . . . . . . . . 39
   6.  Use of Dictionaries  . . . . . . . . . . . . . . . . . . . . . 40
   7.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 40
     7.1.  The 'application/zstd' Media Type  . . . . . . . . . . . . 40
     7.2.  Content Encoding . . . . . . . . . . . . . . . . . . . . . 41
     7.3.  Structured Syntax Suffix . . . . . . . . . . . . . . . . . 42
     7.4.  Dictionaries . . . . . . . . . . . . . . . . . . . . . . . 42
   8.  Security Considerations  . . . . . . . . . . . . . . . . . . . 42
   9.  Implementation Status  . . . . . . . . . . . . . . . . . . . . 43
   10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Show full document text