Zstandard Compression and The application/zstd Media Type
draft-kucherawy-dispatch-zstd-01

Document Type Active Internet-Draft (individual in art area)
Last updated 2017-11-12
Stream IETF
Intended RFC status Informational
Formats plain text xml pdf html bibtex
Stream WG state (None)
Document shepherd No shepherd assigned
IESG IESG state AD is watching
Consensus Boilerplate Unknown
Telechat date
Responsible AD Alexey Melnikov
Send notices to (None)
Network Working Group                                          Y. Collet
Internet-Draft                                         M. Kucherawy, Ed.
Intended status: Standards Track                                Facebook
Expires: May 16, 2018                                  November 12, 2017

       Zstandard Compression and The application/zstd Media Type
                    draft-kucherawy-dispatch-zstd-01

Abstract

   Zstandard, or "zstd" (pronounced "zee standard"), is a data
   compression mechanism.  This document describes the mechanism, and
   registers a media type to be used when transporting zstd-compressed
   via Multipurpose Internet Mail Extensions (MIME).

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on May 16, 2018.

Copyright Notice

   Copyright (c) 2017 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Collet & Kucherawy        Expires May 16, 2018                  [Page 1]
Internet-Draft              application/zstd               November 2017

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Compression Algorithm  . . . . . . . . . . . . . . . . . . . .  3
     2.1.  Frames . . . . . . . . . . . . . . . . . . . . . . . . . .  3
       2.1.1.  Zstandard Frames . . . . . . . . . . . . . . . . . . .  3
         2.1.1.1.  Frame Header . . . . . . . . . . . . . . . . . . .  4
         2.1.1.2.  Blocks . . . . . . . . . . . . . . . . . . . . . .  8
         2.1.1.3.  Compressed Blocks  . . . . . . . . . . . . . . . . 10
     2.2.  Sequence Execution . . . . . . . . . . . . . . . . . . . . 22
       2.2.1.  Repeat Offsets . . . . . . . . . . . . . . . . . . . . 23
     2.3.  Skippable Frames . . . . . . . . . . . . . . . . . . . . . 23
     2.4.  Entropy Encoding . . . . . . . . . . . . . . . . . . . . . 24
       2.4.1.  FSE  . . . . . . . . . . . . . . . . . . . . . . . . . 24
         2.4.1.1.  FSE Table Description  . . . . . . . . . . . . . . 25
       2.4.2.  Huffman Coding . . . . . . . . . . . . . . . . . . . . 27
         2.4.2.1.  Huffman Tree Description . . . . . . . . . . . . . 28
         2.4.2.2.  Huffman-coded Streams  . . . . . . . . . . . . . . 32
     2.5.  Dictionary Format  . . . . . . . . . . . . . . . . . . . . 33
   3.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 34
     3.1.  The 'application/zstd' Media Type  . . . . . . . . . . . . 34
     3.2.  Content Encoding . . . . . . . . . . . . . . . . . . . . . 35
   4.  Security Considerations  . . . . . . . . . . . . . . . . . . . 36
   5.  Implementation Status  . . . . . . . . . . . . . . . . . . . . 36
   6.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 37
     6.1.  Normative References . . . . . . . . . . . . . . . . . . . 37
     6.2.  Informative References . . . . . . . . . . . . . . . . . . 38
   Appendix A.  Acknowledgments . . . . . . . . . . . . . . . . . . . 38
   Appendix B.  Decoding Tables for Predefined Codes  . . . . . . . . 39
     B.1.  Literal Length Code Table  . . . . . . . . . . . . . . . . 39
     B.2.  Match Length Code Table  . . . . . . . . . . . . . . . . . 42
     B.3.  Offset Code Table  . . . . . . . . . . . . . . . . . . . . 44

Collet & Kucherawy        Expires May 16, 2018                  [Page 2]
Internet-Draft              application/zstd               November 2017

1.  Introduction

   Zstandard, or "zstd" (pronounced "zee standard") is a data
   compression mechanism, akin to gzip [RFC1952].
Show full document text