Brotli Compressed Data Format
draft-alakuijala-brotli-11

[Ballot comment]
For what it is worth, I do believe words in the document to the effect that the specification is intended to be complete and sufficient for implementing the format would be helpful.

[Ballot discuss]
Thank you for writing this specification. I am looking forward to recommending its approval soon.

However, before doing so I would like to understand what the answer is to Paul Kyzivat's question regarding being an authoritative specification:

It is unclear to me whether this document seeks to be an authoritative specification of the Brotli format, or simply an explanation of the format the supplements another specification.

Do the authors have an answer?

[Ballot comment]
Note for IESG: The reason why IESG is reviewing this document is because the IANA registration requires IETF Consensus RFC.

Thank you for addressing my DISCUSS points.

[Ballot comment]

- The IPR declaration confused me a bit. Do we know if it
is saying that any implementation of the RFC for any
purpose is RF or if it only means that RF applies to W3C
stuff? (What's confusing is the last bit where it says
the company "provides a W3C RF commitment" which is just
a bit of an odd thing to see in an IETF IPR declaration.)
Note: I think this is ok, as-is, but it's not entirely
clear to me. If the folks who made the IPR declaration
wanted to clarify that might be good, but I don't think
it's absolutely needed.

- The security considerations could do with some more
references, at least one to a zip bomb and one to the
crime attack. I think those would help some
readers/implementers get the issues better.

- I think it'd have been good to add a section on when
the authors think it's a good and bad idea to use this
algorithm.  For example, would it make sense to ever use
this for arbitrary (but not random) binary data? I don't
mean a full analysis, but just some basic guidance. Not a
big deal, but including that might avoid some repetition
later on, esp if there are places where we know now that
one ought not use this.

- I agree with Alexey's discuss point #1.

[Ballot discuss]
Please discuss or address the following points with me before I can vote Yes on this document:

1)

Page 28: UTF-8 upper-cased - UTF-8 needs a normative reference. Did you mean ASCII uppercasing, Unicode uppercasing or something else? This also needs a normative reference to UTF-8 (RFC 3629).

If you really meant Unicode uppercasing, then how can I verify that the algorithm specified in code in the same section is correct? Is it described in one of Unicode documents somewhere?

2) This is a nit, but I think an important one:

In Section 11:

As with any compressed file formats, decompressor implementations should handle all compressed data byte sequences, not only those that conform to this specification, where non-conformant compressed data sequences should be discarded.

Discarded? Does this mean ignoring data and just continuing or rejecting the whole payload?

[Ballot comment]
Note for IESG: The reason why IESG is reviewing this document is because the IANA registration requires IETF Consensus RFC.

In 7.1:

The lengths and the zlib CRC-32 (ITU-T Recommendation V.42)

Can you please add a Normative Reference for this?

[Ballot comment]
Two valid questions asked by Dan Romascanu in his OPS DIR review:

I believe the document is 'Almost Ready' for publication. I am no expert in compression, but my impression is that the document is clear and precise in defining the algorithm and the flow of operations on compression or decompression. From an operational point of view I have however two reservations:

-          There is no justification of any kind (mathematical proof, measurement results, etc.) that supports the claims that the performance resulting from the application of the algorithms results in ‘with an efficiency comparable to the best currently available general-purpose compression methods’. Even so, if the resulting performance is only ‘comparable’ with existing methods, why define a new one and publish it in an RFC?

-          There is no information or recommendations about the applicability of this format (what type of platforms, or network protocols) or documented results of its deployment – when should a software developer use it? Where should a network operator or IT manager apply it or activate it if available?

(Via drafts-lastcall-comment@iana.org): IESG/Authors/WG Chairs:

IANA has completed its review of draft-alakuijala-brotli-08.txt. If any part of this review is inaccurate, please let us know.

IANA understands that, upon approval of this document, there is a single action which IANA must complete.

In the HTTP Content Coding Registry in the Hypertext Transfer Protocol (HTTP) Parameters registry located at:

https://www.iana.org/assignments/http-parameters/

a single, new value will be registered as follows:

Name: br
Description: Brotli Compressed Data Format
Reference: [ RFC-to-be ]

IANA understands that this is the only action required to be completed upon approval of this document.

Note:  The actions requested in this document will not be completed until the document has been approved for publication as an RFC. This message is only to confirm what actions will be performed. 


Thank you,

Sabrina Tanamal
IANA Specialist
ICANN

RFC4858-style writeup for draft-alakuijala-brotli-08

Title: "Brotli Compressed Data Format"

Document Shepherd:  Nevil Brownlee (Independent SUbmissions Editor)

Intent (Abstract):
  "This specification defines a lossless compressed data format that
  compresses data using a combination of the LZ77 algorithm and Huffman
  coding, with efficiency comparable to the best currently available
  general-purpose compression methods."

Intended RFC Status:  Informational

Discussion:
  The brotli draft's authors, Zoltan Szabadka and Jyrki Alakuijala,
  submitted it to the Independent Stream in April 2014.  It took me
  rather a long time to find suitable reviewers for a new compression
  technique, and that wasn't helped by its authors not prodding me
  until February this year.  However, I finally got a brief review
  of it from Ulrich Speidel(U Auckland, a coding throry researcher),
  and a more thorough review
  from Jean-Loup Gailly (the author of gzip); I sent it to Zoltan
  on 12 February, and told him that once he's made changes in response
  to that review he shouldpublish its -09 version, then it will be
  ready for an IESG Conflict Review.  In other words, it was now ready
  for publication in the Independent Stream.

  On 11 March 2016 Mark Nottingham emailed me to say that W3C would
  like to see it published, however it requests IANA to assign an
  HTTP Content Coding, but that Registry requires IETF Review.  To
  makwe that happen, Barry Leiba has agreed to sponsor it as as
  Individual draft; we've moved it into the IETF Stream.

IPR:
  There is one IPR decalaration on Tracker for this, it was made by
  Jyrki Alakuijal (one of its authors): Google holds a patent for it.

Other Points:
  This draft has IANA Considerations, as pointed out above.
  It is rather long - 120 pages, but 80 of those pages are in Appendix
  A, "Static dictionary data", and will not need editing!

ISE write-up for: draft-alakuijala-brotli-08

Abstract:
  "This specification defines a lossless compressed data format that
  compresses data using a combination of the LZ77 algorithm and Huffman
  coding, with efficiency comparable to the best currently available
  general-purpose compression methods."


It was reviewed for me by Ulrich Speidel (U Auckland, a coding theory
researcher), and by Jean-Loup Gailly (the author of gzip), its author
has made the changes discussed between them.

This draft has IANA Considerations, it asks for an entry in the
HTTP Content Coding Registry.  IANA will need to review that.

- - - - - - -

ISE write-up for: draft-alakuijala-brotli-08

Abstract:
  "This specification defines a lossless compressed data format that
  compresses data using a combination of the LZ77 algorithm and Huffman
  coding, with efficiency comparable to the best currently available
  general-purpose compression methods."


It was reviewed for me by Ulrich Speidel (U Auckland), and by Jen-Loup
Gailly (the author of gzip), its author has made the changes discussed
between them.

This draft has IANA Considerations, it asks for an entry in the
HTTP Content Coding Registry.  IANA will need to review that.

- - - - - - -

ISE write-up for: draft-alakuijala-brotli-08

Abstract:
  "This document describes a procedure for when an Military Message
  Handling System (MMHS) message is composed by one user and is only
  released to the mail transfer system when one or more authorizing
  users authorize release of the message by adding the MMHS-
  Authorizing-Users header field.  The resulting message can be
  optionally signed by the sender and/or reviewer, allowing recipients
  to verify both the original signature (if any) and review signatures."


It was reviewed for me by Russ Housley and Sean Turner, its author
has made the changes discussed between them.

This draft has IANA Considerations, it asks for a new 'Provisional
Message Header Field Name, IANA have reviewed that request.

- - - - - - -

The following Last Call announcement was sent out:

From: The IESG
To: "IETF-Announce"
CC: draft-alakuijala-brotli@ietf.org, n.brownlee@auckland.ac.nz, "HTTP Working Group" , barryleiba@gmail.com, "Nevil Brownlee"
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Brotli Compressed Data Format) to Informational RFC


The IESG has received a request from an individual submitter to consider
the following document:
- 'Brotli Compressed Data Format'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2016-04-08. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract

  This specification defines a lossless compressed data format that
  compresses data using a combination of the LZ77 algorithm and Huffman
  coding, with efficiency comparable to the best currently available
  general-purpose compression methods.

This document was originally submitted to the Independent Stream Editor,
and has been reviewed there.  It was switched to the IETF Stream because
it requests a registration in the HTTP Content Coding Registry, which has
a registration policy of IETF Review.  The document is needed as a normative
reference by W3C.

The file can be obtained via
https://datatracker.ietf.org/doc/draft-alakuijala-brotli/

When IESG discussion begins, it can be tracked via
https://datatracker.ietf.org/doc/draft-alakuijala-brotli/ballot/

The following IPR Declarations may be related to this I-D:
  https://datatracker.ietf.org/ipr/2396/