Extensible Binary Meta Language
RFC 8794

Received changes through RFC Editor sync (created alias RFC 8794, changed abstract to 'This document defines the Extensible Binary Meta Language (EBML) format as a binary container format designed for audio/video storage.  EBML is designed as a binary equivalent to XML and uses a storage-efficient approach to build nested Elements with identifiers, lengths, and values.  Similar to how an XML Schema defines the structure and semantics of an XML Document, this document defines how EBML Schemas are created to convey the semantics of an EBML Document.', changed pages to 51, changed standardization level to Proposed Standard, changed state to RFC, added RFC published event at 2020-07-22, changed IESG state to RFC Published)

[Ballot comment]
The -16 addresses my DISCUSS point; thanks!
I'm given to understand that email discussion of the comments (preserved
below) is forthcoming, but do have one note on the new text in the -16:
In Section 7.3 we imply that a float is 32-bit and 64-bit at the same time;
I think s/and/or/ makes more sense (and, of course, the EBML element
length indicates which one is present).

I support Adam's Discuss regarding the Abstract.

Section 2

  "Parent Element": A relative term to describe the "Master Element"
  which contains a specified element.  For any specified "EBML Element"
  that is not at "Root Level", the "Parent Element" refers to the
  "Master Element" in which that "EBML Element" is contained.

It sounds like this is intended to be "directly" or "immediately"
contained (in order to be unique), right?  If not, then it sould be
''refers to a "Master Element" in which [...]''

Section 4.1

  Each Variable Size Integer begins with a VINT_WIDTH which consists of
  zero or many zero-value bits.  The count of consecutive zero-values
  of the VINT_WIDTH plus one equals the length in octets of the
  Variable Size Integer.  [...]

Does the following attempted rewording change the meaning?

%  Each Variable Size Integer begins with a VINT_WIDTH which consists of
%  zero or more bits set to zero.  The length in octets of the entire
%  Variable Size Integer is determined as one plus the number of
%  consecutive bits set to zero.

(I find the current formulation rather hard to parse.)

Section 6.2

    | "\root\level1\level2\"    | Global Element cannot be  |
    |                                    | assumed to have this path, |
    |                                    | while parsing "elt" it can |
    |                                    | only be a child of "elt"  |

Cannot be assumed by who/what?  My brain is trying to parse this as just
"cannot assume this path".

Section 7.5

Should we say anything about termination of a UTF-8 string needing to
still result in valid UTF-8 (i.e., not insert NULs in the middle of a
codepoint)?

Section 7.7

  stored within Master Elements SHOULD only consist of EBML Elements
  and SHOULD NOT contain any data that is not part of an EBML Element.

When might this SHOULD (NOT) be violated?

Section 8.2

  part of an EBML Element.  This document defines precisely which EBML
  Elements are to be used within the EBML Header, but does not name or

(for EBMLVersion 1 only, right?)

Section 11.1

  Element; for example matroska or webm (see Section 11.2.6).  The
  DocType value for an EBML Document Type MUST be unique and
  persistent.

It might be appropriate to refer to Section 17.2 and/or the IANA
registry for DocType values, here.

  EBMLVersion to only support a value of "1".  If an EBML Schema adopts
  the EBML Header Element as-is, then it is not required to document
  that Element within the EBML Schema.  If an EBML Schema constrains

Does "as-is" imply some level of future-compatibility/extensibility for
when EBMLVersions other than "1" are defined?

Section 11.1.1

It's a little amusing that we bother to provide "default" attributes
when the "range" attribute uniquely determines the allowed value.

Section 11.1.4

  Each "" defines one EBML Element through the use of several
  attributes that are defined in Section 11.1.3.  EBML Schemas MAY

I think this makes more sense as "Section 11.1.5".

Section 11.1.5.2

This ABNF seems to only allow "direct" recursion where element
appears directly inside element , without any intermediate elements.
I assume that's the intent, though it would be surprising in a
general-purpose markup language.

  In some cases the EBMLLastParent part of the path is an
  EBMLGlobalParent.  A path with a EBMLGlobalParent defines a
  Section 11.3.  Any path that starts with the EBMLFixedParent of the

That second sentence doesn't parse.

  As an example, a "path" of "1*(\Segment\Info)" means the element Info
  is found inside the Segment elements at least once and with no
  maximum iteration.  An element SeekHead with path
  "0*2(\Segment\SeekHead)" may not be found at all in its Segment
  parent, once or twice but no more than that.

The way this text is written makes me want to interpret the path
occurence counts more like the (regular) minOccurs/maxOccurs element
attributes, as opposed to applying to the path components to get to the
specific element in question.

Section 11.1.9.2

 
   
      A set of items.

Is this "name" supposed to be "Item" or "Items"?

Section 11.1.10-11.1.12

I'm not sure I have a full understanding of how / are
used; perhaps a reference to the corresponding XML behavior is in order?

Section 11.1.13-11.1.14

The  usage seems underspecified.

Section 11.1.15

     
       
     

Why do we include this commented-out snippet?

     
     

Don't we effectively set default values for these two in the prose
description?

Section 11.1.16

  Identically Recurring Elements SHOULD include a CRC-32 Element as a
  Child Element; this is especially recommended when EBML is used for
  long-term storage or transmission.  If a Parent Element contains more

I'm not sure if the "long-term" is intended to also bind as "long-term
transmission" (though I'm not sure what it would mean in that case).
It's also not entirely clear what kinds of transmission would benefit
from this, as reliable media presumably don't need redundancy for
reliability, but unreliable media can't really be used to carry EBML
without some framing requirements to know when elements start.

Section 11.1.18

  If a Mandatory EBML Element has no default value declared by an EBML
  Schema and its Parent Element is present then the EBML Element MUST
  be present as well.  If a Mandatory EBML Element has a default value
  declared by an EBML Schema and its Parent Element is present and the
  value of the EBML Element is NOT equal to the declared default value
  then the EBML Element MUST be present.

This seems almost tautological, in that how would an EBML Element have a
value if it was not present?  (The following paragraph that talks about
when to write such elements, does make more sense.)

Section 11.3.1

  path: "*1((1*\)\CRC-32)"

Using backslash as both an escape character and a path separator makes
my head hurt, and I did not have enough caffeine yet this morning to
figure it out.

  8.1.1.6.2 of [ITU.V42.1994], with initial value of 0xFFFFFFFF.  The
  CRC value MUST be computed on a little endian bitstream and MUST use
  little endian storage.

bitstream or bytestream?

Section 12

  If a Master Element contains a CRC-32 Element that doesn't validate,
  then the EBML Reader MAY ignore all contained data except for
  Descendant Elements that contain their own valid CRC-32 Element.

Ignoring only part of the known questionable content could have
significant security considerations, if (e.g.) security-relevant
restrictions are in the garbled part of the document but the sensitive
content has a (valid) redundant CRC.

[review terminated early]

[Ballot comment]
I support Adam's Discuss regarding the Abstract.

Section 2

  "Parent Element": A relative term to describe the "Master Element"
  which contains a specified element.  For any specified "EBML Element"
  that is not at "Root Level", the "Parent Element" refers to the
  "Master Element" in which that "EBML Element" is contained.

It sounds like this is intended to be "directly" or "immediately"
contained (in order to be unique), right?  If not, then it sould be
''refers to a "Master Element" in which [...]''

Section 4.1

  Each Variable Size Integer begins with a VINT_WIDTH which consists of
  zero or many zero-value bits.  The count of consecutive zero-values
  of the VINT_WIDTH plus one equals the length in octets of the
  Variable Size Integer.  [...]

Does the following attempted rewording change the meaning?

%  Each Variable Size Integer begins with a VINT_WIDTH which consists of
%  zero or more bits set to zero.  The length in octets of the entire
%  Variable Size Integer is determined as one plus the number of
%  consecutive bits set to zero.

(I find the current formulation rather hard to parse.)

Section 6.2

    | "\root\level1\level2\"    | Global Element cannot be  |
    |                                    | assumed to have this path, |
    |                                    | while parsing "elt" it can |
    |                                    | only be a child of "elt"  |

Cannot be assumed by who/what?  My brain is trying to parse this as just
"cannot assume this path".

Section 7.5

Should we say anything about termination of a UTF-8 string needing to
still result in valid UTF-8 (i.e., not insert NULs in the middle of a
codepoint)?

Section 7.7

  stored within Master Elements SHOULD only consist of EBML Elements
  and SHOULD NOT contain any data that is not part of an EBML Element.

When might this SHOULD (NOT) be violated?

Section 8.2

  part of an EBML Element.  This document defines precisely which EBML
  Elements are to be used within the EBML Header, but does not name or

(for EBMLVersion 1 only, right?)

Section 11.1

  Element; for example matroska or webm (see Section 11.2.6).  The
  DocType value for an EBML Document Type MUST be unique and
  persistent.

It might be appropriate to refer to Section 17.2 and/or the IANA
registry for DocType values, here.

  EBMLVersion to only support a value of "1".  If an EBML Schema adopts
  the EBML Header Element as-is, then it is not required to document
  that Element within the EBML Schema.  If an EBML Schema constrains

Does "as-is" imply some level of future-compatibility/extensibility for
when EBMLVersions other than "1" are defined?

Section 11.1.1

It's a little amusing that we bother to provide "default" attributes
when the "range" attribute uniquely determines the allowed value.

Section 11.1.4

  Each "" defines one EBML Element through the use of several
  attributes that are defined in Section 11.1.3.  EBML Schemas MAY

I think this makes more sense as "Section 11.1.5".

Section 11.1.5.2

This ABNF seems to only allow "direct" recursion where element
appears directly inside element , without any intermediate elements.
I assume that's the intent, though it would be surprising in a
general-purpose markup language.

  In some cases the EBMLLastParent part of the path is an
  EBMLGlobalParent.  A path with a EBMLGlobalParent defines a
  Section 11.3.  Any path that starts with the EBMLFixedParent of the

That second sentence doesn't parse.

  As an example, a "path" of "1*(\Segment\Info)" means the element Info
  is found inside the Segment elements at least once and with no
  maximum iteration.  An element SeekHead with path
  "0*2(\Segment\SeekHead)" may not be found at all in its Segment
  parent, once or twice but no more than that.

The way this text is written makes me want to interpret the path
occurence counts more like the (regular) minOccurs/maxOccurs element
attributes, as opposed to applying to the path components to get to the
specific element in question.

Section 11.1.9.2

 
   
      A set of items.

Is this "name" supposed to be "Item" or "Items"?

Section 11.1.10-11.1.12

I'm not sure I have a full understanding of how / are
used; perhaps a reference to the corresponding XML behavior is in order?

Section 11.1.13-11.1.14

The  usage seems underspecified.

Section 11.1.15

     
       
     

Why do we include this commented-out snippet?

     
     

Don't we effectively set default values for these two in the prose
description?

Section 11.1.16

  Identically Recurring Elements SHOULD include a CRC-32 Element as a
  Child Element; this is especially recommended when EBML is used for
  long-term storage or transmission.  If a Parent Element contains more

I'm not sure if the "long-term" is intended to also bind as "long-term
transmission" (though I'm not sure what it would mean in that case).
It's also not entirely clear what kinds of transmission would benefit
from this, as reliable media presumably don't need redundancy for
reliability, but unreliable media can't really be used to carry EBML
without some framing requirements to know when elements start.

Section 11.1.18

  If a Mandatory EBML Element has no default value declared by an EBML
  Schema and its Parent Element is present then the EBML Element MUST
  be present as well.  If a Mandatory EBML Element has a default value
  declared by an EBML Schema and its Parent Element is present and the
  value of the EBML Element is NOT equal to the declared default value
  then the EBML Element MUST be present.

This seems almost tautological, in that how would an EBML Element have a
value if it was not present?  (The following paragraph that talks about
when to write such elements, does make more sense.)

Section 11.3.1

  path: "*1((1*\)\CRC-32)"

Using backslash as both an escape character and a path separator makes
my head hurt, and I did not have enough caffeine yet this morning to
figure it out.

  8.1.1.6.2 of [ITU.V42.1994], with initial value of 0xFFFFFFFF.  The
  CRC value MUST be computed on a little endian bitstream and MUST use
  little endian storage.

bitstream or bytestream?

Section 12

  If a Master Element contains a CRC-32 Element that doesn't validate,
  then the EBML Reader MAY ignore all contained data except for
  Descendant Elements that contain their own valid CRC-32 Element.

Ignoring only part of the known questionable content could have
significant security considerations, if (e.g.) security-relevant
restrictions are in the garbled part of the document but the sensitive
content has a (valid) redundant CRC.

[review terminated early]

[Ballot discuss]
1. Section 5:

  The Element ID is encoded as a Variable Size Integer.

    +-----------------------+-------------------------+---------------+
    | VINT Length in octets |  Range of Possible IDs  | Number of IDs |
    +=======================+=========================+===============+
    |          1          |      0x81 - 0xFE      |          126 |
    +-----------------------+-------------------------+---------------+
    |          2          |    0x407F - 0x7FFE    |        16,256 |
    +-----------------------+-------------------------+---------------+
    |          3          |  0x203FFF - 0x3FFFFE  |    2,080,768 |
    +-----------------------+-------------------------+---------------+
    |          4          | 0x101FFFFF - 0x1FFFFFFE |  268,338,304 |
    +-----------------------+-------------------------+---------------+

To me it appears that this whole section can't decide if the Element ID is encoded integer using VINT or an VINT format octet sequence that is self describing in length? If it is the first then the above quoted table would to me state that the IDs are 1-126 for 1 octet, and the second two-octet 127-16382. But based on later section it is actually the later. As the ID values defined in Section 11.2 for the various elements are actually the encoded form rather than a representation of the Integer value encoded. This needs to be clarified.

[Ballot comment]
1. Section 5:

Any Element ID with the VINT_DATA
  component set as all zero values or all one values MUST be ignored.

What does it mean to ignore an Element ID in the general case. Can you really say this in the general case, rather than state that it is not a valid Element ID? Or is the intention that an Element ID with all data is the equivalent to padding and simply skipped and a parser needs to expect to find the real Element ID in the next octet sequence? Considering the Element Length that do allow zero values and non-efficient encoding, if the element should be ignored or not depends on the expected element to find by the parser.

I might have missed some later explanation of this, as I didn't manage to read the whole document in detail.

[Ballot comment]
I am trusting my ART Area Directors colleagues for this document. Did a quick browse through and it looks good to me.

Still wondering why the three authors have no affiliation at all...

[Ballot discuss]
Section 7.7 says:

  A Master Element MUST declare a length in octets from zero to
  VINTMAX.  The Master Element MAY also use an unknown length.  See
  Section 6 for rules that apply to elements of unknown length.

but the second sentence contradicts the immediately prior MUST.  We need
to resolve the internal inconsistency.

[Ballot comment]
This was a very difficult read: I found a lot of the document to be convoluted and hard to follow, and I considered balloting Abstain, as I’m not sure that DISCUSS is appropriate for my complaints, but I can’t really say that I have “no objection”.  In the end I decided to go with a “No Objection” ballot, to call out the worst of the issues, and to hope that you will consider the changes I suggest and that others will follow the text more easily than I could.

— Section 4.1 —

  Each Variable Size Integer begins with a VINT_WIDTH which consists of
  zero or many zero-value bits.  The count of consecutive zero-values
  of the VINT_WIDTH plus one equals the length in octets of the
  Variable Size Integer.  For example, a Variable Size Integer that
  starts with a VINT_WIDTH which contains zero consecutive zero-value
  bits is one octet in length and a Variable Size Integer that starts
  with one consecutive zero-value bit is two octets in length.  The
  VINT_WIDTH MUST only contain zero-value bits or be empty.

I found this very hard to follow, and had to read it several times before I understood what you’re getting at.  I found things such as “zero or many zero-value bits” to be confusing.  May I suggest alternative text, which describes the concept and then gets to the details?:

NEW
Each Variable Size Integer begins with a VINT_WIDTH followed by a VINT_MARKER.  VINT_WIDTH is a sequence of zero or more bits of value 0, and is terminated by the VINT_MARKER, which is a single bit of value 1.  The total number of bits (VINT_WIDTH and VINT_MARKER combined) is the number of octets of the Variable Size Integer.

Thus, the single bit “1” describes a Variable Size Integer with a length of one octet.  The sequence of bits “01” describes a Variable Size Integer with a length of two octets.  “001” describes a Variable Size Integer with a length of three octets, and so on, with each additional 0-bit adding one octet to the length of the Variable Size Integer.
END

I, at least, find that easier to follow.  Does it work for you?

For the next paragraph, which limits the length under various circumstances, I suggest putting it in terms of the number of octets in the integer, rather than the number of bits in the VINT_WIDTH, which might be better put into Section 4.3, rather than 4.1.  Text such as, “A Variable Size Integer in an EBML Header can be at most 4 octets long, except [...] , where it can be up to 8 octets long,” is easier to understand than the text explaining limits on the number of bits in VINT_WIDTH.

— Section 4.4 —
Table 2 and the text that introduces it would be better if they talked about the integer that’s represented (2), rather than the binary value (0b10 in the text and 10 in the table), considering that it is a Variable Size INTEGER, yes?

— Section 6.2 —

  An EBML Element with an unknown Element Data Size
  is referred to as an Unknown-Sized Element.  A Master Element MAY be
  an Unknown-Sized Element; however an EBML Element that is not a
  Master Element MUST NOT be an Unknown-Sized Element.  Master Elements
  MUST NOT use an unknown size unless the unknownsizeallowed attribute
  of their EBML Schema is set to true (see Section 11.1.5.10).

This also seems confusing and perhaps contradictory because of how it uses the BCP 14 key words.  May I suggest this, which neither uses nor needs the key words?:

NEW
  An EBML Element with an unknown Element Data Size
  is referred to as an Unknown-Sized Element.  Only a Master Element
  is allowed to be of unknown size, and it can only be so if the
  unknownsizeallowed attribute of its EBML Schema is set to true
  (see Section 11.1.5.10).
END

— Section 7.7 —

  The Master Element MAY also use an unknown length.

The “MAY” isn’t really correct, is it?  There are restrictions that make it not entirely optional, as noted in the next sentence.  I suggest not using BCP 14 here, and just saying, “The Master Element may be of unknown length.”

  The Master Element contains zero, one, or many other elements.

Does this mean anything more than the simpler, “The Master Element contains zero or more other elements.”?  As written, one tends to ask what “many” means here.

— Section 8.2 —

  The EBML Body MUST NOT contain any data that is not
  part of an EBML Element.

Why is this repetition needed?  Doesn’t the similar sentence in Section 8 cover this?

— Section 10 —

  An EBML Document handles 2 different versions: the version of the
  EBML Header and the version of the EBML Body.  Both versions are
  meant to be backward compatible.

I don’t see how that’s practical, as, taken strictly, it means you’ll never be able to make a significant change that is not backward compatible, so you’ll be stuck with errors or limitations forever.  Are you sure you won’t need to allow for incompatible versions at some point?

— Section 17.1 —

  Values from 1 to 126 are to
  be allocated according to the "RFC Required" policy [RFC8126].

Why did you choose that policy?  Are you aware that this allows registrations from non-IETF-stream RFCs?  In particular, anyone can get an RFC published in the Independent stream with a very light level of review.  Did you consider IETF Review, which requires an RFC in the IETF stream (including Informational and Experimental RFCs)?  Or even Standards Action, which requires standards-track RFCs?

The same comment applies to "matroska" and "webm" in Section 17.2.

[Ballot comment]
I support Adam's DISCUSS point about the abstract.

I'm not clear on why the IANA registry names are prefixed with "CELLAR." Are there more general EBML Element ID and DocType registries envisioned? If not, I would suggest dropping the "CELLAR."

For future documents I would expect the authors to respond to the Gen-ART reviewer's comments via email after fixes have been applied to address the issues raised in the review.

[Ballot discuss]
Thanks to the authors and the participants of the CELLAR working group for
the work that has gone into documenting the EBML format. I have a handful
of comments that I believe need to be addressed prior to publication, and
a handful of suggestions for improvement.

---------------------------------------------------------------------------

Abstract:

I see that the Introduction has been revised to address Ben Campbell's AD
review comment regarding the document positioning itself as a general-purpose
data format rather than being scoped to its use in Matroska. The Abstract
still claims the much broader scope -- please update it to match the reduced
scope in the Introduction.

---------------------------------------------------------------------------

§7.3:

>  A Float Element stores a floating-point number as defined in
>  [IEEE.754.1985].

This is not sufficiently precise to interoperate, as IEEE-754 defines multiple
floating-point representations at each bit length. To differentiate from,
e.g., decimal representation and arithmetic format (neither of which are
probably what you want), please specify the use of "binary interchange
format" (unless some other format is intended).

---------------------------------------------------------------------------

§14.1.3:

>  For String Elements and UTF-8 Elements the length of Element Data MAY
>  be reduced by adding Null Octets
...
>  Note that this method is NOT RECOMMENDED.

These two normative statements conflict with each other: when using RFC 2119
language, MAY is a very different level than "NOT RECOMMENDED" (which is
equivalent to "SHOULD NOT"). Please pick one, and eliminate the other.

---------------------------------------------------------------------------

§17.1:

>  The VINT Data value of one-octet Element IDs MUST be between 0x01 and
>  0x7E.  These items are valuable because they are short, and need to
>  be used for commonly repeated elements.  Values from 1 to 126 are to
>  be allocated according to the "RFC Required" policy [RFC8126].

This, combined with the values that are being registered, is extremely
confusing, and I don't know how IANA is supposed to understand what's going on
without reading and understanding the VINT bit encoding scheme (which is way
too much to ask of them). This is because of the document-wide practice
of speaking of IDs in their VINT-encoded values (e.g., 0xBF) instead of their
data values (e.g., 63 or 0x3F), including in the initial registry in this section.

Please either revise the prose to speak in terms of VINT-encoded values
(e.g., "MUST be between 0x81 and 0xFE"), or revise the registration
tables to indicate the VINT data values (e.g., "0x3F" for CRC-32).

[Ballot comment]
§5:

>              Table 3: Examples of valid and invalid VINTs

This label is a bit confusing, as all the shown VINTs are valid;
they're just not valid for use as Element IDs.

---------------------------------------------------------------------------

§6.3:

>                +--------------+----------------------+
>                | Octet Length | Possible Value Range |
>                +==============+======================+
>                | 1            | 0 to 2^(7-2)        |
>                +--------------+----------------------+
>                | 2            | 0 to 2^(14-2)        |
>                +--------------+----------------------+
>                | 3            | 0 to 2^(21-2)        |
>                +--------------+----------------------+
>                | 4            | 0 to 2^(28-2)        |
>                +--------------+----------------------+
>                | 5            | 0 to 2^(35-2)        |
>                +--------------+----------------------+
>                | 6            | 0 to 2^(42-2)        |
>                +--------------+----------------------+
>                | 7            | 0 to 2^(49-2)        |
>                +--------------+----------------------+
>                | 8            | 0 to 2^(56-2)        |
>                +--------------+----------------------+

I think the value ranges here are indicated incorrectly. If I understand the
encoding scheme correctly, the range for one octet is 0 through 126,
which is "0 to (2^7)-2" rather than "0 to 2^(7-2)" (which means 0 through 32).

The other lengths have identical errors.

---------------------------------------------------------------------------


§8:

>  An EBML Document is comprised of only two components, an EBML Header

Please change to either of the following:

  An EBML Document is composed of only two components...

  An EBML Document comprises only two components...

---------------------------------------------------------------------------

§11.1.5.2:

>  A path with a EBMLGlobalParent defines a
>  Section 11.3.

This doesn't parse. I think you mean:

  A path with a EBMLGlobalParent defines a Global Element; see Section 11.3.

---------------------------------------------------------------------------

§ 11.1.5.12:

>  A boolean to express if an EBML Element is defined as an Identically
>  Recurring Element or not.

As the term "Identically Recurring Element" has not been defined prior to this
section, A forward reference to 11.1.16 would be very useful here.

[Ballot comment]
Section 1.  Please add a reference to Matroska instead of an inline URL

Section 1.  Please add a reference for WebM

Section 2 and Table 4 of Section 5.  These sections define EBML Class.  However, it isn’t used else where in the document.  What is it supposed to be used for?

Section 4.4.  Recommend replacing “This table” text to be the name of specific table in question (i.e., Table 1, Table 2)

Section 6.2.  Per “Unknown-Sized Element MUST NOT be used or defined unnecessarily; however if the Element Data Size is not known before the Element Data is written, such as in some cases of data streaming, then Unknown- Sized Elements MAY be used.”, should this text be read as “the Unknown- Sized Elements MUST only be used if the Element Data Size is not known before the Element Data is written”.  I’m having trouble understanding how to handle normative language for a qualitative statement of “unnecessarily”

Section 11.1.5.1.  Double checking on the grammar of the name attribute – it is permitted to start with a “-“ or a “.”?

Section 11.1.5.3.  Are there any uniqueness properties for an id attribute?  Drawing a parallel from XML,  I would have thought that each EBML element would have unique ID per doctype (say like an xml:id)

Section 11.1.7.2.  documentation@purpose has a number of possible enumerated values, however, none are defined in the text (they are only listed)

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has completed its review of draft-ietf-cellar-ebml-13. If any part of this review is inaccurate, please let us know.

The IANA Functions Operator has a question about one of the actions requested in the IANA Considerations section of this document.

The IANA Functions Operator understands that, upon approval of this document, there are two actions which we must complete.

First, a new registry is to be created called the CELLAR EBML Element ID Registry.

IANA Question --> Where should this new registry be located? Should it be added to an existing registry page? If not, does it belong in an existing category at http://www.iana.org/protocols?

Values in the registry between 0x01 and 0x7E are assigned using the RFC Required policy as defined in RFC 8126.

Values in the registry between 0x007F and 0x3FFE are assigned using the Specification Required policy as defined in RFC 8126.

Values 0x3FFF and 0x4000 are reserved.

Values between 0x400100 and 0x1FFFFE are assigned using the First Come, First Served policy as defined in RFC 8126.

Values 0x1FFFFF and 0x200000 are reserved.

Values between 0x101FFFFF and 0x1FFFFFFE are allocated using the Specification Required policy as defined in RFC 8126.

IANA Question --> The current draft says that "Other four-octet Element IDs may be allocated by the "First Come First Served" policy." Could the authors enumerate the ranges for those Element Ids?

The values 0xFFFFFFF and 0x1000000 are reserved.

IANA Question --> The current draft says that "Five octet Element IDs (values from 0x10000001 upwards) are RESERVED according to the "Experimental Use" policy." Could the authors be explicit about the range of values for the five octet Element Ids?

There are initial registrations in the new registry as follows:

+------------+-------------------------+-----------------+
| ID | Element Name | Reference |
+============+=========================+=================+
| 0x1A45DFA3 | EBML | [ RFC-to-be ] |
| | | Section 11.2.1 |
+------------+-------------------------+-----------------+
| 0x4286 | EBMLVersion | [ RFC-to-be ] |
| | | Section 11.2.2 |
+------------+-------------------------+-----------------+
| 0x42F7 | EBMLReadVersion | [ RFC-to-be ] |
| | | Section 11.2.3 |
+------------+-------------------------+-----------------+
| 0x42F2 | EBMLMaxIDLength | [ RFC-to-be ] |
| | | Section 11.2.4 |
+------------+-------------------------+-----------------+
| 0x42F3 | EBMLMaxSizeLength | [ RFC-to-be ] |
| | | Section 11.2.5 |
+------------+-------------------------+-----------------+
| 0x4282 | DocType | [ RFC-to-be ] |
| | | Section 11.2.6 |
+------------+-------------------------+-----------------+
| 0x4287 | DocTypeVersion | [ RFC-to-be ] |
| | | Section 11.2.7 |
+------------+-------------------------+-----------------+
| 0x4285 | DocTypeReadVersion | [ RFC-to-be ] |
| | | Section 11.2.8 |
+------------+-------------------------+-----------------+
| 0x4281 | DocTypeExtension | [ RFC-to-be ] |
| | | Section 11.2.9 |
+------------+-------------------------+-----------------+
| 0x4283 | DocTypeExtensionName | [ RFC-to-be ] |
| | | Section 11.2.10 |
+------------+-------------------------+-----------------+
| 0x4284 | DocTypeExtensionVersion | [ RFC-to-be ] |
| | | Section 11.2.11 |
+------------+-------------------------+-----------------+
| 0xBF | CRC-32 | [ RFC-to-be ] |
| | | Section 11.3.1 |
+------------+-------------------------+-----------------+
| 0xEC | Void | [ RFC-to-be ] |
| | | Section 11.3.2 |
+------------+-------------------------+-----------------+

Second, a new registry is to be created called the CELLAR EBML DocType registry.

IANA Question --> Where should this new registry be located? Should it be added to an existing registry page? If not, does it belong in an existing category at http://www.iana.org/protocols?

The registry is managed via First Come, First Served as defined in RFC 8126.

The registry would look like:

DocType Description Change Controller Reference
--------+----------------------+------------------+----------+

IANA Question --> The current draft says, "DocType string values of "matroska" and "webm" are RESERVED to the IETF for future use. These can be assigned via the "IESG Approval" or "RFC Required" policies [RFC8126]." Is it correct that these values do not appear in the initial version of the registry? Are there an initial values for the new registry?

The IANA Functions Operator understands that these are the only actions required to be completed upon approval of this document.

Note:  The actions requested in this document will not be completed until the document has been approved for publication as an RFC. This message is meant only to confirm the list of actions that will be performed.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist

The following Last Call announcement was sent out (ends 2019-11-07):

From: The IESG
To: IETF-Announce
CC: villereal@gmail.com, cellar-chairs@ietf.org, cellar@ietf.org, Steven Villereal , draft-ietf-cellar-ebml@ietf.org, alexey.melnikov@isode.com
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Extensible Binary Meta Language) to Proposed Standard


The IESG has received a request from the Codec Encoding for LossLess
Archiving and Realtime transmission WG (cellar) to consider the following
document: - 'Extensible Binary Meta Language'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2019-11-07. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  This document defines the Extensible Binary Meta Language (EBML)
  format as a generalized file format for any type of data in a
  hierarchical form.  EBML is designed as a binary equivalent to XML
  and uses a storage-efficient approach to build nested Elements with
  identifiers, lengths, and values.  Similar to how an XML Schema
  defines the structure and semantics of an XML Document, this document
  defines how EBML Schemas are created to convey the semantics of an
  EBML Document.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/ballot/


No IPR declarations have been submitted directly on this I-D.

This is my AD Evaluation of draft-ietf-cellar-ebml-08.

Thanks for this work. It's generally on the right track, but I have a number of comments that I would like to discuss prior progressing.

*** Substantive Comments ***

- The Shepherd Report says "There are minor nits to be resolved (adding one RFC 2119 keyword and some schema  attributes which are mentioned but not defined."

Do that comment still apply? To be clear, those are more than nits. Adding a 2119 keyword is substantive change that needs to be agreed to but the working group. Undefined attributes suggest that the draft is incomplete. If the comment is still accurate, then these need to be fixed prior to progressing the draft. If it is not accurate, then I ask the shepherd to please update the shepherd report.

- I'm a little surprised to see the draft describe EBML as a general purpose markup language format, as opposed to one designed to be used by Matroska and similar technologies. I suspect others will also be surprised during the IETF LC and IESG review. The review bar is higher for the IETF to recommend EBML as a general purpose markup language than it is to recommend it for specific purposes. I also note that the CELLAR charter talks about EBML in terms of being used by Matroska and FFV1.

Would it be acceptable to add some scoping language to the effect of "EBML is used by Matroska[citation] and FFV1[citation]. It MAY be used for use cases similar to those. The applicability of EBML for other use cases is beyond the scope of this document".

- There are several defined elements/attributes related to versioning, but I don't find an explanation of how they all work together. Please add a section describing versioning in general.

§2: Please use the newer boilerplate from RFC 8174.

§3: Can you offer guidance on the specific impacts of the mentioned attacks, and how to mitigate them? (If there is no way to mitigate an attack, it's okay to say that.)

§5: Please elaborate on how this is similar to UTF8. I assume this refers to using prefix bits to indicate the field length. UTF8 does that to maintain backwards compatibility with Ascii. What is the design motivation for using that approach for EBML, which I presume doesn't have such a requirement?

§5.3: "If the number of bits required for "VINT_DATA"
are less than the bit size of "VINT_DATA", then "VINT_DATA" SHOULD be
zero-padded to the left to a size that fits."

Why not MUST? What happens if this is violated?

§6:
- "The "VINT_DATA" component of the "Element ID" MUST
NOT be either defined or written as either all zero values or all one
values."
Why?

§8.7: "In this case, the "EBML Reader" should skip data
until a valid..."
Should that be a normative statement? If so, why should and not MUST?

§8.8: If the EBML reader does not interpret Binary Elements, why add them at all? What does interpret them? is the point that the EBML reader treats Binary Elements as opaque, and just hands them to the application?

§10.1.3: "NOT RECOMMENDED" seems overly strong, especially in light of the MAY in the first paragraph. Is the point to say "MUST NOT... except when the implementation needs to update an element without rewriting the entire document"?

§11: "An "EBML Document" consists of "EBML Elements" and MUST
NOT contain any data that is not part of an "EBML Element"."

That contradicts text in §8.7 that says you can do this in streaming use cases.

§13.1: "The "DocType"
value for an "EBML Document Type" SHOULD be unique and persistent."

What is the scope of uniqueness? How should it be achieved? Is there an expectation to register Document Types?  (Also, why not MUST)?

§13.1.4.1: Are there uniqueness requirements for name?

§13.1.4.2:
- The idea behind "path" needs elaboration. I'd like to see some high-level discussion of how you indicate where an element is allowed, and how you encode the structure. An example (local to the section) would be helpful.

§13.1.4.4:
- "The "minOccurs" value MUST be
equal to the "EBMLMinOccurrence" value of the "path"."

Why have both if they have to be the same? (Same question for §13.1.4.5)

§13.1.4.12:
- "If the "recurring" attribute
is not present then the "EBML Element" MUST be considered to NOT be
an "Identically Recurring Element"."

"be considered to" is vague for a MUST. What concrete action must implementations take in this case?

§13.1.6.2:
- It seems likely readers will confuse "type" with "data type". Would it be reasonable to call this "document-type" or something similar?

§13.1.10:
- Please state (and cite) which XML schema format is used here.

- Has the schema been mechanically verified?

§13.3.1:
- "The CRC value MUST be computed on a little
endian bitstream and MUST use little endian storage."

Please elaborate. Most elements have been big-endian so far. Do there have to be converted to calculate the CRC?

§15.1:
- "The numbers 0x3FFF and 0x4000 are RESERVED.", "The numbers 0x1FFFFF and 0x200000 are RESERVED.", and "The numbers 0xFFFFFFF and 0x1000000 are RESERVED."
What is the purpose of reserving them?

- "Four octet Element IDs whose lower three octets (as
encoded) would make printable 7-bit ASCII values may be allocated
only Specification Required."

Can you state a range or a list? Don't make IANA judge which values are printable.

§15.2:
- "The strings may be allocated according to
First Come First Served"

Is there a requirement to register them, or is this optional?




*** Editorial Comments and Nits ***

- General: The draft puts double quotes around every mention of terms that it defines. That is unconventional for IETF documents. I personally find that it makes the draft harder to read. I suggest quoting them on the first mention, then just capitalizing them in subsequent mentions.

- General (style comment): Please consider removing most instances of the word "considered". When the text says "X is considered to be equal to Y", that sounds like a weaker (and harder to read) statement than "X is equal to Y". Along the same lines, "considered" is rarely appropriate to use in a normative statement. For example, instead of saying "Implementations MUST consider X to be an error", it is better to state what concrete actions you expect the implementation to take when X occurs.

§3:

- The Security Considerations are required to be one of the last two sections in the body of the document (that is, before the references). More precisely, Security Considerations and IANA considerations should be the last two sections in the documents.


- first paragraph: Please expand CRC on first mention. (Which may or may not be this instance once the Security Considerations are moved to the proper place.)

§5.2: "The "VINT_MARKER" MUST be one bit in length and
contain a bit with a value of one."
That's really more of a definition or statement of fact than a normative requirement. Please consider replacing MUST with "is".

§5.3:
- "The bits required for the "VINT_WIDTH" and the
"VINT_MARKER" combined use one out of eight bits"
Consider "use one out of every eight bits"

§5.4, paragraph after first table: "Data encoded as a "Variable Size Integer" MAY be rendered at octet
lengths larger than needed to store the data."
Is that intended as permission, or a statement of fact? If the latter, then the normative keyword is not appropriate.

§6:
- "The "Element ID" MUST be encoded as a "Variable Size Integer"."
That seems like a definition. Consider s/MUST/is
- "MUST NOT be considered an error in the "EBML Document"."
"be considered" is vague for a normative requirement. What concrete action is being prohibited (or required)?

§7:
- "The "Element Data Size" itself MUST be encoded as a "Variable
Size Integer"."
Consider s/MUST/is

- 'Only "Master Elements" SHALL be "Unknown-Sized Elements".'
The use of "only" in normative statements can be ambiguous. In this case it can be read that non-master elements MUST NOT be of unknown size, or that the SHALL only applies to master elements, and there is no rule for non-master elements. Please consider formulating this as "MUST NOT".

- "For example, an "EBML Element" with an octet length of 127
MUST NOT be encoded in an "Element Data Size" encoding with a one
octet length."

Examples are not normative, therefore they should not include normative keywords.

§8.1: "Signed Integers MUST be stored with two’s
complement notation"
Please consider s/MUST/are

§10.1.2: "This method is only RECOMMENDED for reducing "Element Data" by a
single octet..."
Please avoid using "only" in normative requirements (see previous comment for explanation).

§11.1
- 'that uses an"EBMLVersion" of "1" MUST only contain "EBML Elements"'
Please avoid "only" on normative statements.

- last paragraph: Please consider dropping "All" from the beginning of each sentence.

§11.1:
- Is the concept of a file concrete (in the sense of a unit of persistent storage) or abstract (as in any stream of data)? I had assumed the latter, but with the different rules about data outside of EBML elements for "files" and "streaming applications", I am not so sure. If you mean "file" concretely, does there need to be additional text in this section describing the structure for streaming applications?

- "The "EBML Body" MUST consist
only of "EBML Elements""
Please avoid "only" on normative statements.
- 'what "EBML Elements"' (several occurances). s/what/which

§13.1:
- "one or many" (several times throughout draft): Should this be "one or more"? Some people do not tend to think of numbers like 2 or 3 as counting as "many".

- "An "EBML Schema" must be
expressed as well-formed XML."
Normative?

- "An "EBML Schema" MUST declare exactly one "EBML Element" at "Root
Level" (referred to as the "Root Element") that MUST occur exactly
once within an "EBML Document"."

The second "MUST" seems like a statement of fact.

- If an "EBML Schema"
adopts the "EBML Header Element" as-is, then it is not REQUIRED to
document that Element within the "EBML Schema"."

"REQUIRED" is not used normatively, and therefore should not be capitalized.

§13.1.1.2:
- "The "version" lists an incremental non-negative integer..."

I'm not sure what it means for a single integer to be "incremental". (But see substantive comment concerning versioning.)


§13.4.1.2:
- The ""*"", ""("" and "")"" symbols MUST be interpreted as they are
defined in the ABNF."
This _is_ the ABNF. Do you mean "as defined in RFC 5234?  Also, the normative MUST is not appropriate here; it's a matter of definition, not a normative requirement.

- "The "EBMLElementOccurrence" part is interpreted as an ABNF Variable
Repetition.

I don't understand what that means. There won't be ABNF in an actual schema, will there? (Same for "VariableParentOccurrence")

- ""EBML Elements" with "EBMLMinOccurrence" set to "1" that also have a
"default" value (see Section 13.1.4.8) declared are not REQUIRED to
be stored but are REQUIRED to be interpreted,"

Statement of fact?

§13.1.4.7
- The attribute is called "size" but defined to be "length". Why not call it length? (Or describe it as "size")?

§13.1.4.10:
- "A boolean to express if an "EBML Element" MAY be used as an "Unknown-
Sized Element""
Statement of fact.

§13.1.11: It would be helpful to have the example closer to the element descriptions. I suggesting putting this section before the schema section.

§13.1.13: Why is the material is this section separate from the range element description?


§13.1.14:
- "When a float value is represented textually in an "EBML Schema", such
as within a "default" or "range" value,..."

The examples only talk about ranges. Can there be an example for a default value?

§13.1.15:
- "In this case, the default value of the "Mandatory EBML
Element" MUST be interpreted by the "EBML Reader" although the "EBML
Element" is not present within its "Parent Element"."

I'm not sure the intent here. Is "interpreted" the correct word choice?

§13.3.2:
- "Used to void damaged data, to avoid unexpected behaviors
when using damaged data."

Comma splice.

§14
- "If a "Master Element" contains a "CRC-32 Element" that doesn’t
validate, then the "EBML Reader" MAY ignore all contained data except
for "Descendant Elements" which contain their own valid "CRC-32
Element"."

s/ ", which" / " that"

§15.1:
- "Values from 1 to 126 are to
be allocated according to RFC Required."

I suggest '... allocated according to the "RFC Required" policy.' (Note: This pattern occurs several times in the RFC considerations section.) Also, please cite 8126 on the first mention of each policy (or state in advance that all registration policies mentioned herein are defined in 8126).

- "Numbers MAY be allocated within this range according to
Specification Required."
s/MAY/are

1. Summary


The document shepherd is Steven Villereal.  villereal at gmail dot com
The responsible Area Director is Ben Campbell.


This document defines format considerations for Extensible Binary Meta Language, a hierarchical file format for efficient delivery of binary data in defined Elements inspired by XML. It proposes definitions for creating and validating EBML Schemas that define the use and interpretation of Elements within EBML Documents. 


2. Review and Consensus


This document has been adequately reviewed by working group members and others, through both mailing list discussion and Github issues and pull requests.


One minor issue was how to define EBML Elements in a Header document in a forward-compatible way. It was decided Headers could require a specific, minimum or maximum EBMLVersion (which might have different implementations of Elements).


Most recent discussion has focused on how Element IDs (unique identifiers for EBML Elements used within a Schema and Document) should be defined and encoded (as Variable Size Integers) with regard to IANA registration.


This document creates a new IANA Registry called "CELLAR EBML ElementID Registry" but to simplify IANA registration, the group agreed this document should only apply to Element IDs of EBML Header and global elements. More specific implementations of EBML (such as matroska) found in an EBML Body will have their own Element IDs that are not defined in this document.


This document creates a new IANA Registry called "CELLAR EBML DocTypeRegistry". This document reserves DocType string values of "matroska" and "webm" in the First Come First Served registry.




3. Intellectual Property


The authors of this draft submit it in full conformance with the provisions of BCP 78 and BCP 79.




4. Other Points
There are minor nits to be resolved (adding one RFC 2119 keyword and some schema  attributes which are mentioned but not defined.
This is a very readable and clearly written document, and represents the inclusion of multiple recent revisions resolving minor outstanding issues. There appears to be strong group consensus on this document’s readiness to move forward in the standardization process.

This starts a ~2 week WGLG on the EBML document.
The WGLC will proceed until Friday, October 12, 2018.

Please read the document and indicate if you feel it is ready to proceed to the IESG for publication.