Guidelines for Development of an Audio Codec within the IETF
draft-ietf-codec-guidelines-08

[Ballot discuss]
The Abstract section speaks about 'developing and specifying a codec within the IETF', then the Introduction speaks about 'standardization of a codec that is optimized for use in interactive Internet applications' and only in Section 2 the word 'audio' appears for the first time when text about 'Work on audio codec development ' is included. I suggest to make clear from start (maybe even from the title) that the document is about guidelines for the development of an audio codec optimized for use in interactive Internet applications. Later similar work may deal with codecs for non-interactive applications, or with codecs for video applications, and there is a need for clear delimitation. This seems important enough to raise previous COMMENTs to DISCUSS level, and should be a DISCUSS easy to resolve.

[Ballot comment]
The WG charter says quite a lot about liaising its work with other SDOs. Although this I-D may be a bit foundational, it would not be harmful to offer other SDOs the chance to comment on the way we plan to proceed.

I sense that a lot of voices were raised in the WG, and that is good, but I son't see any mention of communication with other SDOs in the write-up.

I should like the ADs and chairs to ensure they have in place a proper mechanism to ensure the right documents are liaised at the right time in the cycle.

[Ballot comment]
Section 6 would be much improved if bullet item 2 (IETF's own codec) stated that such codecs MUST be entirely separate in terms of their name, code points, and usage in applications from other codecs. That is, if IETF develops its own codec, it should assign specific code points for its use as well, not, e.g., reuse some code points that were already being used by a codec developed elsewhere.

[Ballot comment]
- I tend to agree with Stephen's comment that this document really is about an audio codec and it should say that. But let me go further and say that this document really is about *the* audio codec being developed in the codec working group. The document has some text that alludes to that (e.g., the title mentions "the Codec" and there are several mentions of "the group" in the document), but other places that is not clear and it sounds like it is giving guidelines for *any* codec (audio, video, otherwise) in *any* IETF working group (e.g., the intro says, "This document describes a suggested process for work at the IETF..."). I don't think this document intends to do the latter, and it would be useful to clarify some of that language.

- The intro says "This document describes a suggested process...", and elsewhere it is worded as a "suggestion". I wouldn't mind if that got tightened up into more of a "guideline" or a "process" that the WG *will* follow. Of course, if the WG came to consensus that they wanted some latitude to ignore this guidance from time to time, I have no strong objection to leaving in the softer language.

[For the moment, I will make the following a comment. However, if others on the IESG feel strongly about it, I can switch it to a discuss.]

- I don't think BCP 79's language about "preferring" no-known-IPR or royalty-free-IPR technology is normative; I think it's descriptive. That is, it is fine if this particular WG wants to explicitly prefer no-known-IPR (or, secondarily, royalty-free-IPR), but don't blame BCP 79 as if it says that the WG *ought* to prefer them. I think the statements regarding doing things "in accordance with BCP 79" are bogus. They should be replaced with statements that the WG is doing things "as described by BCP 79".

[Ballot comment]
So I'm not sure you need to do this, but a lot of times WG's develop requirements drafts and then they fret over whether to progress the before the solution draft or keep it as a living draft until after the solution draft is published.  If the WG has considered this, then it might be worth saying whether or not you're going to progress the requirements draft prior to completing the solutions draft.

You could add an informative reference to RFC 4648 for base64.

[Ballot comment]
- I'm wondering if this is really only about audio - the title implies
its more general but the body of the document is only specifically
about audio. Suggest maybe adding audio to the title if that's
correct and matches the wg's intent.

- Step 5 of the list in section 2: this doesn't specify criteria
for "sufficient" but I expected that's a normal WG consensus call for
the lucky chairs. More interesting though is whether this has already
happened or not? If so, then you can and maybe should use the
past-tense. If not, when is this process going to be run?

- p5 mentions "the requirements document" - why not add a reference?

- p9 says errata "should be maintained" - that happens for all RFCs
already.

[Ballot comment]
Although I strongly support this work, I am recusing myself because I co-authored the first several versions of draft-valin-codec-guidelines, which was used as the starting point for the WG document.

State changed to In Last Call from Last Call Requested.

The following Last Call Announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Subject: Last Call:  (Guidelines for the Codec Development Within the IETF) to Informational RFC


The IESG has received a request from the Internet Wideband Audio Codec WG
(codec) to consider the following document:
- 'Guidelines for the Codec Development Within the IETF'
  as an Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2011-10-19. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  This document provides general guidelines for work on developing and
  specifying a codec within the IETF.  These guidelines cover the
  development process, evaluation, requirements conformance, and
  intellectual property issues.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-codec-guidelines/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-codec-guidelines/


No IPR declarations have been submitted directly on this I-D.

>  (1.a) Who is the Document Shepherd for this document? Has the
>        Document Shepherd personally reviewed this version of the
>        document and, in particular, does he or she believe this
>    version is ready for forwarding to the IESG for publication?

Jonathan Rosenberg is the Document Shepherd and has personally reviewed
this version of the document and believes it is ready for publication.

>  (1.b) Has the document had adequate review both from key WG members
>    and from key non-WG members? Does the Document Shepherd have
>    any concerns about the depth or breadth of the reviews that
>    have been performed?

The document has been adequately discussed and commented on by a number
of participants and has been discussed in multiple WG meetings.

>  (1.c) Does the Document Shepherd have concerns that the document
>    needs more review from a particular or broader perspective,
>    e.g., security, operational complexity, someone familiar with
>    AAA, internationalization or XML?

No.

>  (1.d) Does the Document Shepherd have any specific concerns or
>    issues with this document that the Responsible Area Director
>    and/or the IESG should be aware of? For example, perhaps he
>    or she is uncomfortable with certain parts of the document, or
>    has concerns whether there really is a need for it. In any
>    event, if the WG has discussed those issues and has indicated
>    that it still wishes to advance the document, detail those
>        concerns here. Has an IPR disclosure related to this document
>    been filed? If so, please include a reference to the
>        disclosure and summarize the WG discussion and conclusion on
>    this issue.

The CODEC working group is, in general, a contentious area of work. This
document has been amongst the least contentious of our work items. The
parts which have gotten most discussion surround intellectual property
guidelines and testing processes. We do believe this document reflects
consensus in the group around these points.

>  (1.e) How solid is the WG consensus behind this document? Does it
>        represent the strong concurrence of a few individuals, with
>    others being silent, or does the WG as a whole understand and
>    agree with it?

A WGLC was issued for this document on July 6, 2011. Minor comments were
received. An issue was raised around whether the group would perform
codec characterization as part of the process and what this means. This
was discussed at the IETF meeting in Quebec, and consensus was reached,
resulting in a document update just after the meeting. In general we
believe there is good consensus behind this document.

>  (1.f) Has anyone threatened an appeal or otherwise indicated extreme
>        discontent? If so, please summarise the areas of conflict in
>        separate email messages to the Responsible Area Director. (It
>    should be in a separate email because this questionnaire is
>    entered into the ID Tracker.)

No appeals have been threatened.

>  (1.g) Has the Document Shepherd personally verified that the
>        document satisfies all ID nits? (See the Internet-Drafts Checklist
>    and http://tools.ietf.org/tools/idnits/). Boilerplate checks are
>    not enough; this check needs to be thorough. Has the document
>    met all formal review criteria it needs to, such as the MIB
>    Doctor, media type and URI type reviews?

Yes.

>  (1.h) Has the document split its references into normative and
>    informative? Are there normative references to documents that
>    are not ready for advancement or are otherwise in an unclear
>    state? If such normative references exist, what is the
>        strategy for their completion? Are there normative references
>    that are downward references, as described in [RFC3967]? If
>    so, list these downward references to support the Area
>        Director in the Last Call procedure for them [RFC3967].

Yes. Though the specification is targeted for Informational, it has
normative references to several IETF process RFCs, all of which are at
RFC status.

>  (1.i) Has the Document Shepherd verified that the document IANA
>        consideration section exists and is consistent with the body
>    of the document? If the document specifies protocol
>        extensions, are reservations requested in appropriate IANA
>        registries? Are the IANA registries clearly identified? If
>    the document creates a new registry, does it define the
>        proposed initial contents of the registry and an allocation
>        procedure for future registrations? Does it suggest a
>        reasonable name for the new registry? See [RFC5226]. If the
>        document describes an Expert Review process has Shepherd
>        conferred with the Responsible Area Director so that the IESG
>    can appoint the needed Expert during the IESG Evaluation?

There is no IANA consideration.

>  (1.j) Has the Document Shepherd verified that sections of the
>        document that are written in a formal language, such as XML
>    code, BNF rules, MIB definitions, etc., validate correctly in
>    an automated checker?

Not applicable.

>  (1.k) The IESG approval announcement includes a Document
>        Announcement Write-Up. Please provide such a Document
>        Announcement Write-Up? Recent examples can be found in the
>        "Action" announcements for approved documents. The approval
>        announcement contains the following sections:

This document provides general guidelines for work on developing and
    specifying a codec within the IETF.  These guidelines cover the
    development process, evaluation, requirements conformance, and
    intellectual property issues.


> Technical Summary
>        Relevant content can frequently be found in the abstract
>    and/or introduction of the document. If not, this may be
>    an indication that there are deficiencies in the abstract
>    or introduction.

The abstract conveys the technical summary well:

This document provides general guidelines for work on developing and
    specifying a codec within the IETF.  These guidelines cover the
    development process, evaluation, requirements conformance, and
    intellectual property issues.


>
> Working Group Summary
>    Was there anything in WG process that is worth noting? For
>        example, was there controversy about particular points or
>    were there decisions where the consensus was particularly
>    rough?


The document was the subject of much discussion, as the work of the
CODEC working group in general is contentious. In general the testing
aspect of this has been the most contentious, due to the differing
methodologies used by IETF relative to other SDOs. The IPR sections also
got much discussion. These were resolved without too much difficulty and
we believe the document reflects consensus of the group.


>
> Document Quality
>    Are there existing implementations of the protocol? Have a
>        significant number of vendors indicated their plan to
>        implement the specification? Are there any reviewers that
>    merit special mention as having done a thorough review,
>    e.g., one that resulted in important changes or a
>        conclusion that the document had no substantive issues? If
>    there was a MIB Doctor, Media Type or other expert review,
>    what was its course (briefly)? In the case of a Media Type
>    review, on what date was the request posted?

This document does not specify a protocol and therefore there are no
implementations. However it has been looked at, and commented on, by
numerous experts in voice coding as well as experts in IETF process.
--