Definition of the Opus Audio Codec
draft-ietf-codec-opus-16

[Ballot comment]
I am clearing my discuss on the agreement that text similar to the following will be added to header of the files:

"This file is extracted from RFCXXXX. Please see that RFC for additional information."

Where XXXX will be replaced by the actual RFC number.

I understand similar but more grammatically appropriate text will be added to the README file.

[Ballot comment]
Glad to see the updated copyright statements. Thanks.

As I stated earlier, the files in the git repository cited in the document probably also should have the IETF Trust copyright. Moreover, I would prefer to see the repository hosted on an IETF site rather than on xiph.org. I am concerned about drift of the code on xiph from the code in the RFC and ending up in a situation where the xiph version is -- de facto -- the place to go for the standard.

Like others, I don't feel I can give a substantive review to much of this document and will simply trust that it has gotten sufficient review. The one technical thing I noticed: A few variables that need to be unsigned because they're used in shift right (>>) operations are not explicitly specified as unsigned, in particular ft (used in 4.1.5) and b0 (used in 4.1.1). You do note other variables as unsigned, so I took these as omissions, but it may be clear from context. It seems to me a review of such variables might be useful.

Like other ADs, I have some concerns about the use of source code for the normative spec:

- First of all, this does turn the entire concept of "independent interoperable implementations" on its head. If the source code is normative, I don't see how we could ever judge that an implementation was independent and therefore how this spec would be advanced along the standards track.

- Furthermore, I've got some concerns that we may, at some point in the future, encounter a small edge case where the code simply has a bug, but the text of the document makes it clear what the intention was. I would hate for us to say, "Well, the code is the normative reference, so the bug must stand and we must update the descriptive text to suit." I doubt that would happen, but we are in unchartered waters by using source as the normative spec.

- Finally, the source code here is in base64, compressed, tarred format. It is completely unreadable in the spec. I do not know of any other RFC that has humanly-unreadable text in it. Again, we are in completely unchartered waters.

And it is these "unchartered waters" that are the real issue all around. We may need to discuss how we handle this document now and in the future. But we signed up for this in the charter, so I'm willing to see it play out. There is no doubt that this document is an entirely odd duck.

[Ballot comment]



- I agree that Pete's discuss needs to be resolved. It may be
that there's a way to do that without the IETF trust owning the
code, I don't know. I did also note that celt/pitch.c also has a
CSIRO copyright from 2007-2008.  Were people from CSIRO involved
in the WG? If so, does whatever code is involved constitute an
IETF contribution (I would think so) and has someone checked
that no IPR declarations from CSIRO are needed?  (Same question
applies for all such contributors.) This is (I think) a varition
on Pete's discuss since I guess that people from Xiph and Skype
(the examples he quoted) have been involved in this work so any
IPR declartions from them will have been done already.

- If code comments conflict with the text of the body of the
draft, then which is normative? Its clear that code wins but I'm
not sure about comments within the code.  I'm ok with any
reasonable answer here, but wanted to check.

- The code contains things like "#ifdef FIXED_POINT" which is
commented out in the top level Makefile. If the normative
specification is the source code, what is the normative status
of code that's only conditionally compiled and that is not
compiled with the distributed Makefile? Have all such
conditionally compiled code fragments been tested to the same
extent as the code that is compiled with the default Makefile?
I'm ok with any reasonable answer here, but wanted to check.

- I think a few references to introductory materials about
codecs would be useful in the intro for the poor victim^Wreader
who's handed this and told to implement or optimise opus.

- p13, [SRTP-VBR], RFC 6562, says that VBR SHOULD NOT be used
for "highly sensitive" voice and goes on to say that VBR is ok
if you're just matching the bandwidth but maybe not if the
variation is driven by the speech signal. I think it'd be good
to note both of those aspects here. I'd also personally prefer
if both 6562 and this said "possibly sensitive" rather than
"highly sensitive" since I don't know how code at this layer
could know what's really sensitive or not. This might also be
worth repeating in (or moving to) the security considerations as
well as it would perhaps be more visible there.

nits

- p28, says "This same data MUST also be returned as raw bits
when requested." Requested by whom? I didn't get that.

- p30, says "this draft" s/draft/memo/ or whatever

- p31, says "SHOULD take measures to conceal the error" but
doesn't say what those might be

- p32, all those "1/8th bit" phrases read oddly to me but maybe
that's standard fare for codec folks.

- p151, says that padding is used in CBR mode but doesn't seem
to say what kind of padding. If its the same as for my discuss
(i.e. zero padding) then maybe the same considerations apply
here? (Not sure)

[Ballot discuss]
At this stage the authors do not need to do anything. I wish to discuss the following with my IETF collegues and will update this discuss accordingly.

Given that the IETF knows of IPR claims and is publishing the software package as part of an RFC, I am concerned that we have an obligation to have included in the copyright notice a pointer to those IPR claims, lest at some point in the future the IETF are accused of failing to warn implementers of those claims.

This may be something that needs clarifying with IETF council.

[Ballot comment]
I am saddened that the IETF failed to meet the following goal for this work, i.e. to address the following:

"There exist codecs that are standardized, but that cannot be widely
implemented and easily distributed; according to reports, the presence
of various usage restrictions (e.g., in the form of requirements to pay
royalty fees, obtain a license, enter into a business agreement, or meet
other special conditions imposed by a patent holder) has hindered
adoptions of such codecs in interactive Internet applications."

[Ballot discuss]
Examining the source files and some of the other files, I find things similar to this:

  Copyright (c) 2011 Xiph.Org Foundation, Skype Limited

I think the copyright for the source code should all be held by the IETF Trust with appropriate template information. The reason I am concerned that this might take a while is that in at least the case of the COPYING file in the root directory, the copyright is held by several organizations and individuals. I believe all of those folks must be contacted and explicitly agree to allowing their code to also be copyright by the IETF Trust.

(The files in the git repository cited in the document probably also should have the IETF Trust copyright. Moreover, I would prefer to see the repository hosted on an IETF site rather than on github. I am concerned about drift of the code on github from the code in the RFC and ending up in a situation where the github version is -- de facto -- the place to go for the standard.)

[Ballot discuss]
Examining the source files and some of the other files, I find things similar to this:

  Copyright (c) 2011 Xiph.Org Foundation, Skype Limited

I think the copyright for the source code should all be held by the IETF Trust with appropriate template information. The reason I am concerned that this might take a while is that in at least the case of the COPYING file in the root directory, the copyright is held by several organizations and individuals. I believe all of those folks must explicitly agree to allowing their code to also be copyright by the IETF Trust.

(The files in the git repository cited in the document probably also should have the IETF Trust copyright. Moreover, I would prefer to see the repository hosted on an IETF site rather than on github. I am concerned about drift of the code on github from the code in the RFC and ending up in a situation where the github version is -- de facto -- the place to go for the standard.)

[Ballot comment]
Like others, I don't feel I can give a substantive review to much of this document and will simply trust that it has gotten sufficient review. The one technical thing I noticed: A few variables that need to be unsigned because they're used in shift right (>>) operations are not explicitly specified as unsigned, in particular ft (used in 4.1.5) and b0 (used in 4.1.1). You do note other variables as unsigned, so I took these as omissions, but it may be clear from context. It seems to me a review of such variables might be useful.

Like other ADs, I have some concerns about the use of source code for the normative spec:

- First of all, this does turn the entire concept of "independent interoperable implementations" on its head. If the source code is normative, I don't see how we could ever judge that an implementation was independent and therefore how this spec would be advanced along the standards track.

- Furthermore, I've got some concerns that we may, at some point in the future, encounter a small edge case where the code simply has a bug, but the text of the document makes it clear what the intention was. I would hate for us to say, "Well, the code is the normative reference, so the bug must stand and we must update the descriptive text to suit." I doubt that would happen, but we are in unchartered waters by using source as the normative spec.

- Finally, the source code here is in base64, compressed, tarred format. It is completely unreadable in the spec. I do not know of any other RFC that has humanly-unreadable text in it. Again, we are in completely unchartered waters.

And it is these "unchartered waters" that are the real issue all around. We may need to discuss how we handle this document now and in the future. But we signed up for this in the charter, so I'm willing to see it play out. There is no doubt that this document is an entirely odd duck.

[Ballot comment]
(non-blocking comments for the AD and editors)

reference [Burg] has nothing but an author and title; it has to be locatable somehow other than google

wikipedia references should be replaced by an actual textbook or other materials, in my opinion

[Ballot discuss]
p20 says you MUST zero out padding bits. I wondered if
there are any situations where so many zero'd padding bits would
be sent and where the bits are being stream-enciphered that
that'd create a problem. Dodgy stream-cipher setups have been
seen in the wild in the past so this could in such a case be a
real problem. One could say that these padding bits MUST be
random or MUST be set to zero except when a stream cipher is
used in which case they MUST be set randomly. While neither
seems perfect, either would be better than exposing a key.  You
can't tell from the ciphertext if there's a covert channel in
any case, so zero'ing the padding bits is no help to any
intermediary once they're sent with confidentiality. If this is
a potential problem, it'd be good to know how often padding
might happen and if there's any way to force it to happen by
manipulating the channel. Perhaps the best thing to do here is
change the MUST to a SHOULD and document the potential problem
in the security considerations section.

[Ballot comment]

- I agree that Pete's discuss needs to be resolved. It may be
that there's a way to do that without the IETF trust owning the
code, I don't know. I did also note that celt/pitch.c also has a
CSIRO copyright from 2007-2008.  Were people from CSIRO involved
in the WG? If so, does whatever code is involved constitute an
IETF contribution (I would think so) and has someone checked
that no IPR declarations from CSIRO are needed?  (Same question
applies for all such contributors.) This is (I think) a varition
on Pete's discuss since I guess that people from Xiph and Skype
(the examples he quoted) have been involved in this work so any
IPR declartions from them will have been done already.

- If code comments conflict with the text of the body of the
draft, then which is normative? Its clear that code wins but I'm
not sure about comments within the code.  I'm ok with any
reasonable answer here, but wanted to check.

- The code contains things like "#ifdef FIXED_POINT" which is
commented out in the top level Makefile. If the normative
specification is the source code, what is the normative status
of code that's only conditionally compiled and that is not
compiled with the distributed Makefile? Have all such
conditionally compiled code fragments been tested to the same
extent as the code that is compiled with the default Makefile?
I'm ok with any reasonable answer here, but wanted to check.

- I think a few references to introductory materials about
codecs would be useful in the intro for the poor victim^Wreader
who's handed this and told to implement or optimise opus.

- p13, [SRTP-VBR], RFC 6562, says that VBR SHOULD NOT be used
for "highly sensitive" voice and goes on to say that VBR is ok
if you're just matching the bandwidth but maybe not if the
variation is driven by the speech signal. I think it'd be good
to note both of those aspects here. I'd also personally prefer
if both 6562 and this said "possibly sensitive" rather than
"highly sensitive" since I don't know how code at this layer
could know what's really sensitive or not. This might also be
worth repeating in (or moving to) the security considerations as
well as it would perhaps be more visible there.

nits

- p28, says "This same data MUST also be returned as raw bits
when requested." Requested by whom? I didn't get that.

- p30, says "this draft" s/draft/memo/ or whatever

- p31, says "SHOULD take measures to conceal the error" but
doesn't say what those might be

- p32, all those "1/8th bit" phrases read oddly to me but maybe
that's standard fare for codec folks.

- p151, says that padding is used in CBR mode but doesn't seem
to say what kind of padding. If its the same as for my discuss
(i.e. zero padding) then maybe the same considerations apply
here? (Not sure)

[Ballot comment]
I support both Pete's and Sean's DISCUSS issues.

I will also add that the instructions for extracting the source code does not work on a Mac running OS X 10.7.4.  The base64 utility present in OS X generates an unrecognizable archive format that cannot be decompressed by tar.

[Ballot discuss]
Updated ...

I haven't complete my review (this one is a beast), but I thought I'd start the process rolling:

1) If we're going to be distributing source code:

Can we include a sha1/sha256 checksums or something in the Appendix?  It's pretty common to do so when distributing source code.

Could we also include the output of the idsigcheck code (https://tools.ietf.org/tools/idsigcheck/code) and place that somewhere permanent to make sure the contents haven't been doctored (and we're not distributing a virus).

2) I glossed right over the pointer to the RFC in the previous paragraph.  cleared this one...

3) (right out of my league here and I'm sure there was some discussion about this)  There's also a file called copying - shouldn't it include the following:

Copyright (c)  IETF Trust and the persons identified as authors of the code.  All rights reserved. 

and shouldn't it be in all the .h, .c, etc. files?

4) (right out of my league here and I'm sure there was some discussion about this)  Some of the copyright years in the .h files are farther back the readme.  opus_types goes back to 1994. Does the readme need to cover all the dates?

[Ballot discuss]
I haven't complete my review (this one is a beast), but I thought I'd start the process rolling:

1) If we're going to be distributing source code:

Can we include a sha1/sha256 checksums or something in the Appendix?  It's pretty common to do so when distributing source code.

Could we also include the output of the idsigcheck code (https://tools.ietf.org/tools/idsigcheck/code) and place that somewhere permanent to make sure the contents haven't been doctored (and we're not distributing a virus).

2) I don't know if this matters but in the readme it says:

  1) Clone the repository (latest implementation of this standard at the time
    of publication)

    % git clone git://git.opus-codec.org/opus.git
    % cd opus

shouldn't this be pointing at the RFC too?

3) (right out of my league here and I'm sure there was some discussion about this)  There's also a file called copying - shouldn't it include the following:

Copyright (c)  IETF Trust and the persons identified as authors of the code.  All rights reserved. 

and shouldn't it be in all the .h, .c, etc. files?

4) (right out of my league here and I'm sure there was some discussion about this)  Some of the copyright years in the .h files are farther back the readme.  opus_types goes back to 1994. Does the readme need to cover all the dates?

[Ballot comment]
The following command in the appendix don't work:

cat draft-ietf-codec-opus.txt | grep '^\ \ \ ###' | sed -e
      's/...###//' | base64 -d > opus_source.tar.gz

I think you need to r/draft-ietf-codec-opus.txt/rfcXXXX.txt and note that the RFC editor should update this when it's published.

[Ballot discuss]
I haven't complete my review (this one is a beast), but I thought I'd start the process rolling:

If we're going to be distributing source code:

Can we include a sha1/sha256 checksums or something in the Appendix?  It's pretty common to do so when distributing source code.

Could we also include the output of the idsigcheck code (https://tools.ietf.org/tools/idsigcheck/code) and place that somewhere permanent to make sure the contents haven't been doctored (and we're not distributing a virus).

[Ballot discuss]
[I have not completed my evaluation, but I wanted to get rolling and send this one since it may take some time to fix.]

Examining the source files and some of the other files, I find things similar to this:

  Copyright (c) 2011 Xiph.Org Foundation, Skype Limited

I think the copyright for the source code should all be held by the IETF Trust with appropriate template information. The reason I am concerned that this might take a while is that in at least the case of the COPYING file in the root directory, the copyright is held by several organizations and individuals. I believe all of those folks must agree to relinquishing their copyright to the code in order for the IETF Trust to have it.

(The files in the git repository cited in the document probably also should have the IETF Trust copyright. Moreover, I would prefer to see the repository hosted on an IETF site rather than on github. I am concerned about drift of the code on github from the code in the RFC and ending up in a situation where the github version is -- de facto -- the place to go for the standard.)

[Ballot comment]
A nit:
Section 3., paragraph 1:

[...]
>    the internal framing used to pack multiple frames into a single
>    packet.  This framing is not self-delimiting.  Instead, it assumes
>    that a higher layer (such as UDP or RTP [RFC3550] or Ogg [RFC3533] or
>    Matroska [Matroska-website]) will communicate the length, in bytes,

  You mean that some lower layer (UDP, RTP, etc) will take care
of this. Those are not higher layers in the IETF sense.

[Ballot comment]
A nit:
Section 3., paragraph 1:

>    The Opus encoder produces "packets", which are each a contiguous set
>    of bytes meant to be transmitted as a single unit.  The packets
>    described here do not include such things as IP, UDP, or RTP headers
>    which are normally found in a transport-layer packet.  A single
>    packet may contain multiple audio frames, so long as they share a
>    common set of parameters, including the operating mode, audio
>    bandwidth, frame size, and channel count (mono vs. stereo).  This
>    section describes the possible combinations of these parameters and
>    the internal framing used to pack multiple frames into a single
>    packet.  This framing is not self-delimiting.  Instead, it assumes
>    that a higher layer (such as UDP or RTP [RFC3550] or Ogg [RFC3533] or
>    Matroska [Matroska-website]) will communicate the length, in bytes,

  You mean that some lower layer (UDP, RTP, etc) will take care
of this. Those are not higher layers in the IETF sense.

[Ballot comment]
Substantive comments; these are non-blocking, but please consider them
seriously, and feel free to chat with me about them:

-- 1.1.3 --
                    clamp(lo,x,hi) = max(lo,min(x,hi))

  With this definition, if lo > hi, the lower bound is the one that is
  enforced.

I'm not sure I understand what that text means.  Are you saying that
if, for example, you write "clamp(5,x,2)", it means the same as
"clamp(2,x,5)" ?  If that's the case, then maybe it should be defined
as this:

  clamp(a,x,b) = max(min(a,b),min(x,(max(a,b)))

In any case, please re-phrase the text to make clearer what you mean.

-- 4.4 --
Have packet loss and clock drift been modelled, and have the recommendations in 4.4 and 4.4.1 thus been validated?  There seems to be a lot of speculation there, and I wonder how much basis in real experimentation there is.

========
Other comments; no need to respond to these:

This is easily the most technically dense and esoteric document I've every read, in the IETF or elsewhere.

I'm not qualified to judge most of this document, and I expect that the number of people in the IETF who are can be counted on one hand.  I'm afraid that includes the participants in the codec working group, and that was one of the concerns some of us had when the WG was chartered.  How many in the working group *really* reviewed this entire document, and understand it?  How much consensus is there in the WG and in the IETF as a whole behind this, in all its details?

That said, everything I can tell about this codec is that it's a good one, and that it was worth doing, despite the extraordinary number of IPR statements that apply to it (considering the WG's goals).  So I approve of the publication of it, and I've reviewed it as best I could.

There was concern brought up in the Applications Directorate review about the mechanism of using C code as a specification.  I don't have a concern about that, and find it an acceptable mechanism.

I did find myself wondering idly, as I read section 4.2.7.5.6, whether "cos(pi*n[k])" was intentional, and whether anyone asked Victoria's Secret about an IPR statement with respect to trademark....

[Ballot comment]
I am balloting No Objection on this document on the basis that it has
no impact on the internet's routing infrastructure, and that the WG and
ADs have done a substantive review

---

Per the Apps Dir review by Claudio Allocchio, I like the idea of adding
a note (IESG Note?) to this document to highlight and make an exception
of the use of code as a specification language.

Updated shepherd writeup

(1.a) Who is the Document Shepherd for this document? Has the Document
Shepherd personally reviewed this version of the document and, in
particular, does he or she believe this version is ready for
forwarding to the IESG for publication?

Jonathan Rosenberg is the Document Shepherd and has personally reviewed
this version of the document and believes it is ready for publication.

(1.b) Has the document had adequate review both from key WG members
and from key non-WG members? Does the Document Shepherd have any
concerns about the depth or breadth of the reviews that have been
performed?

The document has received good discussion by the group. It has been
reviewed top to bottom by the co-chairs of the group, who have focused
on breadth. However, the details of the algorithms, include parameters,
have not received deep review by anyone in the group outside of the
small author team. This is not unexpected given the technical depth of
the work and the domain expertise required to understand the nuances of
all aspects of the codec.


(1.c) Does the Document Shepherd have concerns that the document needs
more review from a particular or broader perspective, e.g., security,
operational complexity, someone familiar with AAA,
internationalization or XML?

No. The document is very narrowly focused on speech coding, and has no
interactions with areas outside of this space.

(1.d) Does the Document Shepherd have any specific concerns or issues
with this document that the Responsible Area Director and/or the IESG
should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there
really is a need for it. In any event, if the WG has discussed those
issues and has indicated that it still wishes to advance the document,
detail those concerns here. Has an IPR disclosure related to this
document been filed? If so, please include a reference to the
disclosure and summarize the WG discussion and conclusion on this
issue.

The CODEC working group is, in general, a contentious area of work. Much
of this is in the area of intellectual property. The ADs need to be
aware that IPR declarations have been made against this
specification. They are:

Microsoft: https://datatracker.ietf.org/ipr/1670/
Skype: https://datatracker.ietf.org/ipr/1602/
Broadcom: https://datatracker.ietf.org/ipr/1526/
Xiph: https://datatracker.ietf.org/ipr/1524/
Qualcomm: https://datatracker.ietf.org/ipr/1520/
Huawei: https://datatracker.ietf.org/ipr/1712/
Huawei: https://datatracker.ietf.org/ipr/1741/

Broadcom, Xiph and Skype/Microsoft have all revised their IPR
declarations throughout the process; the above represents the most
recent set.

The Qualcomm declaration is the most concerning; it does not provide a
royalty-free grant. Achieving RF has been a goal of the activity. None
of the participants in the working group appear to work for, or
represent Qualcomm. The chairs have advised the group that the IETF
cannot take positions on validity of IPR claims, and that individuals
must make their own determinations on this matter. The AD should be
aware that one of the participants from Xiph did post on the list his
belief that, after analysis, none of the patents listed in the Qualcomm
declaration apply to Opus
(http://www.ietf.org/mail-archive/web/codec/current/msg02704.html).

The Huawei declaration is very recent - posted on March 12, 2012. There
has not been any discussion of this on the list. The chairs have decided
to proceed with publication requested, and give the working group the
opportunity to complain during IETF LC if they so wish. Informally, a
few participants have indicated that they do not believe this IPR
impacts Opus.

Some working group members had concerns (expressed privately to the
chairs) about the nature of the RF grants made by Skype; specifically
around termination clauses under conditions of lawsuit. These clauses
have been modified in the most recent disclosure above

The working group was explicitly pointed to all of these disclosures and
was asked whether they wish to proceed given the nature of the
disclosures and their terms. The chairs believe there has been consensus
to proceed based on the IPR declarations made at the time of WGLC. The
only thing that has changed since the consensus call was an updated
Microsoft/Skype license that provides more liberal terms (as well as
availability under the original terms), and the Huawei statement.

The document has also been the source of commentary from external
standards body, most notably ITU, 3gpp and ISO/MPEG. Numerous liaison
statements have been exchanged back and forth. The chairs advised these
groups of our progress throughout the process, and also explicitly asked
for feedback as part of the last call process. The majority of the
comments expressed concerns on the testing process followed by the IETF,
which lacks the rigor typically seen in more formalized ITU
processes. That said, the group itself has consensus that the document
should be published.

(1.e) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it?

The document has had two WGLC. The first was on 8 July 2011. The second
was on 21 October 2011. Comments were received by the group after both
of these WGLC. The chairs believe there is good consensus behind the
document, particularly around the technology. There are several vocal
participants who continue to express dissatisfaction over the testing
and codec validation associated with the work. There is also a
participant who has expressed dissatisfaction with the applicability of
the Qualcomm IPR and has asked for loosening decoder conformance
definitions to comply. The chairs believe there was not consensus for
such changes.

(1.f) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is entered into the ID
Tracker.)

No appeals have been threatened. As noted above, disagreements have
taken place primarily in the areas of testing and validation, and around
IPR and compliance. We do not see those as outside of the normal scope
of disagreement that is common within a working group.

(1.g) Has the Document Shepherd personally verified that the document
satisfies all ID nits? (See the Internet-Drafts Checklist and
http://tools.ietf.org/tools/idnits/). Boilerplate checks are not
enough; this check needs to be thorough. Has the document met all
formal review criteria it needs to, such as the MIB Doctor, media type
and URI type reviews?

Yes, idnits has been run and generated no errors. No MIB, media type or
URI reviews are required.

(1.h) Has the document split its references into normative and
informative? Are there normative references to documents that are not
ready for advancement or are otherwise in an unclear state? If such
normative references exist, what is the strategy for their completion?
Are there normative references that are downward references, as
described in [RFC3967]? If so, list these downward references to
support the Area Director in the Last Call procedure for them
[RFC3967].

Yes. The document is intended for proposed standard. It has a single
normative reference, RFC2119. All others are informative.

(1.i) Has the Document Shepherd verified that the document IANA
consideration section exists and is consistent with the body of the
document? If the document specifies protocol extensions, are
reservations requested in appropriate IANA registries? Are the IANA
registries clearly identified? If the document creates a new registry,
does it define the proposed initial contents of the registry and an
allocation procedure for future registrations? Does it suggest a
reasonable name for the new registry? See [RFC5226]. If the document
describes an Expert Review process has Shepherd conferred with the
Responsible Area Director so that the IESG can appoint the needed Expert
during the IESG Evaluation?

There is no IANA consideration.

(1.j) Has the Document Shepherd verified that sections of the document
that are written in a formal language, such as XML code, BNF rules,
MIB definitions, etc., validate correctly in an automated checker?

The document contains the source-code for the reference implementation,
in uuencoded format. The chairs have verified that the code from
draft-ietf-codec-opus-11 extracts and compiles on a recent version of
OSX 10.8.

(1.k) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up? Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:


This document specifies a speech and audio codec designed for usage on
the Internet.

Technical Summary
Relevant content can frequently be found in the
abstract and/or introduction of the document. If not, this may be an
indication that there are deficiencies in the abstract or
introduction.

The abstract conveys the technical summary well:

This document defines the Opus interactive speech and audio codec. Opus
is designed to handle a wide range of interactive audio applications,
including Voice over IP, videoconferencing, in-game chat, and even live,
distributed music performances. It scales from low bitrate narrowband
speech at 6 kb/s to very high quality stereo music at 510 kb/s. Opus
uses both linear prediction (LP) and the Modified Discrete Cosine
Transform (MDCT) to achieve good compression of both speech and music.


Working Group Summary
Was there anything in WG process that is worth
noting? For example, was there controversy about particular points or
were there decisions where the consensus was particularly rough?

The chairs believe there is good consensus behind the document,
particularly around the technology. There has not been any significant
disagreement on any of the technical aspects of the codec. However, the
working group has left the detailed specification work to the small
author team. There are several vocal participants who continue to
express dissatisfaction over the testing and codec validation associated
with the work. The WG chairs do not believe that there was consensus to
make these changes.

This document has been the subject of a number of IPR declarations.
See:

Microsoft: https://datatracker.ietf.org/ipr/1670/
Skype: https://datatracker.ietf.org/ipr/1602/
Broadcom: https://datatracker.ietf.org/ipr/1526/
Xiph: https://datatracker.ietf.org/ipr/1524/
Qualcomm: https://datatracker.ietf.org/ipr/1520/
Huawei: https://datatracker.ietf.org/ipr/1712/
Huawei: https://datatracker.ietf.org/ipr/1741/

The WG has had an opportunity to review these disclosures in Last Call
and has opted to proceed with document publication.


Document Quality
Are there existing implementations of the protocol? Have a
significant number of vendors indicated their plan to implement the
specification? Are there any reviewers that merit special mention as
having done a thorough review, e.g., one that resulted in important
changes or a conclusion that the document had no substantive issues?
If there was a MIB Doctor, Media Type or other expert review, what was
its course (briefly)? In the case of a Media Type review, on what date
was the request posted?

There is an existing implementation - the reference implementation which
is included in the appendix of the document and has been maintained by
the authors of the specification. One of the authors developed several
independent decoder implementations in order to help validate the
specification. There are no known alternative encoder
implementations. There are no significant reviewers worth noting beyond
the author team.

The codec has gone through a great degree of testing that demonstrates
its quality. Test results can be found at:
http://datatracker.ietf.org/doc/draft-ietf-codec-results/.


Personnel
Who is the Document Shepherd for this document? Who is the
Responsible Area Director?

Shepherd: Jonathan D. Rosenberg, Ph.D.
Responsible AD: Robert Sparks 

IANA has reviewed draft-ietf-codec-opus-12, which is currently in
Last Call, and has the following comments:

We understand that this document doesn't require any IANA actions.

State changed to In Last Call from Last Call Requested

The following Last Call Announcement was sent out:

From: The IESG

To: IETF-Announce

CC:

Reply-To: ietf@ietf.org

Subject: Last Call:  (Definition of the Opus Audio Codec) to Proposed Standard





The IESG has received a request from the Internet Wideband Audio Codec WG

(codec) to consider the following document:

- 'Definition of the Opus Audio Codec'

  as a Proposed Standard



The IESG plans to make a decision in the next few weeks, and solicits

final comments on this action. Please send substantive comments to the

ietf@ietf.org mailing lists by 2012-05-10. Exceptionally, comments may be

sent to iesg@ietf.org instead. In either case, please retain the

beginning of the Subject line to allow automated sorting.



Abstract





  This document defines the Opus interactive speech and audio codec.

  Opus is designed to handle a wide range of interactive audio

  applications, including Voice over IP, videoconferencing, in-game

  chat, and even live, distributed music performances.  It scales from

  low bitrate narrowband speech at 6 kb/s to very high quality stereo

  music at 510 kb/s.  Opus uses both linear prediction (LP) and the

  Modified Discrete Cosine Transform (MDCT) to achieve good compression

  of both speech and music.









The file can be obtained via

http://datatracker.ietf.org/doc/draft-ietf-codec-opus/



IESG discussion can be tracked via

http://datatracker.ietf.org/doc/draft-ietf-codec-opus/ballot/





The following IPR Declarations may be related to this I-D:



  http://datatracker.ietf.org/ipr/1712/

  http://datatracker.ietf.org/ipr/1602/

  http://datatracker.ietf.org/ipr/1445/

  http://datatracker.ietf.org/ipr/1446/

  http://datatracker.ietf.org/ipr/1447/

  http://datatracker.ietf.org/ipr/1741/

  http://datatracker.ietf.org/ipr/1670/

  http://datatracker.ietf.org/ipr/1520/

  http://datatracker.ietf.org/ipr/1524/

  http://datatracker.ietf.org/ipr/1525/

  http://datatracker.ietf.org/ipr/1526/

Bleah - I fumbled the tracker UI fairly badly this morning - This draft is still in AD Evaluation, please fogive the noise introduced by the incorrect state transitions between Cindy's addition of the shepherd writeup and the transition into AD Evaluation.

>  (1.a) Who is the Document Shepherd for this document? Has the
>        Document Shepherd personally reviewed this version of the
>        document and, in particular, does he or she believe this
>    version is ready for forwarding to the IESG for publication?

Jonathan Rosenberg is the Document Shepherd and has personally reviewed this version of the document and believes it is ready for publication.

>  (1.b) Has the document had adequate review both from key WG members
>    and from key non-WG members? Does the Document Shepherd have
>    any concerns about the depth or breadth of the reviews that
>    have been performed?

The document has received good discussion by the group. It has been reviewed top to bottom by the co-chairs of the group, who have focused on breadth. However, the details of the algorithms, include parameters, have not received deep review by anyone in the group outside of the small author team. This is not unexpected given the technical depth of the work and the domain expertise required to understand the nuances of all aspects of the codec.


>  (1.c) Does the Document Shepherd have concerns that the document
>    needs more review from a particular or broader perspective,
>    e.g., security, operational complexity, someone familiar with
>    AAA, internationalization or XML?

No. The document is very narrowly focused on speech coding, and has no interactions with areas outside of this space.

>  (1.d) Does the Document Shepherd have any specific concerns or
>    issues with this document that the Responsible Area Director
>    and/or the IESG should be aware of? For example, perhaps he
>    or she is uncomfortable with certain parts of the document, or
>    has concerns whether there really is a need for it. In any
>    event, if the WG has discussed those issues and has indicated
>    that it still wishes to advance the document, detail those
>        concerns here. Has an IPR disclosure related to this document
>    been filed? If so, please include a reference to the
>        disclosure and summarize the WG discussion and conclusion on
>    this issue.

The CODEC working group is, in general, a contentious area of work. Much of this is in the area of intellectual property. The ADs need to be aware that IPR declarations have been made against this specification. They are:

Microsoft/Skype: https://datatracker.ietf.org/ipr/1670/
Broadcom: https://datatracker.ietf.org/ipr/1526/
Xiph: https://datatracker.ietf.org/ipr/1524/
Qualcomm: https://datatracker.ietf.org/ipr/1520/
Huawei: https://datatracker.ietf.org/ipr/1712/

Broadcom, Xiph and Skype/Microsoft have all revised their IPR declarations throughout the process; the above represents the most recent set.

The Qualcomm declaration is the most concerning; it does not provide a royalty-free grant. Achieving RF has been a goal of the activity. None of the participants in the working group appear to work for, or represent Qualcomm. The chairs have advised the group that the IETF cannot take positions on validity of IPR claims, and that individuals must make their own determinations on this matter. The AD should be aware that one of the participants from Xiph did post on the list his belief that, after analysis, none of the patents listed in the Qualcomm declaration apply to Opus (http://www.ietf.org/mail-archive/web/codec/current/msg02704.html).

The Huawei declaration is very recent - posted on March 12, 2012. There has not been any discussion of this on the list. The chairs have decided to proceed with publication requested, and give the working group the opportunity to complain during IETF LC if they so wish. Informally, a few participants have indicated that they do not believe this IPR impacts Opus.

Some working group members had concerns (expressed privately to the chairs) about the nature of the RF grants made by Skype; specifically around termination clauses under conditions of lawsuit. These clauses have been modified in the most recent disclosure above

The working group was explicitly pointed to all of these disclosures and was asked whether they wish to proceed given the nature of the disclosures and their terms. The chairs believe there has been consensus to proceed based on the IPR declarations made at the time of WGLC. The only thing that has changed since the consensus call was an updated Microsoft/Skype license that provides more liberal terms (as well as availability under the original terms), and the Huawei statement.

The document has also been the source of commentary from external standards body, most notably ITU, 3gpp and ISO/MPEG. Numerous liaison statements have been exchanged back and forth. The chairs advised these groups of our progress throughout the process, and also explicitly asked for feedback as part of the last call process. The majority of the comments expressed concerns on the testing process followed by the IETF, which lacks the rigor typically seen in more formalized ITU processes. That said, the group itself has consensus that the document should be published.

>  (1.e) How solid is the WG consensus behind this document? Does it
>        represent the strong concurrence of a few individuals, with
>    others being silent, or does the WG as a whole understand and
>    agree with it?

The document has had two WGLC. The first was on 8 July 2011. The second was on 21 October 2011. Comments were received by the group after both of these WGLC. The chairs believe there is good consensus behind the document, particularly around the technology. There are several vocal participants who continue to express dissatisfaction over the testing and codec validation associated with the work. There is also a participant who has expressed dissatisfaction with the applicability of the Qualcomm IPR and has asked for loosening decoder conformance definitions to comply. The chairs believe there was not consensus for such changes.

>  (1.f) Has anyone threatened an appeal or otherwise indicated extreme
>        discontent? If so, please summarise the areas of conflict in
>        separate email messages to the Responsible Area Director. (It
>    should be in a separate email because this questionnaire is
>    entered into the ID Tracker.)

No appeals have been threatened.As noted above, disagreements have taken place primarily in the areas of testing and validation, and around IPR and compliance. We do not see those as outside of the normal scope of disagreement that is common within a working group.

>  (1.g) Has the Document Shepherd personally verified that the
>        document satisfies all ID nits? (See the Internet-Drafts Checklist
>    and http://tools.ietf.org/tools/idnits/). Boilerplate checks are
>    not enough; this check needs to be thorough. Has the document
>    met all formal review criteria it needs to, such as the MIB
>    Doctor, media type and URI type reviews?

Yes, idnits has been run and generated no errors. No MIB, media type or URI reviews are required.

>  (1.h) Has the document split its references into normative and
>    informative? Are there normative references to documents that
>    are not ready for advancement or are otherwise in an unclear
>    state? If such normative references exist, what is the
>        strategy for their completion? Are there normative references
>    that are downward references, as described in [RFC3967]? If
>    so, list these downward references to support the Area
>        Director in the Last Call procedure for them [RFC3967].

Yes. The document is intended for proposed standard. It has a single normative reference, RFC2119. All others are informative.

>  (1.i) Has the Document Shepherd verified that the document IANA
>        consideration section exists and is consistent with the body
>    of the document? If the document specifies protocol
>        extensions, are reservations requested in appropriate IANA
>        registries? Are the IANA registries clearly identified? If
>    the document creates a new registry, does it define the
>        proposed initial contents of the registry and an allocation
>        procedure for future registrations? Does it suggest a
>        reasonable name for the new registry? See [RFC5226]. If the
>        document describes an Expert Review process has Shepherd
>        conferred with the Responsible Area Director so that the IESG
>    can appoint the needed Expert during the IESG Evaluation?

There is no IANA consideration.

>  (1.j) Has the Document Shepherd verified that sections of the
>        document that are written in a formal language, such as XML
>    code, BNF rules, MIB definitions, etc., validate correctly in
>    an automated checker?

The document contains the source-code for the reference implementation, in uuencoded format. The chairs have verified that the code from draft-ietf-codec-opus-11
extracts and compiles on a recent version of OSX 10.8.

>  (1.k) The IESG approval announcement includes a Document
>        Announcement Write-Up. Please provide such a Document
>        Announcement Write-Up? Recent examples can be found in the
>        "Action" announcements for approved documents. The approval
>        announcement contains the following sections:

This document specifies a speech and audio codec designed for usage on the Internet.

> Technical Summary
>        Relevant content can frequently be found in the abstract
>    and/or introduction of the document. If not, this may be
>    an indication that there are deficiencies in the abstract
>    or introduction.

The abstract conveys the technical summary well:

This document defines the Opus interactive speech and audio codec.
  Opus is designed to handle a wide range of interactive audio
  applications, including Voice over IP, videoconferencing, in-game
  chat, and even live, distributed music performances.  It scales from
  low bitrate narrowband speech at 6 kb/s to very high quality stereo
  music at 510 kb/s.  Opus uses both linear prediction (LP) and the
  Modified Discrete Cosine Transform (MDCT) to achieve good compression
  of both speech and music.


>
> Working Group Summary
>    Was there anything in WG process that is worth noting? For
>        example, was there controversy about particular points or
>    were there decisions where the consensus was particularly
>    rough?


See notes above on testing and intellectual property considerations. There hasn’t been any significant disagreement on any of the technical aspects of the codec. The working group has left the detailed specification work to the small author team.


>
> Document Quality
>    Are there existing implementations of the protocol? Have a
>        significant number of vendors indicated their plan to
>        implement the specification? Are there any reviewers that
>    merit special mention as having done a thorough review,
>    e.g., one that resulted in important changes or a
>        conclusion that the document had no substantive issues? If
>    there was a MIB Doctor, Media Type or other expert review,
>    what was its course (briefly)? In the case of a Media Type
>    review, on what date was the request posted?

There is an existing implementation - the reference implementation which is included in the appendix of the document and has been maintained by the authors of the specification. One of the authors developed several independent decoder implementations in order to help validate the specification. There are no known alternative encoder implementations. There are no significant reviewers worth noting beyond the author team.

The codec has gone through a great degree of testing that demonstrates its quality. Test results can be found at: http://datatracker.ietf.org/doc/draft-ietf-codec-results/.
--

Proto Writeup

Hi Robert,

We'd like to request publication of draft-ietf-codec-opus-11 as proposed standard. (http://datatracker.ietf.org/doc/draft-ietf-codec-opus/).

Proto writeup below.

(1.a) Who is the Document Shepherd for this document? Has the
      Document Shepherd personally reviewed this version of the
      document and, in particular, does he or she believe this
  version is ready for forwarding to the IESG for publication?

Jonathan Rosenberg is the Document Shepherd and has personally reviewed this version of the document and believes it is ready for publication.

(1.b) Has the document had adequate review both from key WG members
  and from key non-WG members? Does the Document Shepherd have
  any concerns about the depth or breadth of the reviews that
  have been performed?

The document has received good discussion by the group. It has been reviewed top to bottom by the co-chairs of the group, who have focused on breadth. However, the details of the algorithms, include parameters, have not received deep review by anyone in the group outside of the small author team. This is not unexpected given the technical depth of the work and the domain expertise required to understand the nuances of all aspects of the codec.


(1.c) Does the Document Shepherd have concerns that the document
  needs more review from a particular or broader perspective,
  e.g., security, operational complexity, someone familiar with
  AAA, internationalization or XML?

No. The document is very narrowly focused on speech coding, and has no interactions with areas outside of this space.

(1.d) Does the Document Shepherd have any specific concerns or
  issues with this document that the Responsible Area Director
  and/or the IESG should be aware of? For example, perhaps he
  or she is uncomfortable with certain parts of the document, or
  has concerns whether there really is a need for it. In any
  event, if the WG has discussed those issues and has indicated
  that it still wishes to advance the document, detail those
      concerns here. Has an IPR disclosure related to this document
  been filed? If so, please include a reference to the
      disclosure and summarize the WG discussion and conclusion on
  this issue.

The CODEC working group is, in general, a contentious area of work. Much of this is in the area of intellectual property. The ADs need to be aware that IPR declarations have been made against this specification. They are:

Microsoft/Skype: https://datatracker.ietf.org/ipr/1670/
Broadcom: https://datatracker.ietf.org/ipr/1526/
Xiph: https://datatracker.ietf.org/ipr/1524/
Qualcomm: https://datatracker.ietf.org/ipr/1520/
Huawei: https://datatracker.ietf.org/ipr/1712/

Broadcom, Xiph and Skype/Microsoft have all revised their IPR declarations throughout the process; the above represents the most recent set.

The Qualcomm declaration is the most concerning; it does not provide a royalty-free grant. Achieving RF has been a goal of the activity. None of the participants in the working group appear to work for, or represent Qualcomm. The chairs have advised the group that the IETF cannot take positions on validity of IPR claims, and that individuals must make their own determinations on this matter. The AD should be aware that one of the participants from Xiph did post on the list his belief that, after analysis, none of the patents listed in the Qualcomm declaration apply to Opus (http://www.ietf.org/mail-archive/web/codec/current/msg02704.html).

The Huawei declaration is very recent - posted on March 12, 2012. There has not been any discussion of this on the list. The chairs have decided to proceed with publication requested, and give the working group the opportunity to complain during IETF LC if they so wish. Informally, a few participants have indicated that they do not believe this IPR impacts Opus.

Some working group members had concerns (expressed privately to the chairs) about the nature of the RF grants made by Skype; specifically around termination clauses under conditions of lawsuit. These clauses have been modified in the most recent disclosure above

The working group was explicitly pointed to all of these disclosures and was asked whether they wish to proceed given the nature of the disclosures and their terms. The chairs believe there has been consensus to proceed based on the IPR declarations made at the time of WGLC. The only thing that has changed since the consensus call was an updated Microsoft/Skype license that provides more liberal terms (as well as availability under the original terms), and the Huawei statement.

The document has also been the source of commentary from external standards body, most notably ITU, 3gpp and ISO/MPEG. Numerous liaison statements have been exchanged back and forth. The chairs advised these groups of our progress throughout the process, and also explicitly asked for feedback as part of the last call process. The majority of the comments expressed concerns on the testing process followed by the IETF, which lacks the rigor typically seen in more formalized ITU processes. That said, the group itself has consensus that the document should be published.

(1.e) How solid is the WG consensus behind this document? Does it
      represent the strong concurrence of a few individuals, with
  others being silent, or does the WG as a whole understand and
  agree with it?

The document has had two WGLC. The first was on 8 July 2011. The second was on 21 October 2011. Comments were received by the group after both of these WGLC. The chairs believe there is good consensus behind the document, particularly around the technology. There are several vocal participants who continue to express dissatisfaction over the testing and codec validation associated with the work. There is also a participant who has expressed dissatisfaction with the applicability of the Qualcomm IPR and has asked for loosening decoder conformance definitions to comply. The chairs believe there was not consensus for such changes.

(1.f) Has anyone threatened an appeal or otherwise indicated extreme
      discontent? If so, please summarise the areas of conflict in
      separate email messages to the Responsible Area Director. (It
  should be in a separate email because this questionnaire is
  entered into the ID Tracker.)

No appeals have been threatened.As noted above, disagreements have taken place primarily in the areas of testing and validation, and around IPR and compliance. We do not see those as outside of the normal scope of disagreement that is common within a working group.

(1.g) Has the Document Shepherd personally verified that the
      document satisfies all ID nits? (See the Internet-Drafts Checklist
  and http://tools.ietf.org/tools/idnits/). Boilerplate checks are
  not enough; this check needs to be thorough. Has the document
  met all formal review criteria it needs to, such as the MIB
  Doctor, media type and URI type reviews?

Yes, idnits has been run and generated no errors. No MIB, media type or URI reviews are required.

(1.h) Has the document split its references into normative and
  informative? Are there normative references to documents that
  are not ready for advancement or are otherwise in an unclear
  state? If such normative references exist, what is the
      strategy for their completion? Are there normative references
  that are downward references, as described in [RFC3967]? If
  so, list these downward references to support the Area
      Director in the Last Call procedure for them [RFC3967].

Yes. The document is intended for proposed standard. It has a single normative reference, RFC2119. All others are informative.

(1.i) Has the Document Shepherd verified that the document IANA
      consideration section exists and is consistent with the body
  of the document? If the document specifies protocol
      extensions, are reservations requested in appropriate IANA
      registries? Are the IANA registries clearly identified? If
  the document creates a new registry, does it define the
      proposed initial contents of the registry and an allocation
      procedure for future registrations? Does it suggest a
      reasonable name for the new registry? See [RFC5226]. If the
      document describes an Expert Review process has Shepherd
      conferred with the Responsible Area Director so that the IESG
  can appoint the needed Expert during the IESG Evaluation?

There is no IANA consideration.

(1.j) Has the Document Shepherd verified that sections of the
      document that are written in a formal language, such as XML
  code, BNF rules, MIB definitions, etc., validate correctly in
  an automated checker?

The document contains the source-code for the reference implementation, in uuencoded format. The chairs have verified that the code from draft-ietf-codec-opus-11
extracts and compiles on a recent version of OSX 10.8.

(1.k) The IESG approval announcement includes a Document
      Announcement Write-Up. Please provide such a Document
      Announcement Write-Up? Recent examples can be found in the
      "Action" announcements for approved documents. The approval
      announcement contains the following sections:

This document specifies a speech and audio codec designed for usage on the Internet.

Technical Summary
      Relevant content can frequently be found in the abstract
  and/or introduction of the document. If not, this may be
  an indication that there are deficiencies in the abstract
  or introduction.

The abstract conveys the technical summary well:

This document defines the Opus interactive speech and audio codec.
Opus is designed to handle a wide range of interactive audio
applications, including Voice over IP, videoconferencing, in-game
chat, and even live, distributed music performances.  It scales from
low bitrate narrowband speech at 6 kb/s to very high quality stereo
music at 510 kb/s.  Opus uses both linear prediction (LP) and the
Modified Discrete Cosine Transform (MDCT) to achieve good compression
of both speech and music.



Working Group Summary
  Was there anything in WG process that is worth noting? For
      example, was there controversy about particular points or
  were there decisions where the consensus was particularly
  rough?


See notes above on testing and intellectual property considerations. There hasn’t been any significant disagreement on any of the technical aspects of the codec. The working group has left the detailed specification work to the small author team.



Document Quality
  Are there existing implementations of the protocol? Have a
      significant number of vendors indicated their plan to
      implement the specification? Are there any reviewers that
  merit special mention as having done a thorough review,
  e.g., one that resulted in important changes or a
      conclusion that the document had no substantive issues? If
  there was a MIB Doctor, Media Type or other expert review,
  what was its course (briefly)? In the case of a Media Type
  review, on what date was the request posted?

There is an existing implementation - the reference implementation which is included in the appendix of the document and has been maintained by the authors of the specification. One of the authors developed several independent decoder implementations in order to help validate the specification. There are no known alternative encoder implementations. There are no significant reviewers worth noting beyond the author team.

The codec has gone through a great degree of testing that demonstrates its quality. Test results can be found at: http://datatracker.ietf.org/doc/draft-ietf-codec-results/.
--
Jonathan D. Rosenberg, Ph.D.                 
jdrosen@jdrosen.net