Network Working Group JM. Valin
Internet-Draft Octasic Inc.
Intended status: Standards Track S. Borilin
Expires: July 25, 2011 SPIRIT DSP
K. Vos
Skype Technologies S.A.
C. Montgomery
Xiph.Org Foundation
R. Chen
Broadcom Corporation
January 21, 2011
Guidelines for the Codec Development Within the IETF
draft-ietf-codec-guidelines-00
Abstract
This document provides general guidelines for work on developing and
specifying a codec within the IETF. These guidelines cover the
development process, evaluation, requirements conformance, and
intellectual property issues.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on July 25, 2011.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
Valin, et al. Expires July 25, 2011 [Page 1]
Internet-Draft Codec Guidelines January 2011
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Development Process . . . . . . . . . . . . . . . . . . . . . 4
3. Evaluation, Testing, and Characterization . . . . . . . . . . 7
4. Requirements Conformance . . . . . . . . . . . . . . . . . . . 8
5. Intellectual Property . . . . . . . . . . . . . . . . . . . . 10
6. Relationship with Other SDOs . . . . . . . . . . . . . . . . . 12
7. Security Considerations . . . . . . . . . . . . . . . . . . . 14
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15
9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 16
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 17
10.1. Normative References . . . . . . . . . . . . . . . . . . 17
10.2. Informative References . . . . . . . . . . . . . . . . . 17
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 19
Valin, et al. Expires July 25, 2011 [Page 2]
Internet-Draft Codec Guidelines January 2011
1. Introduction
This document describes a suggested process for work at the IETF on
standardization of a codec that is optimized for use in interactive
Internet applications and that can be widely implemented and easily
distributed among application developers, service operators, and end
users.
Valin, et al. Expires July 25, 2011 [Page 3]
Internet-Draft Codec Guidelines January 2011
2. Development Process
The process outlined here is intended to make the work on an audio
codec within the IETF transparent, predictable, and well organized.
Such work might involve development of a completely new codec,
adaptation of an existing codec to meet the requirements, or
integration between two or more existing codecs that results in an
improved codec combining the best aspects of each codec. To enable
such procedural transparency, the contributor of an existing codec
must be willing to cede change control to the IETF and should have
sufficient knowledge of the codec to assist in the work of adapting
it or applying some of its technology to the development or
improvemnet of other codecs. Furthermore, contributors need to be
aware that any codec that results from work within the IETF is likely
to be different from any existing codec that was contributed to the
Internet Standards Process.
Work on codec development is expected to proceed as follows:
1. IETF participants will identify the requirements to be met by an
Internet codec, in the form of an Internet-Draft.
2. Interested parties are encouraged to make contributions proposing
existing or new codecs, or elements thereof, to the codec WG as
long as these contributions are within the scope of the WG.
Ideally, these contributions should be in the form of Internet
Drafts, although other forms of contributions are also possible
as discussed in [PROCESS] and in the IETF's Note Well. As
always, contributions to the IETF are subject, among other
process oriented RFCs, to [PROCESS], [TRUST], and [IPR].
Considering the field of technology, IPR transparency may be
particularly high on the priority list of many codec WG
participants. Accordingly, contributors are specifically
reminded of their IPR disclosure requirement, and all
participants are reminded of the solicitation of the disclosure
of third party IPR, both as codified in [IPR].
3. As contributions are received and discussed within the working
group, the group should gain a clearer understanding of what is
achievable within the design space. As a result, the authors of
the requirements document should iteratively clarify and improve
their document to reflect the emerging working group consensus.
This is likely to involve collaboration with IETF working groups
in other areas, such as collaboration with working groups in the
Transport area to identify important aspects of packet
transmission over the Internet and to understand the degree of
rate adaptation desirable, and with working groups in the RAI
area to ensure that information about and negotiation of the
Valin, et al. Expires July 25, 2011 [Page 4]
Internet-Draft Codec Guidelines January 2011
codec can be easily represented at the signalling layer. In
parallel with this work, interested parties should evaluate the
contributions at a higher level to see which requirements might
be met by each codec.
4. Once a sufficient number of proposals has been received, the
interested parties will identify the strengths, weaknesses, and
innovative aspects of the contributed codecs. This step will
consider not only the codecs as a whole, but also key features of
the individual algorithms (predictors, quantizers, transforms,
etc.).
5. It is expected that none of the contributed codecs will meet all
of the defined requirements. Therefore, it is expected that IETF
participants will accept a _baseline codec_ as a WG item to
facilitate the development process. This baseline codec will
meet as many of the requirements as possible, but probably will
need to be adjusted through an iterative development process in
order to meet all of the requirements (or as many requirements as
possible). The baseline codec might be one of the contributed
codecs (especially if it is the only codec that meets most of the
requirements), a combination of two or more of the contributed
codecs, or an entirely new codec. None of the decisions taken at
this step will be definitive. In particular, IETF participants
will not provide a "rubber stamp" for any contributed codec.
6. IETF participants should then attempt to iteratively improve each
component of the baseline codec reference implementation, where
by "component" we mean individual algorithms such as predictors,
transforms, quantizers, and entropy coders. The participants
should proceed by trying new designs, applying ideas from the
contributed codecs, evaluating "proof of concept" ideas, and
using their expertise in codec development to improve the
baseline codec. Any aspect of the baseline codec might be
changed (even the fundamental principles of the codec) or the
participants might start over entirely by scrapping the baseline
codec and designing a completely new one. The overriding goal
shall be to design a codec that will meet the requirements
defined in the requirements document. Given the IETF's open
standards process, any interested party will be able to
contribute to this work, whether or not they submitted an
Internet-Draft for one of the contributed codecs. The codec
itself should be normatively specified with code in an Internet-
Draft.
7. In parallel with work on the codec reference implementation,
developers and other interested parties should perform evaluation
of the codec as described under Section 3, IETF participants
Valin, et al. Expires July 25, 2011 [Page 5]
Internet-Draft Codec Guidelines January 2011
should define (within the AVT Working Group) the codec's payload
format for use with the Real-time Transport Protocol [RTP], and
application developers should start testing the codec by
implementing it in code and deploying it in actual Internet
applications to identify any potential problems.
8. Once IETF participants agree that the codec being developed meets
the requirements, IETF participants can begin the task of
characterizing the codec. The characterization process is
described under Section 3.
Valin, et al. Expires July 25, 2011 [Page 6]
Internet-Draft Codec Guidelines January 2011
3. Evaluation, Testing, and Characterization
Lab evaluation of the codec being developed should happen throughout
the development process because it will help ensure that progress is
being made toward fulfillment of the requirements. There are many
ways in which continuous evaluation can be performed. For minor,
uncontroversial changes to the codec it should usually be sufficient
to use objective measurements (e.g., PESQ, PEAQ, and SegSNR)
validated by informal subjective evaluation. For more complex
changes (e.g., when psychoacoustic aspects are involved) or for
controversial issues, internal testing should be performed. An
example of internal testing would be to have individual participants
rate the decoded samples using one of the established testing
methodologies, such as ITU-R BS.1534 (MUSHRA).
Throughout the process, it will be important to make use of the
Internet community at large for real-world distributed testing. This
will enable many different people with different equipment and use
cases to test the codec and report any problems they experience. In
the same way, third-party software developers will be encouraged to
integrate the codec (with a warning about the bit-stream not being
final) and provide feedback on its performance in real-world use
cases.
Characterization of the final codec must be based on the reference
implementation only (and not on any "private implementation"). This
can be performed by independent testing labs or, if this is not
possible, using the testing labs of the organizations that contribute
to the Internet Standards Process. Packet loss robustness should be
evaluated using actual loss patterns collected from use over the
Internet, rather than theoretical models. The goals of the
characterization phase are to:
o ensure that the requirements have been fulfilled
o guide the IESG in its evaluation of the resulting work
o assist application developers in understanding whether the codec
is suitable for a particular application
The exact methodology for the characterization phase is still subject
to discussion within the working group.
Valin, et al. Expires July 25, 2011 [Page 7]
Internet-Draft Codec Guidelines January 2011
4. Requirements Conformance
It is the responsibility of the working group to define criteria for
evaluating conformance, including but not limited to comparison tools
and test vectors. The following text provides suggestions for
consideration by the working group:
1. Any codec specified by the IETF must include source code for a
normative C89 implementation, documented in an Internet Draft
destined for standards track RFC. This implementation will be
used to verify conformance of an implementation. Although a text
description of the algorithm should be provided, its use should
be limited to helping the reader in understanding the source
code. Should the description contradict the source code, the
latter shall take precedence. For convenience, the source code
may be provided in compressed form, with base64 encoding.
2. Because of the size of the codec's source code, it is possible
that even after publishing the RFC, bugs would be found from time
to time. An errata of the RFC and its software description
should be maintained, along with a public software repository
containing the current reference implementation.
3. It is the intention of the group to allow the greatest possible
choice of freedom in implementing the specification.
Accordingly, the number of binding RFC2119 keywords is going to
be the minimum still allowing for interopable implementations.
In practice this generally means that only the decoder needs to
be normative, so that the encoder can improve over time. This
also enables different tradeoffs between quality and complexity.
4. To reduce the risk of bias towards certain CPU/DSP architectures,
ideally the decoder specification should not require "bit-exact"
conformance with the reference implementation. The output of a
decoder implementation should only be "close enough" to the
output of the reference decoder. A comparison tool should be
provided along with the codec to verify objectively that the
output of a decoder is likely to be perceptually
indistinguishable from that of the reference decoder. However,
an implementation may still wish to produce an output that is
bit-exact with the reference implementation to simplify the
testing procedure.
5. To ensure freedom of implementation, decoder-side only error
concealment does not need to be specified, although the reference
implementation should include the same PLC algorithm as used in
the testing phase. Is it up to the working group to decide
whether minimum requirements on PLC quality will be required for
Valin, et al. Expires July 25, 2011 [Page 8]
Internet-Draft Codec Guidelines January 2011
compliance with the specification. Obviously, any information
signaled in the bitstream intended to aid PLC needs to be
specified.
6. An encoder implementation should not be required to make use of
all the "features" (tools) in the bit-stream definition.
However, the codec specification may require that an encoder
implementation be able to generate any possible bit-rate. Unless
a particular "profile" is defined in the specification, the
decoder must be able to decode all features of the bit-stream.
The decoder must also be able to handle any combination of bits,
even combinations that cannot be generated by the reference
encoder. It is recommended that the decoder specification shall
define exactly how the decoder should react to "impossible"
packets. However, an encoder must never generate such packets
that do not conform to the bit-stream definition.
7. Compressed test vectors should be provided as a means to verify
conformance with the decoder specification. These test vectors
should exercise all paths in the decoder (100% code coverage).
8. While the exact encoder will not be specified, it is recommended
to specify objective measurement targets for an encoder, below
which use of a particular encoder implementation is not
recommended. For example, one such specification could be: "the
use of an encoder whose PESQ MOS is less than 0.1 below the
reference encoder in the following conditions is not
recommended".
Valin, et al. Expires July 25, 2011 [Page 9]
Internet-Draft Codec Guidelines January 2011
5. Intellectual Property
Producing an unencumbered codec is desirable for the following
reasons:
o It is the experience of a wide variety of application developers
and service providers that encumbrances such as licensing and
royalties make it difficult to implement, deploy, and distribute
audio applications for use by the Internet community.
o It is beneficial to have low-cost options whenever possible
because standalone voice services are being commoditized and
small, innovative development teams often cannot afford to pay
per-channel licensing fees and royalties.
o Many market segments are moving away from selling hard-coded
hardware devices and toward freely distributing end-user software;
this is true of numerous large application providers and even
telcos themselves.
o Compatibility with the licensing of typical open source
applications implies the need to avoid encumbrances, including
even the requirement to obtain a license for implementation,
deployment, or use (even if the license does not require the
payment of a fee).
Therefore, a codec that can be widely implemented and easily
distributed among application developers, service operators, and end
users is preferred. Many existing codecs that might fulfill some or
most of the technical attributes listed above are encumbered in
various ways. For example, patent holders might require that those
wishing to implement the codec in software, deploy the codec in a
service, or distribute the codec in software or hardware need to
request a license, enter into a business agreement, pay licensing
fees or royalties, or adhere to other special conditions or
restrictions. Because such encumbrances have made it difficult to
widely implement and easily distribute high-quality audio codecs
across the entire Internet community, the working group prefers
unencumbered technologies in a way that is consistent with BCP 78 and
BCP 79. In particular, the working group shall heed the preference
stated in BCP 79: "In general, IETF working groups prefer
technologies with no known IPR claims or, for technologies with
claims against them, an offer of royalty-free licensing." Although
this preference cannot guarantee that the working group will produce
an unencumbered codec, the working group shall follow BCP 79, and
adhere to the spirit of BCP 79. The working group cannot explicitly
rule out the possibility of adopting encumbered technologies;
however, the working group will try to avoid encumbered technologies
Valin, et al. Expires July 25, 2011 [Page 10]
Internet-Draft Codec Guidelines January 2011
that require royalties or other encumbrances that would prevent such
technologies from being easy to redistribute and use.
The following guidelines will help to maximize the odds that the
codec will be unencumbered:
1. In accordance with BCP 79 [IPR], contributed codecs should
preferably use technologies with no known IPR claims or
technologies with an offer of royalty-free (RF) licensing.
2. Whenever possible, the working group should use technologies that
are perceived by the participants to be safer with regard to IPR
issues.
3. Contributors must disclose IPR as specified in BCP 79.
4. In cases where no RF license can be obtained regarding a patent,
the group should consider alternative algorithms or methods, even
if they result in lower quality, higher complexity, or otherwise
less desirable characteristics (in most cases, the degradation
will likely be small once the best alternative has been
identified).
5. In accordance with BCP 78 [TRUST], the source code for the
reference implementation must be made available under a BSD-style
license (or whatever license is defined as acceptable by the IETF
Trust when the Internet-Draft defining the reference
implementation is published).
IETF participants should be aware that, given the way patents work in
most countries, the resulting codec can never be guaranteed to be
free of patent claims because some patents may not be known to the
contributors, some patent applications may not be disclosed at the
time the codec is developed, and only courts of law can determine the
validity and breadth of patent claims. However, these observations
are no different within the Internet Standards Process than they are
for standardization of codecs within other SDOs (or development of
codecs outside the context of any SDO), and furthermore are no
different for codecs than for other technologies worked on within the
IETF. In all these cases, the best approach is to minimize the risk
of unknowingly incurring encrumbrance on existing patents. Despite
these precautions, participants need to understand that, practically
speaking, it is nearly impossible to _guarantee_ that implementors
will not incur encumbrance on existing patents.
Valin, et al. Expires July 25, 2011 [Page 11]
Internet-Draft Codec Guidelines January 2011
6. Relationship with Other SDOs
It is understood that other SDOs are also involved in the codec
development and standardization, including but not necessarily
limited to:
o The Telecommunication Standardization Sector (ITU-T) of the
International Telecommunication Union (ITU), in particular Study
Group 16
o The Moving Picture Experts Group (MPEG)
o The European Telecommunications Standards Institute (ETSI)
o The 3rd Generation Partnership Project (3GPP)
o The 3rd Generation Partnership Project 2 (3GPP2)
It is important to ensure that such work does not constitute
uncoordinated protocol development, of the kind described in
[UNCOORD] in the following principle:
[T]he IAB considers an essential principle of the protocol
development process that only one SDO maintains design authority
for a given protocol, with that SDO having ultimate authority over
the allocation of protocol parameter code-points; defining the
intended semantics, interpretation, and actions associated with
those code-points.
The work envisioned by this guidelines document is not
"uncoordinated" in the sense described in the foregoing quote, for
the following reasons:
o Internet signalling technologies are designed to enable the
negotiation of any codecs that are supported in a particular
application (such signalling technologies include the Session
Initiation Protocol [SIP], Session Description Protocol [SDP], and
the Extensible Messaging and Presence Protocol [XMPP] extensions
for media negotiation as specified in [Jingle]).
o Internet transport technologies such as the Real-time Transport
Protocol [RTP] (including secure transport as described in [SRTP])
are designed to support any codec for which RTP packetization
rules have been defined.
o The IETF codec working group will focus on issues that are
specific to the Internet, including robustness to packet loss and
other aspects of packet transmission over the Internet. Issues
Valin, et al. Expires July 25, 2011 [Page 12]
Internet-Draft Codec Guidelines January 2011
that are specific to non-Internet transports (e.g., radio
communication and circuit-switched networks) are specifically out
of scope.
Although there is already sufficient codec expertise available among
IETF participants to complete the envisioned work, additional
contributions are welcome within the framework of the Internet
Standards Process, in the following ways:
o Individuals who are technical contributors to codec work within
other SDOs can participate directly in codec work within the IETF.
o Other SDOs can contribute their expertise (e.g., codec
characterization and evaluation techniques) and thus facilitate
the testing of a codec produced by the IETF.
o Any SDO can provide input to IETF work through liaison statements.
However, it is important to note that final responsibility for the
development process and the resulting codec will remain with the IETF
as governed by BCP 9 [PROCESS].
Finally, there is precedent for the contribution of codecs developed
elsewhere to the ITU-T (e.g., AMR Wideband was standardized
originally within 3GPP). This is a model to explore as the IETF
coordinates further with the ITU-T in accordance with the
collaboration guidelines defined in [COLLAB].
Valin, et al. Expires July 25, 2011 [Page 13]
Internet-Draft Codec Guidelines January 2011
7. Security Considerations
The procedural guidelines for codec development do not have security
considerations. However, the resulting codec needs to take
appropriate security considerations into account, for example as
outlined in [DOS] and [SECGUIDE].
Valin, et al. Expires July 25, 2011 [Page 14]
Internet-Draft Codec Guidelines January 2011
8. IANA Considerations
This document has no actions for IANA.
Valin, et al. Expires July 25, 2011 [Page 15]
Internet-Draft Codec Guidelines January 2011
9. Acknowledgments
We would like to thank all the other people who contributed directly
or indirectly to this document, including Jason Fischl, Gregory
Maxwell, Alan Duric, Jonathan Christensen, Julian Spittka, Michael
Knappe, Timothy Terriberry, Christian Hoene, Stephan Wenger and Henry
Sinnreich. We also like to thank Cullen Jennings and Gregory
Lebovitz for their advice. Special thanks to Peter Saint-Andre, who
originally co-authored this document.
Valin, et al. Expires July 25, 2011 [Page 16]
Internet-Draft Codec Guidelines January 2011
10. References
10.1. Normative References
[IPR] Bradner, S., "Intellectual Property Rights in IETF
Technology", BCP 79, RFC 3979, March 2005.
[PROCESS] Bradner, S., "The Internet Standards Process -- Revision
3", BCP 9, RFC 2026, October 1996.
[TRUST] Bradner, S. and J. Contreras, "Rights Contributors Provide
to the IETF Trust", BCP 78, RFC 5378, November 2008.
10.2. Informative References
[COLLAB] Fishman, G. and S. Bradner, "Internet Engineering Task
Force and International Telecommunication Union -
Telecommunications Standardization Sector Collaboration
Guidelines", RFC 3356, August 2002.
[DOS] Handley, M., Rescorla, E., and IAB, "Internet Denial-of-
Service Considerations", RFC 4732, December 2006.
[Jingle] Ludwig, S., Saint-Andre, P., Egan, S., McQueen, R., and D.
Cionoiu, "Jingle RTP Sessions", XSF XEP 0167, June 2009.
[RTP] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[SDP] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.
[SECGUIDE]
Rescorla, E. and B. Korver, "Guidelines for Writing RFC
Text on Security Considerations", BCP 72, RFC 3552,
July 2003.
[SIP] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
A., Peterson, J., Sparks, R., Handley, M., and E.
Schooler, "SIP: Session Initiation Protocol", RFC 3261,
June 2002.
[SRTP] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K.
Norrman, "The Secure Real-time Transport Protocol (SRTP)",
RFC 3711, March 2004.
[UNCOORD] Bryant, S. and M. Morrow, "Uncoordinated Protocol
Valin, et al. Expires July 25, 2011 [Page 17]
Internet-Draft Codec Guidelines January 2011
Development Considered Harmful", RFC 5704, November 2009.
[XMPP] Saint-Andre, P., Ed., "Extensible Messaging and Presence
Protocol (XMPP): Core", RFC 3920, October 2004.
Valin, et al. Expires July 25, 2011 [Page 18]
Internet-Draft Codec Guidelines January 2011
Authors' Addresses
Jean-Marc Valin
Octasic Inc.
4101, Molson Street
Montreal, Quebec
Canada
Email: jean-marc.valin@octasic.com
Slava Borilin
SPIRIT DSP
Email: borilin@spiritdsp.net
Koen Vos
Skype Technologies S.A.
Stadsgarden 6
Stockholm, 11645
Sweden
Email: koen.vos@skype.net
Christopher Montgomery
Xiph.Org Foundation
Email: xiphmont@xiph.org
Raymond (Juin-Hwey) Chen
Broadcom Corporation
Email: rchen@broadcom.com
Valin, et al. Expires July 25, 2011 [Page 19]
