Skip to main content

H.264 as Mandatory to Implement Video Codec for WebRTC
draft-burman-rtcweb-h264-proposal-00

The information below is for an old version of the document.
Document Type
This is an older version of an Internet-Draft whose latest revision state is "Expired".
Authors Bo Burman , Markus Isomaki , Dr. Bernard D. Aboba , Gaelle Martin-Cocher , Giridhar Mandyam
Last updated 2012-10-15
RFC stream (None)
Formats
Stream Stream state (No stream defined)
Consensus boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-burman-rtcweb-h264-proposal-00
RTCWEB Working Group                                           B. Burman
Internet-Draft                                                  Ericsson
Intended status: Standards Track                              M. Isomaki
Expires: April 18, 2013                                            Nokia
                                                                B. Aboba
                                                   Microsoft Corporation
                                                        G. Martin-Cocher
                                                                     RIM
                                                              G. Mandyam
                                              Qualcomm Innovation Center
                                                        October 15, 2012

         H.264 as Mandatory to Implement Video Codec for WebRTC
                  draft-burman-rtcweb-h264-proposal-00

Abstract

   This document proposes that, and motivates why, H.264 should be a
   Mandatory To Implement video codec for WebRTC.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on April 18, 2013.

Copyright Notice

   Copyright (c) 2012 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect

Burman, et al.           Expires April 18, 2013                 [Page 1]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  3
   3.  H.264 Overview . . . . . . . . . . . . . . . . . . . . . . . .  3
   4.  Implementations  . . . . . . . . . . . . . . . . . . . . . . .  3
   5.  Licensing  . . . . . . . . . . . . . . . . . . . . . . . . . .  4
   6.  Performance  . . . . . . . . . . . . . . . . . . . . . . . . .  5
   7.  Profile/level  . . . . . . . . . . . . . . . . . . . . . . . .  6
   8.  Negotiation  . . . . . . . . . . . . . . . . . . . . . . . . .  7
   9.  Summary  . . . . . . . . . . . . . . . . . . . . . . . . . . .  8
   10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . .  9
   11. Security Considerations  . . . . . . . . . . . . . . . . . . .  9
   12. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . .  9
   13. References . . . . . . . . . . . . . . . . . . . . . . . . . .  9
     13.1.  Normative References  . . . . . . . . . . . . . . . . . .  9
     13.2.  Informative References  . . . . . . . . . . . . . . . . .  9
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 10

Burman, et al.           Expires April 18, 2013                 [Page 2]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

1.  Introduction

   The selection of a Mandatory To Implement (MTI) video codec for
   WebRTC has been discussed for quite some time in the RTCWEB WG.  This
   document proposes that the H.264 video codec should be mandatory to
   implement for WebRTC implementations and gives motivation to this
   proposal.

   The core of the proposal is that H.264 Constrained Baseline Profile
   Level 1.2 MUST be supported as Mandatory To Implement video codec.
   To enable higher quality for devices capable of it, support for H.264
   High Profile Level 1.3, extended to support 720p resolution at 30 Hz
   framerate is RECOMMENDED.

   This draft discusses the advantages of H.264 as the authors of this
   draft see them; a richness of implementations and hardware support,
   well known licensing conditions, good performance, and well defined
   handling of varying device capabilities.

2.  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in BCP 14, RFC 2119
   [RFC2119].

3.  H.264 Overview

   The video coding standard Advanced Video Coding (ITU-T H.264 | ISO/
   IEC 14496-10 [H264]) has been around for almost ten years by now.
   Developed jointly by MPEG and ITU-T in the Joint Video Team, it was
   published in its first version in 2003 and amended with support for
   higher-fidelity video in 2004.  Other significant updates include
   support for scalability (2007) and multiview (2009).  The codec goes
   under the names H.264, AVC and MPEG-4 Part10.  In this memo the term
   "H.264" will be used.

   H.264 was from the start very successful and has become widely
   adopted for (video) content as well as (video) communication services
   worldwide.

4.  Implementations

   Arguably, hardware or DSP acceleration for video encoding/decoding
   would be mostly beneficial for devices that has relatively lower

Burman, et al.           Expires April 18, 2013                 [Page 3]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

   capacity in terms of CPU and power (smaller batteries), and the most
   common devices in this category are phones and tablets.  There is a
   long list of vendors offering hardware or DSP implementations of
   H.264.  In particular all vendors of platforms for mobile high-range
   phones, smartphones, and tablets support H.264/AVC High Profile
   encoding and decoding at least 1080p30, but those platforms are
   currently in general not used for low- to mid-range devices.  These
   vendors are ST-Ericsson, Qualcomm, TI, Nvidia, Renesas, Mediatek,
   Huawei Hisilicon, Intel, Broadcom, Samsung.  Those platforms all
   support H.264/AVC codec with dedicated HW or DSP.  For at least the
   ST-Ericsson and Qualcomm hardware it is verified that the
   implementation has low-delay real-time support, but it seems likely
   that this is the case for at least the majority of the others as
   well.

   Regarding software implementations there is a long list of available
   implementations.  Wikipedia provides an illustration of this with
   their list [Implementations], and more implementations appear, e.g.
   [Woon].  Not only are there standalone implementations available,
   including open source, but in addition recent Windows and Mac OS X
   versions support H.264 encoding and decoding.

5.  Licensing

   H.264 is a mature codec with a mature and well-known licensing model.
   MPEG-LA released their AVC Patent Portfolio License already in 2004
   and in 2010 they announced that H.264 encoded Internet video is free
   to end users will never be charged royalties [MPEGLA].  Real-time
   generated content, the content most applicable to WebRTC, was free
   already from the establishment of the MPEG-LA license.  License fees
   for products that decode and encode H.264 video remain though.  Those
   fees are, and will very likely continue to be for the lifetime of
   MPEG-LA pool, $0.20 per codec or less.  It can be noted that for
   MPEG-LA, since one license covers both an encoder and decoder, there
   is no additional cost of using an encoder to an implementation that
   supports decoding of H.264.

   It is a well-established fact that not all H.264 right holders are
   MPEG-LA pool members.  H.264 is however an ITU/ISO/IEC international
   standard, developed under their respective patent policies, and all
   contributors must license their patents under Reasonable And Non-
   Discriminatory (RAND) terms.  In the field of video coding, most
   major research groups interested in patents do contribute to the ITU/
   ISO/IEC standards process and are therefore bound by those terms.

   VP8 is a much younger codec than H.264 and it is fair to say that the
   licensing situation is less clear than for H.264.  Google has

Burman, et al.           Expires April 18, 2013                 [Page 4]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

   provided their patent rights on VP8 under a open source friendly
   license with very restrictive reciprocity conditions.  According to
   MPEG-LA's web page [MpegLaVp8], MPEG-LA is in the process of forming
   a royalty-bearing patent pool for VP8.  Also, according to press
   reports [DoJ], at least the US Department of Justice investigate
   MPEG-LA for anticompetitive activity in conjunction with the VP8 pool
   formation.  This indicates that the licensing situation for VP8 has
   not settled.

6.  Performance

   Comparing video quality is difficult.  Practically no modern video
   encoding method includes any bit-exact encoding where a given (video)
   input produces a specified encoded output bitstream.  Instead, the
   encoded bitstream syntax and semantics are specified such that a
   decoder can correctly interpret it and produce a known output.  This
   is true both for H.264 and VP8.  Significant freedom is left to the
   encoder implementation to choose how to represent the encoded video,
   for example given a specific targeted bitrate.  Thus it cannot in
   general be expected that any encoded video bitstream represents the
   best possible or most efficient representation, given the defined
   bitstream syntax elements available to that codec.  The actually
   achieved quality for a certain bitstream, how close it is to the
   optimally possible with available syntax, at any given bitrate rather
   depends on the performance of the individual encoder implementation.

   Also, not only is the resulting experienced video quality subjective,
   but also depends on the source material, on the point of operation
   and a number of other considerations.  In addition, performance can
   be measured vs. bitrate, but also vs. e.g. complexity - and here
   another can of worms can be opened because complexity depends on
   hardware used (some platforms have video codec accelerations), SW
   platform (and how efficient it can use the hardware) and so on.  On
   top of this comes that different implementations can have different
   performance, and can be operated in different ways (e.g. tradeoffs
   between complexity and quality can be made).  Regardless of how a
   performance evaluation is carried out it can always be said that it
   is not "fair".  This section nevertheless attempts to shed some light
   on this subject, and specifically the performance (measured against
   bitrate) of H.264 compared to VP8.

   A number of studies [H264perf1][H264perf2][H264perf3] have been made
   to compare the compression efficiency performance between H.264 and
   VP8.  These studies show that H.264 is in general performing better
   than VP8 but the studies are not specifically targeting video
   conferencing.  Therefore, Ericsson made a comparison where a number
   of video conferencing type sequences were encoded using both H.264

Burman, et al.           Expires April 18, 2013                 [Page 5]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

   and VP8.  Eight video conferencing type test sequences were used;
   three were taken from the MPEG/ITU test set (vidyo2-4) and five were
   recorded by Ericsson.  The sequences were all 720p 25/30Hz.

   The focus of that test was to evaluate the best compression
   efficiency that could be achieved with both codecs since it was
   believed to be harder to make a fair comparison trying to use
   complexity constraints.  The results showed that H.264 High Profile
   provides an average bitrate compared to VP8 of -23% (minus here means
   that H.264 is better) using PSNR-based Bjontegaard Delta bitrate (BD-
   rate) [PSNRdiff].  H.264 Constrained High Profile provided -16% and
   Constrained Baseline Profile resulted in +16% (plus here means that
   VP8 is better).

   For H.264, JM 18.3 in low-delay mode without reordering of B or P
   pictures was used.  For VP8 encoding, v1.1.0 with the "best" preset
   was used.

   Again, video quality is difficult to compare.  The authors however
   believe that the data provided in this section shows that H.264 is at
   least on par with VP8.  As a final note, the new HEVC standard
   clearly outperforms both of them, but the authors think it is
   premature to mandate HEVC for WebRTC.

7.  Profile/level

   H.264 [H264] has a large number of encoding tools, grouped in
   functionally reasonable toolsets by codec profiles, and a wide range
   of possible implementation capability and complexity, specified by
   codec levels.  It is typically not reasonable for H.264 encoders and
   decoders to implement maximum complexity capability for all of the
   available tools.  Thus, any H.264 decoder implementation is typically
   not able to receive all possible H.264 streams.  Which streams can be
   received is described by what profile and level the decoder conforms
   to.  Any video stream produced by an H.264 encoder must keep within
   the limits defined by the intended receiving decoder's profile and
   level to ensure that the video stream can be correctly decoded.

   Profiles can be "ranked" in terms of the amount of tools included,
   such that some profiles with few tools are "lower" than profiles with
   more tools.  However, profiles are typically not strictly supersets
   or subsets of each other in terms of which tools are used, so a
   strict ranking cannot be defined.  It is also in some cases possible
   to express compliance to the common subset of tools between two
   different profiles.  This is fairly well described in [RFC6184].

   When choosing a Mandatory To Implement codec, it is desirable to use

Burman, et al.           Expires April 18, 2013                 [Page 6]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

   a profile and level that is as widely supported as possible.
   Therefore, H.264 Constrained Baseline Profile Level 1.2 MUST be
   supported as Mandatory To Implement video codec.  This is possible to
   support with significant margin in hardware devices Section 4 and
   should likely also not cause performance problems for software-only
   implementations.  All Level definitions (Annex A of [H264]) include a
   maximum framesize in macroblocks (16*16 pixels) as well as a maximum
   processing requirement in macroblocks per second.  That number of
   macroblocks per second can be almost freely distributed between
   framesize and framerate.  The maximum framesize for Level 1.2
   corresponds to 352*288 pixels (CIF).  Examples of allowed framesize
   and framerate combinations for Level 1.2 are CIF (352*288 pixels) at
   15 Hz, QVGA (320*240 pixels) at 20 Hz, and QCIF (176*144 pixels) at
   60 Hz.

   Recognizing that while the above profile and level will likely be
   possible to implement in any device, it is also likely not sufficient
   for applications that require higher quality.  Therefore, it is
   RECOMMENDED that devices and implementations that can meet the
   additional requirements also implement at least H.264 High Profile
   Level 1.3, extended to support 720p resolution at 30 Hz framerate,
   but the extension MAY alternatively be made from any Level higher
   than 1.3.

   Note that the lowest non-extended Level that support 720p30 is Level
   3.1, but fully supporting Level 3.1 also requires fairly high
   bitrate, large buffers, and other encoding parameters included in
   that Level definition that are likely not reasonable for the targeted
   communication scenario.  This method of extending a lower level with
   a smaller set of applicable parameters is already used by some video
   conferencing vendors.

8.  Negotiation

   Given that there exist a fairly large set of defined profiles and
   levels (Section 7), the probability is rather low that randomly
   chosen H.264 encoder and decoder implementations have exactly
   matching capabilities.  In any communication scenario, there is
   therefore a need for a decoder to be able to convey its maximum
   supported profile and level that the encoder must not exceed.

   In addition and depending on the wanted use case and the conditions
   that apply at a certain communication instance, there may also be a
   need to describe the currently wanted profile and level at the start
   of the communication session, which may be lower than the maximum
   supported by the implementation.  In this scenario it may also be of
   interest to communicate from the encoder to the decoder both which

Burman, et al.           Expires April 18, 2013                 [Page 7]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

   profile and level that will actually be used and what is the maximum
   supported profile and level.  The reason to communicate not only the
   starting point but also the maximum assumes that communication
   conditions may change during the conditions, maybe multiple times,
   possibly making another profile and level be a more appropriate
   choice.

   Communication of maximum supported profile and level is the only
   mandatory SDP [RFC4566] parameter in the H.264 payload format
   [RFC6184], which also includes a large set of optional parameters,
   describing available use (decoder) and intended use (encoder) of
   those parameters for a specific offered [RFC3264] stream.

   If the above mentioned (Section 7) capability for 720p30 is supported
   as an extension to High Profile Level 1.3 (or higher), the level
   extension SHOULD be signaled in SDP using the following parameters as
   defined in section 8.1 of [RFC6184]:

   o  profile-level-id=64000d (or corresponding to a higher Level of
      High profile)

   o  max-fs=3600 (or greater)

   o  max-mbps=108000 (or greater)

   o  max-br=768 (or greater, whatever the device implementation can
      support)

9.  Summary

   H.264 is widely adopted and used for a large set of video services.
   This in turn is because H.264 offers great performance, reasonable
   licensing terms (and manageable risks).  As a consequence of its
   adoption for many services, a multitude implementations in software
   and hardware are available.  Another result of the widespread
   adoption is that all associated technologies, such as payload
   formats, negotiation mechanisms and so on are well defined and
   standardized.  In addition, using H.264 enables interoperability with
   many other services without video transcoding.

   We therefore propose to the WG that H.264 shall be mandatory to
   implement for all WebRTC endpoints that support video, according to
   the details described in Section 7.

Burman, et al.           Expires April 18, 2013                 [Page 8]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

10.  IANA Considerations

   This document makes no request of IANA.

   Note to RFC Editor: this section may be removed on publication as an
   RFC.

11.  Security Considerations

   No specific considerations apply to the information in this document.

12.  Acknowledgements

   All that provided valuable descriptions, comments and insights about
   the H.264 codec on the IETF mailing lists.

13.  References

13.1.  Normative References

   [H264]     ITU-T Recommendation H.264, "Advanced video coding for
              generic audiovisual services", March 2010.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC3264]  Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
              with Session Description Protocol (SDP)", RFC 3264,
              June 2002.

   [RFC4566]  Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
              Description Protocol", RFC 4566, July 2006.

   [RFC6184]  Wang, Y., Even, R., Kristensen, T., and R. Jesup, "RTP
              Payload Format for H.264 Video", RFC 6184, May 2011.

13.2.  Informative References

   [DoJ]      "Web Video Rivalry Sparks U.S. Probe", <http://
              online.wsj.com/article/
              SB10001424052748703752404576178833590548792.html>.

   [H264perf1]
              Vatolin, D., "MPEG-4 AVC/H.264 Video Codecs Comparison
              2010 - Appendixes",  , May 2010, <http://

Burman, et al.           Expires April 18, 2013                 [Page 9]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

              compression.graphicon.ru/video/codec_comparison/h264_2010/
              appendixes.html#Appendix_8>.

   [H264perf2]
              Shah, K., "Implementation, performance analysis and
              comparison of VP8 and H.264.", University of Texas at
              Arlington Department of Electrical Engineering, 2011, <htt
              p://www-ee.uta.edu/Dip/Courses/EE5359/
              2011SpringFinalReportPPT/
              Shah_EE5359Spring2011FinalPPT.pdf>.

   [H264perf3]
              De Simone, F., Goldmann, L., Lee, J., and T. Ebrahimi,
              "Performance analysis of VP8 image and video compression
              based on subjective evaluations", Ecole Polytechnique
              F'd'rale de Lausanne (EPFL) , Aug 2011, <http://
              infoscience.epfl.ch/record/168259/files/article.pdf>.

   [Implementations]
              Wikipedia, "H.264/MPEG-4 AVC products and
              implementations", October 2012, <http://en.wikipedia.org/
              wiki/H.264/MPEG-4_AVC_products_and_implementations>.

   [MPEGLA]   "MPEG LAs AVC License Will Not Charge Royalties for
              Internet Video that is Free to End Users through Life of
              License", MPEGLA News Release, August 2010, <www.mpegla.co
              m/Lists/MPEG%20LA%20News%20List/Attachments/231/
              n-10-08-26.pdf>.

   [MpegLaVp8]
              "MPEG LA Announces Call for Patents Essential to VP8 Video
              Codec",
              <http://www.mpeg-la.com/main/pid/vp8/default.aspx>.

   [PSNRdiff]
              Bjontegaard, G., "Calculation of Average PSNR Differences
              between RD-Curves", ITU-T SG16 Q.6 Document VCEG-M33,
              April 2001.

   [Woon]     "Polycom Delivers Open Standards-Based Scalable Video
              Coding (SVC) Technology, Royalty-Free to Industry",
              October 2012, <http://www.polycom.com/content/www/en/
              company/news/press-releases/2012/20121004.html>.

Burman, et al.           Expires April 18, 2013                [Page 10]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

Authors' Addresses

   Bo Burman
   Ericsson
   Farogatan 6
   Stockholm  16480
   Sweden

   Email: bo.burman@ericsson.com

   Markus Isomaki
   Nokia
   Keilalahdentie 2-4
   Espoo,   FI-02150
   Finland

   Phone:
   Fax:
   Email: markus.isomaki@nokia.com
   URI:

   Bernard Aboba
   Microsoft Corporation
   One Microsoft Way
   Redmond, WA  98052
   US

   Phone:
   Fax:
   Email: bernard_aboba@hotmail.com
   URI:

   Gaelle Martin-Cocher
   RIM
   1875 Buckhorn Gate
   Mississauga, ON  L4W 5P1
   Canada

   Phone:
   Fax:
   Email: gmartincocher@rim.com
   URI:

Burman, et al.           Expires April 18, 2013                [Page 11]
Internet-Draft        H.264 as Mandatory in WebRTC          October 2012

   Giri Mandyam
   Qualcomm Innovation Center

   Phone:
   Fax:
   Email: mandyam@quicinc.com
   URI:

Burman, et al.           Expires April 18, 2013                [Page 12]