Audio/Video Transport WG T. Kristensen Internet-Draft TANDBERG Updates: 3984 (if approved) January 2007 Intended status: Standards Track Expires: July 5, 2007 Extensions for RDCO and Static Macroblocks in the RTP Payload Format for H.264 Video draft-kristensen-avt-rtp-h264-extension-00 Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on July 5, 2007. Copyright Notice Copyright (C) The IETF Trust (2007). Abstract This document proposes extensions to the RTP payload format specification for H.264 video defined in RFC 3984. It addresses two separate issues: (i) The signalling of the framerate of static macroblocks a decoder is able to process in order to sustain high framerates with high resolutions. (ii) The signalling of a reduced- complexity decoding operation (RCDO) for H.264 Baseline profile Kristensen Expires July 5, 2007 [Page 1]
Internet-Draft H.264 RTP RDCO and MaxStaticMBPS January 2007 bitstreams. Table of Contents 1. Requirements notation . . . . . . . . . . . . . . . . . . . . 3 2. Introduction and Overview . . . . . . . . . . . . . . . . . . 3 3. Static macroblocks . . . . . . . . . . . . . . . . . . . . . . 3 3.1. Payload Format Parameters . . . . . . . . . . . . . . . . 4 3.2. MIME Registration . . . . . . . . . . . . . . . . . . . . 4 3.3. SDP Parameters . . . . . . . . . . . . . . . . . . . . . . 5 4. Reduced-Complexity Decoding Operation (RCDO) . . . . . . . . . 5 4.1. Payload Format Parameters . . . . . . . . . . . . . . . . 5 4.2. MIME Registration . . . . . . . . . . . . . . . . . . . . 6 4.3. SDP Parameters . . . . . . . . . . . . . . . . . . . . . . 7 4.3.1. Mapping of MIME Parameters to SDP . . . . . . . . . . 7 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 6. Security Considerations . . . . . . . . . . . . . . . . . . . 8 7. Normative References . . . . . . . . . . . . . . . . . . . . . 8 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 8 Intellectual Property and Copyright Statements . . . . . . . . . . 10 Kristensen Expires July 5, 2007 [Page 2]
Internet-Draft H.264 RTP RDCO and MaxStaticMBPS January 2007 1. Requirements notation The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [1]. 2. Introduction and Overview ITU-T H.264 [5] and ITU-T H.241 [6], its associated video procedures and signalling recommendation, continue to evolve. The IETF RTP payload formats and parameters need to be updated to include important new functionalities not covered in RFC 3984 [3]. The two extensions described in this Internet-Draft have a common goal: They both offer a solution to support higher resolutions at the same high framerates used in current implementations, but with reduced processing requirements, compared to today's needs. One approach is to utilize the fact that portions of macroblocks in the image are static. Another approach is to reduce the complexity and thus the decoding cost/resource consumption of the video processing. Note: The two extensions described below may later be split into separate drafts, one for static macroblock signalling and one for RCDO. The two approaches are already addressed in the latest version of H.241 [6]. This proposal defines MIME parameters for them, a new MIME subtype for RCDO and allows their use in SDP. The payload format parameters specified in this draft may be considered for inclusion in the ongoing RTP payload format work for H.264/SVC [4]. 3. Static macroblocks Running H.264 encode and decode operations require large amounts of video processing power. The challenge being to sustain high framerate (e.g. 30 frames/sec) with the large framesizes that stems from recent demand for HD resolution. In practice, a certain amount of macroblocks in the video stream, which do not change in a frame, can be defined as static and, this way, free up additional processing cycles for the handling of non-static macroblocks. Based on a given amount of video processing resources and a given framerate, a higher number of static macroblocks enables a correspondingly higher resolution. A new parameter MaxStaticMBPS (maximum static macroblocks per second) was introduced in H.241 to Kristensen Expires July 5, 2007 [Page 3]
Internet-Draft H.264 RTP RDCO and MaxStaticMBPS January 2007 address this issue. The MaxStaticMBPS parameter is specified in Section 8.3.2.8 of H.241 [6]. 3.1. Payload Format Parameters The optional MIME parameter max-smbps specified below comes in addition to the list of optional MIME parameters for the H264 MIME subtype defined in Section 8.1 of RFC 3984 [3], the H264-RCDO MIME subtype in Section 4 of this draft, and eventually the H264-SVC MIME subtype specified in Section 9.1 of the "RTP Payload Format for SVC video" draft [4]. 3.2. MIME Registration A new OPTIONAL parameter max-smbps is introduced to supplement the payload format parameters described in Section 8 of RFC 3984 [3]. As it is the case with max-mbps, max-fs, max-cpb, max-dpb, and max-br; max-smbps MAY be used to signal the capabilities of a receiver implementation. These parameters MUST NOT be used for any other purpose. The profile-level-id parameter MUST be present in the same receiver capability description that contains any of these parameters. max-smbps: The value of max-smbps is an integer indicating the maximum static macroblock processing rate in units of static macroblocks per second, under the hypothetical assumption that all macroblocks are static macroblocks. When max-smbps is signalled the MaxMBPS value in Table A-1 of H.264 [5] should be replaced with the result of the following computation: 1. If the parameter max-mbps is signalled, set a variable MaxMacroblocksPerSecond to the value of max-mbps. Otherwise, set MaxMacroblocksPerSecond equal to the value of MaxMBPS for the level in Table A-1 of H.264 [5]. 2. Set a variable P_non-static to the proportion of non-static macroblocks in picture n. 3. Set a variable P_static to the proportion of static macroblocks in picture n. 4. The value of MaxMBPS in Table A-1 of H.264 [5] should be considered by the encoder to be equal to: 1 ----------------------------------------- P_non-static P_static ------------------------- + ----------- MaxMacroblocksPerSecond max-smbps The encoder should recompute this value for each picture. Kristensen Expires July 5, 2007 [Page 4]
Internet-Draft H.264 RTP RDCO and MaxStaticMBPS January 2007 The value of max-smbps MUST be greater than the value of MaxMBPS for the level given in Table A-1 of H.264 [5]. Senders MAY use this knowledge to send pictures of a given size at a higher picture rate than is indicated in the signalled level. Formal MIME extensions TBD. 3.3. SDP Parameters In terms of mapping of MIME parameters to SDP, the parameter max- smbps is treated the same way as max-mbps, when used in both the SDP Offer/Answer model [2] and declarative session descriptions. Details TBD. 4. Reduced-Complexity Decoding Operation (RCDO) The Reduced-Complexity Decoding Operation (RCDO) for H.264 Baseline profile bitstreams is specified in Annex B of H.241 [6]. RCDO is specified as a separate H.264 mode, and is distinct from any profile defined in H.264. A RCDO bitstream obey to all the constraints of the Baseline profile. In order to signal H.264 additional modes the parameter AdditionalModesSupported is specified in Table 9f of H.241 [6]. Currently, the only mode defined is RCDO. Informational note: Other additional modes may be defined in the future. However, as H.264 additional modes may or may not be distinct from the Profiles in H.264 - these modes would require separate extensions RFC 3984 [3]. To maintain backward compatibility with existing H.264 implementations, this draft proposes a separate MIME subtype for RCDO. 4.1. Payload Format Parameters Section 8 of RFC 3984 [3] applies with the following modification. The sentence ''The parameters are specified here as part of the MIME subtype registration for the ITU-T H.264 | ISO/IEC 14496-10 codec.'' is replaced with Kristensen Expires July 5, 2007 [Page 5]
Internet-Draft H.264 RTP RDCO and MaxStaticMBPS January 2007 ''The parameters are specified here as part of the MIME subtype registration for the RCDO codec.'' Further details TBD. 4.2. MIME Registration Editorial note: Complete formal MIME specification TBD. Decide whether to include the RFC 2984 MIME registration or simply refer to it. For now we mainly describe the changes and differences. The MIME subtype for the ITU-T H.264 | ISO/IEC 14496-10 codec is allocated from the IETF tree. The receiver MUST ignore any unspecified parameter. Media Type name: video Media subtype name: H264-RCDO Required parameters: none OPTIONAL parameters: The optional MIME parameters specified in RFC 3984 [3] and Section 3.1 in this draft apply, with the following constraints: profile-level-id: RCDO is distinct from any profile, this implies that the profile value 0 (no profile) and the profile_idc byte of the profile-level-id parameter are equal to 0. A RCDO bitstream MUST obey to all the constraints of the Baseline profile. Therefore, only constraint_set0_flag is equal to 1 in the profile- iop part of the profile-level-id parameter, the remaining bits are set to 0. For example, if a codec supports level 2.1, the profile-level-id becomes 00800d, in which 00 indicates the "no profile" value, 80 indicates the constraints of the Baseline profile and 0d indicates level 1.3. When level 2.1 is supported, the profile-level-id becomes 008015. If no profile-level-id is present, level 1 MUST be implied, i.e. equivalent to profile-level-id 00800a. Kristensen Expires July 5, 2007 [Page 6]
Internet-Draft H.264 RTP RDCO and MaxStaticMBPS January 2007 Encoding considerations: This type is only defined for transfer via RTP (RFC 3550). Security considerations: See section X of RFC YYYY. Public specification: Please refer to section X of RFC YYYY. Additional information: None File extensions: None Macintosh file type code: None Object identifier or OID: None Person & email address to contact for further information: TBD Intended usage: COMMON Author: TBD Change controller: IETF Audio/Video Transport working group delegated from the IESG. 4.3. SDP Parameters Regarding the usage in the SDP Offer/Answer model [2] and in declarative session descriptions; details TBD. 4.3.1. Mapping of MIME Parameters to SDP The MIME media type video/H264-RCDO string is mapped to fields in the Session Description Protocol (SDP) as follows: o The media name in the "m=" line of SDP MUST be video. o The encoding name in the "a=rtpmap" line of SDP MUST be H264-RCDO (the MIME subtype). o The clock rate in the "a=rtpmap" line MUST be 90000. o The OPTIONAL parameters "profile-level-id", "max-mbps", "max- smbps", "max-fs", "max-cpb", "max-dpb", "max-br", "redundant-pic- cap", "sprop-parameter-sets", "parameter-add", "packetization- mode", "sprop-interleaving-depth", "deint-buf-cap", "sprop-deint- buf-req", "sprop-init-buf-time", "sprop-max-don-diff", and "max- rcmd-nalu-size", when present, MUST be included in the "a=fmtp" line of SDP. These parameters are expressed as a MIME media type string, in the form of a semicolon separated list of parameter=value pairs. Kristensen Expires July 5, 2007 [Page 7]
Internet-Draft H.264 RTP RDCO and MaxStaticMBPS January 2007 An example of media representation of a level 2 bitstream is as follows: m=video 54321 RTP/AVP 101 a=rtpmap:101 H264-RCDO/90000 a=fmtp:101 profile-level-id=008014;max-mbps=60000;max-smbps=360000 5. IANA Considerations TBD. Refer to Section 3.2 and Section 4.2 for updated/new registration of MIME types. 6. Security Considerations No separate considerations introduced in this document. Refer to section 9 of RFC 3984 [3]. 7. Normative References [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [2] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002. [3] Wenger, S., Hannuksela, M., Stockhammer, T., Westerlund, M., and D. Singer, "RTP Payload Format for H.264 Video", RFC 3984, February 2005. [4] Wenger, S., "RTP Payload Format for SVC Video", draft-ietf-avt-rtp-svc-00 (work in progress), December 2006. [5] International Telecommunications Union, "Advanced video coding for generic audiovisual services", ITU-T Recommendation H.264, March 2005. [6] International Telecommunications Union, "Extended video procedures and control signals for H.300-series terminals", ITU- T Recommendation H.241, May 2006. Kristensen Expires July 5, 2007 [Page 8]
Internet-Draft H.264 RTP RDCO and MaxStaticMBPS January 2007 Author's Address Tom Kristensen TANDBERG Philip Pedersens vei 22 N-1366 Lysaker Norway Phone: +47 67125125 Email: tom.kristensen@tandberg.net, tomkri@ifi.uio.no URI: http://www.tandberg.net Kristensen Expires July 5, 2007 [Page 9]
Internet-Draft H.264 RTP RDCO and MaxStaticMBPS January 2007 Full Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). Kristensen Expires July 5, 2007 [Page 10]