AVT                                                            J. Lennox
Internet-Draft                                                     Vidyo
Updates: 3984 (if approved)                            November 11, 2007
Intended status: Standards Track
Expires: May 14, 2008


    Source-Specific Media Format Parameters for H.264 and H.264 SVC
                  draft-lennox-avt-h264-source-fmtp-00

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on May 14, 2008.

Copyright Notice

   Copyright (C) The IETF Trust (2007).

Abstract

   When used in the Session Description Protocol Offer/Answer model,
   several of the media format parameters for the H.264 video format,
   and for its Scalabile Video Codec (SVC) extension, describe
   characterics of the stream an endpoint is prepared to send, not of
   streams it is prepared to receive.  If an endpoint wishes to send
   multiple streams, these parameters may be incompatible.  This
   document defines how such media format parameters may be given on a



Lennox                    Expires May 14, 2008                  [Page 1]


Internet-Draft   H.264 Source-Specific Format Parameters   November 2007


   per-source basis, using SDP source-specific fmtp attributes.


Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . . 3
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 3
   3.  Overview  . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
   4.  Mapping of MIME parameters to SDP source attributes . . . . . . 4
     4.1.  Parameter Sets  . . . . . . . . . . . . . . . . . . . . . . 4
     4.2.  Packetization Mode 2 Parameters . . . . . . . . . . . . . . 5
     4.3.  Capability Parameters . . . . . . . . . . . . . . . . . . . 5
     4.4.  H264 SVC Parameters . . . . . . . . . . . . . . . . . . . . 5
     4.5.  Other Parameters  . . . . . . . . . . . . . . . . . . . . . 6
   5.  Examples  . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
   6.  Backward Compatibility  . . . . . . . . . . . . . . . . . . . . 6
   7.  Security Considerations . . . . . . . . . . . . . . . . . . . . 7
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 7
   9.  Normative References  . . . . . . . . . . . . . . . . . . . . . 7
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . . . 7
   Intellectual Property and Copyright Statements  . . . . . . . . . . 9






























Lennox                    Expires May 14, 2008                  [Page 2]


Internet-Draft   H.264 Source-Specific Format Parameters   November 2007


1.  Introduction

   Unlike many media packetization formats for the Real-Time Transport
   Protocol (RTP) [RFC3550], the the RTP packetization specifications
   for H.264 [RFC3984] and for H.264's Scalable Video Coding extension
   [I-D.ietf-avt-rtp-svc] define a number of MIME media type parameters
   that, when encoded in SDP [RFC4566], define characteristics of the
   media stream an endpoint is prepared to send, not of the streams it
   is prepared to receive.  Most notably, streams' parameter sets
   (initial data required to correctly initialize a decoder) are encoded
   in SDP messages and sent out-of-band, to ensure that they are
   reliably received by a decoder before decoding begins.

   Because RTP is fundamentally a group communication protocol, however,
   an RTP session may contain many different media streams.  In this
   case, different H.264 and H.264 SVC streams in an RTP session may
   need to be described by different and incompatible values for these
   MIME media type parameters.  For example, an endpoint may be
   switching between video streams encoded by separate video encoders.
   In this case, it is not possible, using only the mechanisms of
   [RFC3984], to describe all the sources and to send their parameter
   sets out-of-band.  An endpoint must instead fall back to sending
   parameter sets in-band, and retransmitting them with high enough
   frequency that there is a reasonably high likelihood of their being
   receceived successfully.

   To solve this difficulty, this document uses the framework introduced
   by [I-D.lennox-mmusic-sdp-source-attributes] to describe MIME media
   format parameters of individual H.264 sources in SDP.  This enables
   all the benefits of out-of-band H.264 source description in the case
   when multiple H.264 sources will be sent in an RTP session.


2.  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119] and
   indicate requirement levels for compliant implementations.


3.  Overview

   When used with the SDP [RFC4566] offer/answer model [RFC3264],
   several of the media format parameters of H.264 [RFC3984] and H.264
   SVC [I-D.ietf-avt-rtp-svc] define characteristics of an RTP stream
   (media source) to be sent, not of a session receiver's capabilities.
   When multiple media sources are in use, it is sometimes useful to



Lennox                    Expires May 14, 2008                  [Page 3]


Internet-Draft   H.264 Source-Specific Format Parameters   November 2007


   describe source characteristics individually for each source.

   Per-source media format parameters are defined using the fmtp source
   parameter [I-D.lennox-mmusic-sdp-source-attributes].  This document
   updates the MIME type registration of video/H264 in [RFC3984] and of
   video/H264-SVC in [I-D.ietf-avt-rtp-svc] to specify the encoding of
   the MIME media format parameters into the fmtp source attribute.


4.  Mapping of MIME parameters to SDP source attributes

   H.264 MIME media format parmaeters applicable to a specific source
   are are encoded into an "fmtp" source attribute for the H.264 payload
   type and the source being described.  These parameters are expressed
   in the form of a semicolon-separated list of parameter=value pairs,
   the same syntax as the media-level fmtp value.  For the purposes of
   discussion in this document, MIME media format parameters encoded
   into a source-specific fmtp attribute are called "source-specific
   parameters", while MIME media format parameters encoded into a media-
   level format attribute are called "media parameters".

   Source-specific parameters only describe characteristics of a
   specific source might send, while media parameters describe all
   sources of a stream.  Source-specific parameters do not override
   media parameters, though they do in some cases further constrain them
   or provide additional information.

4.1.  Parameter Sets

   The "sprop-parameter-sets" parameter encodes H.264 sequence parameter
   set (SPS) and picture parameter set (PPS) Network Adaptation Layer
   NAL) units.  These parameter sets provide essential information
   necessary to decode an H.264 bitstream; encoding them in SDP ensures
   that they are delivered reliably.

   The "sprop-parameter-sets" parameter MAY be encoded as a source
   parameter.  However, if the sprop-parameter-sets parameter is also
   present as a media parameter, the H.264 parameter sets described for
   each source MUST include all the H.264 parameter set described for
   the media.

   In an SDP answer, the "sprop-parameter-sets" for a source MUST follow
   the same constraints as for the media.  I.e., the parameter sets for
   a source described in an answer MUST be a superset of the parameter
   sets for the media in the offer, and if an offer indicates
   "parameter-add=0" (false) for the media, the corresponding answer
   MUST NOT add additonal parameter sets for any source.




Lennox                    Expires May 14, 2008                  [Page 4]


Internet-Draft   H.264 Source-Specific Format Parameters   November 2007


4.2.  Packetization Mode 2 Parameters

   The "sprop-deint-buf-req", "sprop-interleaving-depth", "sprop-max-
   don-diff", and "sprop-init-buf-time" parameters describe
   characteristics of H.264 media streams using packetization mode 2
   (interleaved mode).  According to [RFC3984], the first two are
   mandatory for packetization mode 2 streams; the other two are
   optional.  They specify the maximum size and time of the debuffering
   needed to deinterleave streams sent in interleaved mode.

   These parameters MAY be included as source parameters, overriding any
   corresponding values at the media level for the source.  However, if
   they are, they MUST be less than or equal to the value specified for
   the parameter (if any) at the media level.  All constraints for these
   values specified by [RFC3984] still apply.

4.3.  Capability Parameters

   As defined in [RFC3984], the meaning of the capability parameters
   ("max-mbps", "max-fs", "max-cpb", "max-dpb", "max-br", "redundant-
   pic-cap", and "max-rcmd-nalu-size") at the media level depends on a
   media stream's "direction" attribute.  When the "direction" attribute
   is "sendonly", then the parameters describe the limits of media
   sources that the sender is capable of producing.  When the
   "direction" attribute is "sendrecv" or "recvonly", then the
   parameters describe the limitations of what the receiver accepts.

   When encoded as source parameters, these parameters always describe
   the limits of the source being described.  In media streams whose
   "direction" is "sendonly", these parameters MUST be less than or
   equal to the values (if any) in the media parameters.  In media
   streams whose "direction" is "sendrecv", source parameters for these
   values are unconstrained by stream's media parameters (which describe
   what the endpoint is willing to receive).  However, in offer/answer
   mode, the values of these source parameters MUST be less than or
   equal to the values given in media parameters in the most recent
   (accepted) offer or answer for the stream.

   (Media streams whose "direction" is "recvonly" do not encode any
   sources.)

4.4.  H264 SVC Parameters

   The H.264 SVC parameters "sprop-scalability-info" and "sprop-layer-
   ids" are defined in [I-D.ietf-avt-rtp-svc].  They describe the
   scalability structure of an H.264 Scalable Video Codec stream.

   These parameters MUST NOT be given both as source parameters and



Lennox                    Expires May 14, 2008                  [Page 5]


Internet-Draft   H.264 Source-Specific Format Parameters   November 2007


   media parameters.  They MUST be specified at one level at most.

4.5.  Other Parameters

   The parameters "profile-level-id", "packetization-mode", and
   "parameter-add" MUST NOT be used as a source parameters.


5.  Examples

   This section gives examples of SDP descriptions of media sessions
   containing H.264 source parameters.  For brevity, only the media
   sections of the descriptions are given.

   m=video 49170 RTP/AVP 96
   a=rtpmap:96 H264/90000
   a=fmtp:96 packetization-mode=1
   a=ssrc:12345 cname:cif-stream@example.com
   a=ssrc:12345 fmtp:96 sprop-parameter-sets=J0LgDZWWFglk,KM4Ecg==
   a=ssrc:67890 cname:vga-stream@example.com
   a=ssrc:67890 fmtp:96 sprop-parameter-sets=J0LgDZWWCgPZ,KM4Ecg==

       Figure 1: Example: declaration of two sources with different
                              parameter sets

   The example in Figure 1 shows two H.264 streams with different,
   incompatbile parameter sets.  (Specifically, the two streams encode
   different image sizes.)

   (TODO: add more examples.)


6.  Backward Compatibility

   Unless a sender knows via some mechanism (not specified here or in
   [I-D.lennox-mmusic-sdp-source-attributes]) that a receiver definitely
   understands source parameters, it MUST send any parameter sets
   specified in sprop-parameter-sets source attributes in-band in the
   media stream as well.  These parameter sets SHOULD be transmitted
   frequently enough that a receiver has a high probability of receiving
   them even in the presence of packet loss.

   For H.264 SVC streams, the scalability information Supplemental
   Encoder Information (SEI) message encoded in an sprop-scalability-
   info parameter set SHOULD be similarly transmitted in-band as well.






Lennox                    Expires May 14, 2008                  [Page 6]


Internet-Draft   H.264 Source-Specific Format Parameters   November 2007


7.  Security Considerations

   Source-specific encoding of media format parameters does not add any
   additional security considerations beyond those of [RFC3984] and
   [I-D.ietf-avt-rtp-svc].


8.  IANA Considerations

   This document updates the SDP encoding of the video/H264 MIME media
   type specified in [RFC3984], and of the video/H264-SVC MIME media
   type specified in [I-D.ietf-avt-rtp-svc].


9.  Normative References

   [I-D.ietf-avt-rtp-svc]
              Wenger, S., "RTP Payload Format for SVC Video",
              draft-ietf-avt-rtp-svc-02 (work in progress), July 2007.

   [I-D.lennox-mmusic-sdp-source-attributes]
              Lennox, J., "Source-Specific Media Attributes in the
              Session Description Protocol (SDP)",
              draft-lennox-mmusic-sdp-source-attributes-01 (work in
              progress), July 2007.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC3264]  Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
              with Session Description Protocol (SDP)", RFC 3264,
              June 2002.

   [RFC3550]  Schulzrinne, H., Casner, S., Frederick, R., and V.
              Jacobson, "RTP: A Transport Protocol for Real-Time
              Applications", STD 64, RFC 3550, July 2003.

   [RFC3984]  Wenger, S., Hannuksela, M., Stockhammer, T., Westerlund,
              M., and D. Singer, "RTP Payload Format for H.264 Video",
              RFC 3984, February 2005.

   [RFC4566]  Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
              Description Protocol", RFC 4566, July 2006.








Lennox                    Expires May 14, 2008                  [Page 7]


Internet-Draft   H.264 Source-Specific Format Parameters   November 2007


Author's Address

   Jonathan Lennox
   Vidyo, Inc.
   433 Hackensack Avenue
   Sixth Floor
   Hackensack, NJ  07601
   US

   Email: jonathan@vidyo.com









































Lennox                    Expires May 14, 2008                  [Page 8]


Internet-Draft   H.264 Source-Specific Format Parameters   November 2007


Full Copyright Statement

   Copyright (C) The IETF Trust (2007).

   This document is subject to the rights, licenses and restrictions
   contained in BCP 78, and except as set forth therein, the authors
   retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
   THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
   OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   ietf-ipr@ietf.org.


Acknowledgment

   Funding for the RFC Editor function is provided by the IETF
   Administrative Support Activity (IASA).





Lennox                    Expires May 14, 2008                  [Page 9]