Internet Engineering Task Force Yoshihiro Kikuchi - Toshiba
Internet Draft Toshiyuki Nomura - NEC
Document: draft-ietf-avt-rtp-mpeg4-es-02.txt Shigeru Fukunaga - Oki
Yoshinori Matsui - Matsushita
Hideaki Kimata - NTT
July 6, 2000
RTP payload format for MPEG-4 Audio/Visual streams
Status of this Memo
This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC2026 [1].
Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF), its areas, and its working groups. Note that other groups
may also distribute working documents as Internet-Drafts. Internet-Drafts
are draft documents valid for a maximum of six months and may be updated,
replaced, or obsoleted by other documents at any time. It is
inappropriate to use Internet- Drafts as reference material or to cite
them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Abstract
This document describes RTP payload formats for carrying of MPEG-4 Audio
and Visual bitstreams[2][3]. For the purpose of directly mapping MPEG-4
Audio/Visual bitstreams onto RTP packets, it provides specifications for
the use of RTP header fields and also specifies fragmentation rules. It
also provides specifications for MIME type registrations and the use of
SDP.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 1]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
1. Introduction
The RTP payload formats described in this Internet-Draft specify a way of
how MPEG-4 Audio and Visual streams are to be fragmented and mapped
directly onto RTP packets.
These RTP payload formats enable to carry MPEG-4 Audio/Visual streams
without using the synchronization and stream management functionality of
MPEG-4 Systems [6]. Such RTP payload format would be used within systems
where their own stream management functionality is provided and thus such
functionality in MPEG-4 Systems is not necessary. H.323 terminals are an
example of such systems. MPEG-4 Audio/Visual streams are not managed by
MPEG-4 Systems Object Descriptors but by H.245. The streams are directly
mapped onto RTP packets without using the synchronization functionality
of MPEG-4 Systems. Other examples are SIP and RTSP where attribute of the
video stream (e.g. media type, packetization format and configuration) is
specified in MIME and SDP parameters.
The semantics of RTP headers in such cases need to be clearly defined,
including the association with MPEG-4 Audio/Visual data elements. In
addition, it would be beneficial to define the fragmentation rules of RTP
packets for MPEG-4 Video streams so as to enhance error resiliency by
utilizing the error resilience tools provided inside the MPEG-4 Video
stream. These issues, however, have yet to be addressed by other RTP
payload format specifications.
1.1 MPEG-4 Visual RTP payload format
MPEG-4 Visual is a visual coding standard with many new features: high
coding efficiency; high error resiliency; multiple, arbitrary shape
object-based coding; etc. [2]. It covers a wide range of bitrate from
scores of Kbps to several Mbps. It also covers a wide variety of
networks, ranging from those guaranteed to be almost error-free to mobile
networks with high error rates.
With respect to the fragmentation rules for an MPEG-4 visual bitstream
defined in this document, since MPEG-4 Visual is used for a wide variety
of networks, it is desirable not to apply too much restriction on
fragmentation, and a fragmentation rule such as "a single video packet
shall always be mapped on a single RTP packet" may be inappropriate. On
the other hand, careless, media unaware fragmentation may cause
degradation in error resiliency and bandwidth efficiency. The
fragmentation rules described in this document are flexible but manage to
define the minimum rules for preventing meaningless fragmentation and for
utilizing the error resilience of MPEG-4 visual.
While the additional media specific RTP header defined for such video
coding tools as H.261 or MPEG-1/2 is effective in helping to recover
picture headers corrupted by packet losses, in MPEG-4 Visual there are
already error resilience functionalities for recovering corrupt headers,
and these can be used on RTP/IP networks, as well as on other networks.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 2]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
(H.223/mobile, MPEG-2/TS, etc.) That is why no extra RTP header fields
are defined in the MPEG-4 Visual RTP payload format proposed here.
1.2 MPEG-4 Audio RTP payload format
MPEG-4 Audio is a new kind of audio standard that integrates many
different types of audio coding tools. It also supports a mechanism for
representing synthesized sounds. Low-overhead MPEG-4 Audio Transport
Multiplex (LATM) manages the sequences of audio data with relatively
small overhead. In audio-only applications, then, it is desirable for
LATM-based MPEG-4 Audio bitstreams to be directly mapped onto the RTP
packets without using MPEG-4 Systems.
For MPEG-4 Audio coding tools except synthesis tools, as is true for
other audio coders, if the payload of a packet is a single audio frame,
packet loss will not impair the decodability of adjacent packets. On the
other hands, MPEG-4 Audio synthesis tools may be sensitive to error. For
example, an SA_access_unit in the payload may set a global value to a new
value, which is then references throughout the audio content to make a
macro change in the performance. In this case, an error in the payload
influences all audio data produced after the error. In order to enhance
error resiliency, the element of SA_access_unit that makes the above
macro change should be transmitted across several SA_access_unit
repeatedly. The number of repetition will be dependent on the network
condition. Therefore, the additional media specific header for recovering
errors will not be required for MPEG-4 Audio.
2. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119 [7].
3. RTP Packetization of MPEG-4 Visual bitstream
This section specifies RTP packetization rules for MPEG-4 Visual content.
An MPEG-4 Visual bitstream is mapped directly onto the RTP payload
without any addition of extra header fields or any removal of Visual
syntax elements. The Combined Configuration/Elementary stream mode is
used so that configuration information will be carried to the same RTP
port as the elementary stream. (see 6.2.1 "Start codes" of ISO/IEC 14496-
2 [2][9][4]) The configuration information MAY additionally be specified
by some out-of-band means; in H.323 terminals, H.245 codepoint
"decoderConfigurationInformation" MAY be used for this purpose; in
systems using MIME content type and SDP parameters, e.g. SIP and RTSP,
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 3]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
the optional parameter "config" MAY be used to specify the configuration
information. (see 5.1 and 5.2)
When the short video header mode is used, the RTP payload format used MAY
be that specified for H.263 in the relevant RFCs or in other relevant
standards.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 4]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC |M| PT | sequence number | RTP
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp | Header
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| contributing source (CSRC) identifiers |
| .... |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| | RTP
| MPEG-4 Visual stream (byte aligned) | Payload
| |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| :...OPTIONAL RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1 - An RTP packet for MPEG-4 Visual stream
3.1 Use of RTP header fields for MPEG-4 Visual
Payload Type (PT): Payload type is to be specifically assigned as the
MPEG-4 Visual RTP payload format. If this assignment is to be carried out
dynamically, it can be performed by such out-of-band means as H.245, SDP,
etc.
Extension (X) bit: Defined by the RTP profile used.
Sequence Number: Incremented by one for each RTP data packet sent,
starting, for security reasons, with a random initial value.
Marker (M) bit: The marker bit is set to one to indicate the last RTP
packet (or only RTP packet) of a VOP.
Timestamp: The timestamp indicates the composition time, or the
presentation time in a no-compositor decoder. A constant offset, which is
random, is added for security reasons. For a video object plane, it is
defined as vop_time_increment (in units of
1/vop_time_increment_resolution seconds) plus the cumulative number of
whole seconds specified by module_time_base and, if present, time_code of
Group_of_VideoObjectPlane() fields. In the case of interlaced video, a
VOP will consist of lines from two fields, and the timestamp will
indicate the composition time of the first field. If the RTP packet
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 5]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
contains only configuration information and/or
Group_of_VideoObjectPlane() fields, the composition time of the next VOP
in the coding order is used. If the RTP packet contains only
visual_object_sequence_end_code information, the composition time of the
immediately preceding VOP in the coding order is used.
The resolution of the timestamp is set to its default value of 90KHz,
unless specified by an out-of-band means (e.g. SDP parameter or MIME
parameter as defined in section 5).
SSRC, CC and CSRC fields are used as described in RFC 1889 [8].
3.2 Fragmentation of MPEG-4 Visual bitstream
A fragmented MPEG-4 Visual bitstream is mapped directly onto the RTP
payload without any addition of extra header fields or any removal of
Visual syntax elements. The Combined Configuration/Elementary streams
mode is used. The following rules apply for the fragmentation.
(1) Configuration information and Group_of_VideoObjectPlane() fields
SHALL be placed at the beginning of the RTP payload (just after the RTP
header) or just after the header of the syntactically upper layer
function.
(2) If one or more headers exist in the RTP payload, the RTP payload
SHALL begin with the header of the syntactically highest function.
Note: The visual_object_sequence_end_code is regarded as the lowest
function.
(3) A header SHALL NOT be split into a plurality of RTP packets.
(4) Two or more VOPs SHALL be fragmented into different RTP packets so
that one RTP packet consists of the data bytes associated with a unique
presentation time (that is indicated in the timestamp field in the RTP
packet header).
(5) A single video packet SHOULD NOT be split into a plurality of RTP
packets. The size of a video packet SHOULD be adjusted in such a way that
the resulting RTP packet is not larger than the path-MTU. A video packet
MAY be split into a plurality of RTP packets when the size of the video
packet is large.
(Rule (5) does not apply to the enhancement layer of the scalable streams
where the video packet is not supported.)
Here, header means:
- Configuration information (Visual Object Sequence Header, Visual Object
Header and Video Object Layer Header)
- visual_object_sequence_end_code
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 6]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
- The header of the entry point function for an elementary stream
(Group_of_VideoObjectPlane() or the header of VideoObjectPlane(),
video_plane_with_short_header(), MeshObject() or FaceObject())
- The video packet header (video_packet_header() excluding
next_resync_marker())
- The header of gob_layer()
See 6.2.1 "Start codes" of ISO/IEC 14496-2[2][9][4] for the definition of
the configuration information and the entry point functions.
The video packet starts with the VOP header or the video packet header,
followed by motion_shape_texture(), and ends with next_resync_marker() or
next_start_code().
3.3 Examples of packetized MPEG-4 Visual bitstream
Considering the fact that MPEG-4 Visual covers a wide variety of networks
ranging from scores of Kbps to several Mbps, and from those guaranteed to
be almost error-free to mobile networks with high error rates, it is
desirable not to apply too much restriction on fragmentation. On the
other hand, careless, media unaware fragmentation will cause degradation
in error resiliency and bandwidth efficiency. The fragmentation criteria
described in 3.2 are flexible but to define the minimum rules to prevent
meaningless fragmentation.
Figure 2 shows examples of RTP packets generated based on the criteria
described in 3.2
(a) is an example of the first RTP packet or the random access point of
an MPEG-4 visual bitstream containing the configuration information.
According to criterion (1), the Visual Object Sequence Header(VS header)
is placed at the beginning of the RTP payload, preceding the Visual
Object Header and the Video Object Layer Header(VO header, VOL header).
Since the fragmentation rule defined in 3.2 guarantees that the
configuration information, starting with
visual_object_sequence_start_code, is always placed at the beginning of
the RTP payload, RTP receivers can detect the random access point by
checking if the first 32-bit field of the RTP payload is
visual_object_sequence_start_code.
(b) is another example of the RTP packet containing the configuration
information. It differs from example (a) in that the RTP packet also
contains a video packet in the VOP following the configuration
information. Since the length of the configuration information is
relatively short (typically scores of bytes) and an RTP packet containing
only the configuration information may thus increase the overhead, the
configuration information and the immediately following GOV and/or (a
part of) VOP can be effectively packetized into a single RTP packet as in
this example.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 7]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
(c) is an example of the RTP packet that contains
Group_of_VideoObjectPlane(GOV). Following criterion (1), the GOV is
placed at the beginning of the RTP payload. It would be a waste of RTP/IP
header overhead to generate an RTP packet containing only a GOV whose
length is 7 bytes. Therefore, (a part of) the following VOP can be placed
in the same RTP packet as shown in (c).
(d) is an example of the case where one video packet is packetized into
one RTP packet. When the packet-loss rate of the underlying network is
high, this kind of packetization is recommended. It is recommended to set
resync_marker_disable to 0 in the VOL header to enable the adjustment of
the video packet size. Even when the RTP packet containing the VOP header
is discarded by a packet loss, the other RTP packets can be decoded by
using the HEC(Header Extension Code) information in the video packet
header. No extra RTP header field is necessary.
(e) is an example of the case where more than one video packets are
packetized into one RTP packet. This kind of packetization is effective
to save the overhead of RTP/IP headers when the bit-rate of the
underlying network is low. However, it will decrease the packet-loss
resiliency because multiple video packets are discarded by a single RTP
packet loss. The optimal number of video packets in an RTP packet and the
length of the RTP packet can be determined considering the packet-loss
rate and the bit-rate of the underlying network.
Figure 3 shows examples of RTP packets prohibited by the criteria of 3.2.
Fragmentation of a header into multiple RTP packets, as in (a), will not
only increase the overhead of RTP/IP headers but also decrease the error
resiliency. Therefore, it is prohibited by the criterion (3).
When concatenating more than one video packets into an RTP packet, VOP
header or video_packet_header() shall not be placed in the middle of the
RTP payload. The packetization as in (b) is not allowed by criterion (2)
due to the aspect of the error resiliency. Comparing this example with
Figure 2(d), although two video packets are mapped onto two RTP packets
in both cases, the packet-loss resiliency is not identical. Namely, if
the second RTP packet is lost, both video packets 1 and 2 are lost in the
case of Figure 3(b) whereas only video packet 2 is lost in the case of
Figure 2(d).
An RTP packet containing more than one VOPs, as in (c), is not allowed.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 8]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
+------+------+------+------+
(a) | RTP | VS | VO | VOL |
|header|header|header|header|
+------+------+------+------+
+------+------+------+------+------------+
(b) | RTP | VS | VO | VOL |Video Packet|
|header|header|header|header| |
+------+------+------+------+------------+
+------+-----+------------------+
(c) | RTP | GOV |Video Object Plane|
|header| | |
+------+-----+------------------+
+------+------+------------+ +------+------+------------+
(d) | RTP | VOP |Video Packet| | RTP | VP |Video Packet|
|header|header| (1) | |header|header| (2) |
+------+------+------------+ +------+------+------------+
+------+------+------------+------+------------+------+------------+
(e) | RTP | VP |Video Packet| VP |Video Packet| VP |Video Packet|
|header|header| (1) |header| (2) |header| (3) |
+------+------+------------+------+------------+------+------------+
Figure 2 - Examples of RTP packetized MPEG-4 Visual bitstream
+------+-------------+ +------+------------+------------+
(a) | RTP |First half of| | RTP |Last half of|Video Packet|
|header| VP header | |header| VP header | |
+------+-------------+ +------+------------+------------+
+------+------+----------+ +------+---------+------+------------+
(b) | RTP | VOP |First half| | RTP |Last half| VP |Video Packet|
|header|header| of VP(1) | |header| of VP(1)|header| (2) |
+------+------+----------+ +------+---------+------+------------+
+------+------+------------------+------+------------------+
(c) | RTP | VOP |Video Object Plane| VOP |Video Object Plane|
|header|header| (1) |header| (2) |
+------+------+------------------+------+------------------+
Figure 3 - Examples of prohibited RTP packetization for MPEG-4 Visual
bitstream
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 9]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
4. RTP Packetization of MPEG-4 Audio bitstream
This section specifies RTP packetization rules for MPEG-4 Audio
bitstreams. MPEG-4 Audio streams are formatted by LATM (Low-overhead
MPEG-4 Audio Transport Multiplex) tool[5], and the LATM-based streams are
then mapped onto RTP packets as described the three sections below.
4.1 RTP Packet Format
LATM-based streams consist of a sequence of audioMuxElements that include
one or more audio frames. A complete audioMuxElement or a part of one
SHALL be mapped directly onto an RTP payload without any removal of
audioMuxElement syntax elements (see Figure 4). The first byte of each
audioMuxElement SHALL be located at the first payload location in an RTP
packet.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC |M| PT | sequence number |RTP
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp |Header
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| contributing source (CSRC) identifiers |
| .... |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| |RTP
: audioMuxElement (byte aligned) :Payload
| |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| :...OPTIONAL RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4 - An RTP packet for MPEG-4 Audio
In order to decode the audioMuxElement, the following muxConfigPresent
information is required to be indicated by an out-of-band means.
muxConfigPresent: If this value is set to 1, the audioMuxElement SHALL
include an indication bit "useSameStreamMux" and MAY include the
configuration information for audio compression "StreamMuxConfig". The
useSameStreamMux bit indicates whether the StreamMuxConfig element in the
previous frame is applied in the current frame.
4.2 Use of RTP Header Fields for MPEG-4 Audio
Payload Type (PT): Payload type is to be specifically assigned as the
MPEG-4 Audio RTP payload format. If this assignment is to be carried out
dynamically, it can be performed by such out-of-band means as H.245, SDP,
etc.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 10]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
Marker (M) bit: The marker bit indicates audioMuxElement boundaries. It
is set to one to indicate that the RTP packet contains a complete
audioMuxElement or the last fragment of an audioMuxElement.
Timestamp: The timestamp indicates composition time, or presentation time
in a no-compositor decoder. Timestamps are recommended to start at a
random value for security reasons.
Unless specified by an out-of-band means, the resolution of the timestamp
is set to its default value of 90 kHz.
Sequence Number: Incremented by one for each RTP packet sent, starting,
for security reasons, with a random value.
SSRC, CC and CSRC fields are used as described in RFC 1889 [8].
4.3 Fragmentation of MPEG-4 Audio bitstream
It is desirable to put one audioMuxElement in each RTP packet. If the
size of an audioMuxElement can be kept small enough that the size of the
RTP packet containing it does not exceed the size of the path-MTU, this
will be no problem. If it cannot, the audioMuxElement MAY be fragmented
and spread across multiple packets, following the rules below:
(1) "payloadMux", which consists of payload elements, MAY be fragmented
across several RTP packets, so that each of those RTP packets will
contain one or more payload elements. Individual payload elements
themselves SHOULD NOT be fragmented.
(2) If the audioMuxElement includes StreamMuxConfig, StreamMuxConfig
SHALL be included in the RTP packet that contains the first payload
element.
5. MIME type registration for MPEG-4 Audio/Visual streams
The following sections describe the MIME type registrations for MPEG-4
Audio/Visual streams. MIME type registration and SDP usage for the MPEG-4
Visual stream are described in Sections 5.1 and 5.2, respectively, while
MIME type registration and SDP usage for MPEG-4 Audio stream are
described in Sections 5.3 and 5.4, respectively.
(In the following sections, the RFC number "XXXX" represents the RFC
number, which should be assigned for this Internet Draft.)
5.1 MIME type registration for MPEG-4 Visual
MIME media type name: video
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 11]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
MIME subtype name: MP4V
Required parameters: none
Optional parameters:
rate: This parameter is used only for RTP transport. It indicates the
resolution of the timestamp field in the RTP header. If this parameter
is not specified, its default value of 90000 (90KHz) is used.
profile-level-id: A decimal representation of MPEG-4 Visual Profile
Level indication value (profile_and_level_indication) defined in Table
G-1 of ISO/IEC 14496-2 [2][4].
config: A hexadecimal representation of an octet string that expresses
the MPEG-4 Visual configuration information, as defined in subclause
6.2.1 Start codes of ISO/IEC14496-2[2][4][9]. The configuration
information is mapped onto the octet string in an MSB-first basis. The
first bit of the configuration information SHALL be located at the MSB
of the first octet. The configuration information indicated by this
parameter SHALL be the same as the configuration information in the
corresponding MPEG-4 Visual stream, except for
first_half_vbv_occupancy and latter_half_vbv_occupancy, if exist,
which may vary in the repeated configuration information inside an
MPEG-4 Visual stream (See 6.2.1 Start codes of ISO/IEC14496-2).
The parameter "profile-level-id" MAY be used in the capability
exchange procedure to indicate MPEG-4 Visual Profile and Level
combination of which the MPEG-4 Visual codec is capable. The parameter
"config" MAY be used to indicate the configuration of the
corresponding MPEG-4 visual bitstream, but SHALL NOT be used to
indicate the codec capability in the capability exchange procedure.
Example usages for these parameters are:
- MPEG-4 Visual Simple Profile/Level 1:
Content-type: video/mp4v; profile-level-id=1
- MPEG-4 Visual Core Profile/Level 2:
Content-type: video/mp4v; profile-level-id=34
- MPEG-4 Visual Advanced Real Time Simple Profile/Level 1:
Content-type: video/mp4v; profile-level-id=145
Published specification:
The specifications for MPEG-4 Visual streams are presented in ISO/IEC
14469-2[2][4][9]. The RTP payload format is described in RFCXXXX.
Encoding considerations:
Video bitstreams must be generated according to MPEG-4 Visual
specifications (ISO/IEC 14496-2). A video bitstream is binary data and
must be encoded for non-binary transport (for Email, the Base64
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 12]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
encoding is sufficient). This type is also defined for transfer via
RTP. The RTP packets MUST be packetized according to the MPEG-4 Visual
RTP payload format defined in RFCXXXX.
Security considerations:
See section 6 of RFCXXXX.
Interoperability considerations:
MPEG-4 Visual provides a large and rich set of tools for the coding of
visual objects. For effective implementation of the standard, subsets
of the MPEG-4 Visual tool sets have been provided for use in specific
applications. These subsets, called 'Profiles', limit the size of the
tool set a decoder is required to implement. In order to restrict
computational complexity, one or more Levels are set for each Profiles.
A Profile@Level combination allows:
o a codec builder to implement only the subset of the standard he
needs, while maintaining interworking with other MPEG-4 devices
included in the same combination, and
o checking whether MPEG-4 devices comply with the standard
('conformance testing').
The visual stream SHALL be compliant with the MPEG-4 Visual
Profile@Level specified by the parameter "profile-level-id".
Interoperability between a sender and a receiver may be achieved by
specifying the parameter "profile-level-id" in MIME content, or by
arranging in the capability exchange procedure to set this parameter
mutually to the same value.
Applications which use this media type:
Audio and visual streaming and conferencing tools, Internet messaging
and Email applications.
Additional information: none
Person & email address to contact for further information:
The authors of RFCXXXX. (See section 8)
Intended usage: COMMON
Author/Change controller:
The authors of RFCXXXX. (See section 8)
5.2 SDP usage of MPEG-4 Visual
The MIME media type video/MP4V string is mapped to fields in the Session
Description Protocol (SDP), RFC 2327, as follows:
o The MIME type (video) goes in SDP "m=" as the media name.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 13]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
o The MIME subtype (MP4V) goes in SDP "a=rtpmap" as the encoding name.
o The optional parameter "rate" goes in "a=rtpmap" as the clock rate.
o The optional parameter "profile-level-id" and "config" MAY go in the
"a=fmtp" line to indicate the coder capability and configuration,
respectively. These parameters are expressed as a MIME media type string,
in the form of as a semicolon separated list of parameter=value pairs.
The following are some examples of media representation in SDP:
Simple Profile/Level 1, rate=90000(90KHz), "profile-level-id" and
"config" are present in "a=fmtp" line:
m=video 49170/2 RTP/AVP 98
a=rtpmap:98 MP4V/90000
a=fmtp:98 profile-level-id=1;
config=000001B001000001B5090000010000000120008440FA282C2090A21F
Core Profile/Level 2, rate=90000(90KHz), "profile-level-id" is present in
"a=fmtp" line:
m=video 49170/2 RTP/AVP 98
a=rtpmap:98 MP4V/90000
a=fmtp:98 profile-level-id=34
Advance Real Time Simple Profile/Level 1, rate=25(25Hz), "profile-level-
id" is present in "a=fmtp" line:
m=video 49170/2 RTP/AVP 98
a=rtpmap:98 MP4V/25
a=fmtp:98 profile-level-id=145
5.3 MIME type registration of MPEG-4 Audio
MIME media type name: audio
MIME subtype name: MP4A
Required parameters:
rate: the rate parameter indicates the RTP time stamp clock rate. The
default value is 90000. Other rates CAN be specified only if they are
set to the same value as the audio sampling rate (number of samples
per second).
Optional parameters:
profile-level-id: a decimal representation of MPEG-4 Audio Profile
Level indication value defined in ISO/IEC 14496-1 [11]. This parameter
indicates which MPEG-4 Audio tool subsets the decoder is capable of
using.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 14]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
object: a decimal representation of the MPEG-4 Audio Object Type value
defined in ISO/IEC 14496-3 [5]. This parameter specifies the tool to
be used by the coder. It CAN be used to limit the capability within
the specified "profile-level-id".
bitrate: the data rate for the audio bit stream.
cpresent: this parameter indicates whether audio payload configuration
data has been multiplexed into an RTP payload (See section 4.1 in this
document).
config: a hexadecimal representation of an octet string that expresses
the audio payload configuration data "StreamMuxConfig", as defined in
ISO/IEC 14496-3 [5]. Configuration data is mapped onto the octet
string in an MSB-first basis. The first bit of the configuration data
SHALL be located at the MSB of the first octet. In the last octet,
zero-padding bits, if necessary, shall follow the configuration data.
If the size of the configuration data is quite large, such large
config data is RECOMMENDED to be indicated by in-band mode (cpresent
is set to 1).
ptime: RECOMMENDED duration of each packet in milliseconds.
Published specification:
Payload format specifications are described in this document. Encoding
specifications are provided in ISO/IEC 14496-3 [3][5].
Encoding considerations:
This type is only defined for transfer via RTP.
Security considerations:
See Section 6 of RFCXXXX.
Interoperability considerations:
MPEG-4 Audio provides a large and rich set of tools for the coding of
audio objects. For effective implementation of the standard, subsets of
the MPEG-4 Audio tool sets similar to those used in MPEG-4 Visual have
been provided (see section 5.1).
The audio stream SHALL be compliant with the MPEG-4 Audio
Profile@Level specified by the parameter "profile-level-id".
Interoperability between a sender and a receiver may be achieved by
specifying the parameter "profile-level-id" in MIME content, or by
arranging in the capability exchange procedure to set this parameter
mutually to the same value. Furthermore, the "object" parameter can be
used to limit the capability within the specified Profile@Level in
capability exchange.
Applications which use this media type:
Audio and video streaming and conferencing tools.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 15]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
Additional information: none
Personal & email address to contact for further information:
See Section 8 of RFCXXXX.
Intended usage: COMMON
Author/Change controller:
See Section 8 of RFCXXXX.
5.4 SDP usage of MPEG-4 Audio
The MIME media type audio/MP4A string is mapped to fields in the Session
Description Protocol (SDP), RFC 2327, as follows:
o The MIME type (audio) goes in SDP "m=" as the media name.
o The MIME subtype (MP4A) goes in SDP "a=rtpmap" as the encoding name.
o The required parameter "rate" goes in "a=rtpmap" as the clock rate.
o The optional parameter "ptime" goes in SDP "a=ptime" attribute.
o The optional parameter "profile-level-id" goes in the "a=fmtp" line to
indicate the coder capability. The "object" parameter goes in the
"a=fmtp" attribute. The payload-format-specific parameters "bitrate",
"cpresent" and "config" go in the "a=fmtp" line. If the string after
"config=" is quite large, such large config data should not be
transmitted by SDP but should be transmitted by in-band mode. These
parameters are expressed as a MIME media type string, in the form of as a
semicolon separated list of parameter=value pairs.
The following are some examples of the media representation in SDP:
For 6 kb/s CELP bitstreams (with an audio sampling rate of 8 kHz),
m=audio 49230 RTP/AVP 96
a=rtpmap:96 MP4A/8000
a=fmtp:96 profile-level-id=9;object=8;cpresent=0;config=9128B1071070
a=ptime:20
For 64 kb/s AAC LC stereo bitstreams (with an audio sampling rate of 24
kHz),
m=audio 49230 RTP/AVP 96
a=rtpmap:96 MP4A/24000
a=fmtp:96 profile-level-id=1; bitrate=64000; cpresent=0;
config=9122620000
In the above two examples, audio configuration data is not multiplexed
into the RTP payload and is described only in SDP. Furthermore, the
"clock rate" is set to the audio sampling rate.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 16]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
If the clock rate has been set to its default value and it is necessary
to obtain the audio sampling rate, this can be done by parsing the
"config" parameter (see the following example).
m=audio 49230 RTP/AVP 96
a=rtpmap:96 MP4A/90000
a=fmtp:96 object=8; cpresent=0; config=9128B1071070
The following example shows that the audio configuration data appears in
the RTP payload.
m=audio 49230 RTP/AVP 96
a=rtpmap:96 MP4A/90000
a=fmtp:96 object=13; cpresent=1
6. Security Considerations
RTP packets using the payload format defined in this specification are
subject to the security considerations discussed in the RTP specification
[8]. This implies that confidentiality of the media streams is achieved
by encryption. Because the data compression used with this payload format
is applied end-to-end, encryption may be performed on the compressed data
so there is no conflict between the two operations.
This payload type does not exhibit any significant non-uniformity in the
receiver side computational complexity for packet processing to cause a
potential denial-of-service threat.
7. References
1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9,
RFC 2026, October 1996.
2 ISO/IEC 14496-2:1999, "Information technology - Coding of audio-visual
objects - Part2: Visual", December 1999.
3 ISO/IEC 14496-3:1999, "Information technology - Coding of audio-visual
objects - Part3: Audio", December 1999.
4 ISO/IEC 14496-2:1999/FDAM1:2000, December 1999.
5 ISO/IEC 14496-3:1999/FDAM1:2000, December 1999.
6 ISO/IEC 14496-1:1999, "Information technology - Coding of audio-visual
objects - Part1: Systems", December 1999.
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 17]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
7 Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997
8 H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson "RTP: A Transport
Protocol for Real Time Applications", RFC 1889, Internet Engineering
Task Force, January 1996.
9 ISO/IEC 14496-2/COR1, "Information technology - Coding of audio-visual
objects - Part2: Visual, Technical corrigendum 1", March 2000.
8. Author's Addresses
Yoshihiro Kikuchi
Toshiba corporation
1, Komukai Toshiba-cho, Saiwai-ku, Kawasaki, 212-8582, Japan
Email: yoshihiro.kikuchi@toshiba.co.jp
Yoshinori Matsui
Matsushita Electric Industrial Co., LTD.
1006, Kadoma, Kadoma-shi, Osaka, Japan
Email: matsui@drl.mei.co.jp
Toshiyuki Nomura
NEC Corporation
4-1-1,Miyazaki,Miyamae-ku,Kawasaki,JAPAN
Email: t-nomura@ccm.cl.nec.co.jp
Shigeru Fukunaga
Oki Electric Industry Co., Ltd.
1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan.
Email: fukunaga444@oki.co.jp
Hideaki Kimata
Nippon Telegraph and Telephone Corporation
1-1, Hikari-no-oka, Yokosuka-shi, Kanagawa, Japan
Email: kimata@nttvdt.hil.ntt.co.jp
Kikuchi/Nomura/Fukunaga/Kimata/Matsui [Page 18]
RTP payload format for MPEG-4 Audio/Visual streams February 2000
Full Copyright Statement
"Copyright (C) The Internet Society (date). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
