Internet Engineering Task Force Audio Visual Transport WG
Internet-Draft J.Van der Meer/
C.Roux,E.Gouleau,
D.Curet,S.Relier/
P.Clement/
G.Cherry
Philips/FT R&D /Thomcast/
nCube
February, 2001
Expires: August, 2001
RTP Payload Format for MPEG-4 FlexMultiplexed Streams
draft-curet-avt-rtp-mpeg4-flexmux-00.txt
1. Status of this Memo
This document is an Internet-Draft and is in full conformance
with all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet
Engineering Task Force (IETF), its areas, and its working
groups. Note that other groups may also distribute working
documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-
Drafts as reference material or to cite them other than as
'work in progress.'
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed
at http://www.ietf.org/shadow.html.
2. Abstract
This document describes a payload format for transporting
MPEG-4 encoded and multiplexed data using RTP. MPEG-4 is a
recent standard from ISO/IEC for the coding of natural and
synthetic audio-visual data.
Several services provided by RTP are beneficial for MPEG-4
encoded and multiplexed data transport over the Internet.
Additionally, the use of RTP makes it possible to synchronize
MPEG-4 data with other real-time data types.
This specification is a product of the Audio/Video Transport
C.Roux & al. Page 1 23/02/01
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
working group within the Internet Engineering Task Force and
ISO/IEC MPEG-4 ad hoc group on MPEG-4 over Internet. Comments
are solicited and should be addressed to the working group's
mailing list at rem-conf@es.net and/or the authors.
RTP PAYLOAD FORMAT FOR MPEG-4 FLEXMULTIPLEXED STREAMS 1
1. STATUS OF THIS MEMO 1
2. ABSTRACT 1
3. INTRODUCTION TO THE MPEG-4 STANDARD 3
4. OVERVIEW OF MPEG-4 END-SYSTEM ARCHITECTURE 4
5. BENEFITS OF USING RTP FOR TRANSPORT:5
6. CONVENTIONS USED IN THIS DOCUMENT 5
6.1 GENERAL: 5
6.2 MPEG-4 GLOSSARY: 5
7. THE RTP PACKET 6
7.1 THE RTP PACKET HEADER 6
7.2 RTP HEADER FIELDS USAGE STREAMS: 6
7.3 THE TWO RTP PACKET PAYLOADS 7
7.4 MPEG-4 FLEXMUX SIGNALING DESCRIPTORS 8
8. FLEX MULTIPLEXING 12
8.1 SOME OF THE ADVANTAGES : 12
8.2 DISADVANTAGES: 13
8.3. TRANSPORT OF MPEG-4 FLEXMUX STREAMS 13
9. SDP SYNTAX 15
9.1 ATTRIBUTES15
9.2 MIME TYPES16
C.Roux & al. expires 14/08/01 2
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
10. RTSP USAGE: 16
11. SECURITY CONSIDERATIONS 17
12. REFERENCES17
13. AUTHORS' ADDRESSES 18
3. Introduction to the MPEG-4 standard
The MPEG-4 standard (ISO/IEC 14496) can be represented in a
layered architecture, where three layers can be identified as
follows:
+---------------------------------------+
media aware | COMPRESSION LAYER: |
and | Elementary Streams (ES) encoding |
delivery unaware| MPEG-4 part 2 Visual |
layer | MPEG-4 part 3 Audio |
| MPEG-4 part 1 Bifs,OD,IPMP,OCI |
+---------------------------------------+
================================================ ESI Interface
+---------------------------------------+
media and | SYNC LAYER (SL) |
delivery unaware| Elementary streams management |
layer | and synchronisation |
+---------------------------------------+
================================================DAI Interface
+---------------------------------------+
delivery aware | DELIVERY LAYER (DMIF) |
media unaware |provides Flexmultiplexing of SL streams|
layer | and transparent access |
| to the delivery technology |
+---------------------------------------+
Although the Delivery Layer mostly focuses on the control plane
it also encompasses multiplexing tools and defines a bitstream
syntax, called the Flexmux, to multiplex MPEG-4 SL streams. The
reconstruction of the correct timing of MPEG-4 bitstreams is
supported both by the MPEG-4 SL stream syntax and by the MPEG-4
FlexMux stream syntax. The reconstruction of the correct timing
C.Roux & al. expires 14/08/01 3
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
of an MPEG-4 Flexmux stream is possible under various QoS
constraints related to the reduction of network jitter. MPEG
makes the assumption that all Flexmux packets transmitted over
the network should have a nearly constant transmission delay.
The reconstruction of the correct timing of the MPEG-4 Flexmux
streams is based on the fact that this assumption can be
verified, in accordance with the required accuracy of the
reconstructed timing of the MPEG-4 FlexMux stream. This
document will specify a RTP payload format to enable the
carriage of Flexmux streams.
4. Overview of MPEG-4 End-System Architecture
The Compression Layer processes the traditional individual
audio/visual elementary streams (ES) and some associated
'systems' elementary streams (ES) such as Bifs, OD, IPMP and
OCI elementary streams.
The MPEG-4 audio/visual ES syntaxes are defined in[2] and[3].
The ôsystemsö ES syntaxes are described in [1]: the Bifs ES
syntax allows a dynamic scene description. The OD ES syntax
allows the description of the hierarchical relations, location
and properties of different ESs through a dynamic set of Object
Descriptors. The ôsystemö ES may require to be carried with a
better protection than the traditional audio/visual ESs.
The compressed content produced by this layer are organised
into Elementary Streams (ESs). The compression layer is unaware
of a specific delivery technology, but it can react to the
characteristics of a particular delivery layer such as the
path-MTU or packet loss or bit error characteristics.
The MPEG-4 SL stream syntax is defined in [1]. It provides a
unique and homogeneous encapsulation of any of the MPEG-4 ESs.
That layer primarily provides the synchronisation between ESs.
ESs are organised in Access Units (AU). An AU is the smallest
element to which timestamps can be assigned. Integer or
fractional AUs are then encapsulated in SL packets.
The MPEG-4 Delivery Layer consists of the Delivery Multimedia
Integration Framework defined in [4]. This layer is media
unaware but delivery technology aware. It provides transparent
access to and delivery of content irrespective of the
technologies used. The DAI interface between the SL layer and
the DMIF layer is called the DMIF Application Interface. This
interface supports content location independent protocols
firstly for establishing the MPEG-4 session and secondly for
accessing to transport channels. DMIF monitors transport
channels on the QoS requirements assigned to the SL streams,
and supports the multiplexing of the SL streams, by the means
C.Roux & al. expires 14/08/01 4
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
of the MPEG-4 FlexMux tools. There are two possible FlexMux
tools. FlexMux streams delivery is defined in [4], FlexMux
tools are defined within [1].
MPEG makes the assumption that the carriage of MPEG-4 Flexmux
streams over the network should affect packets with an ôidealö
constant transmission delay; the reconstruction of the correct
timing of a MPEG-4 Flexmux streams is based on that assumption.
This draft specifies an RTP [5] payload format for transporting
multiplexed MPEG-4 encoded data streams. It can be presented as
an instance of the MPEG-4 Delivery layer.
5. Benefits of using RTP for transport:
i. Ability to synchronize MPEG-4 streams with other RTP
payloads
ii. Monitoring MPEG-4 delivery performance through RTCP
iii. Combining MPEG-4 and other real-time data streams
received from multiple end-systems into a set of
consolidated streams through RTP mixers
iv. Converting data types, etc. through the use of RTP
translators
6. Conventions used in this document
6.1 general:
The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL', 'SHALL
NOT','SHOULD', 'SHOULD NOT', 'RECOMMENDED', 'MAY', and
'OPTIONAL' in this document are to be interpreted as described
in RFC-2119 [6].
6.2 MPEG-4 glossary:
AU :Access Unit Bifs: Binary format for scene
DMIF: Delivery Multimedia Integration Framework,
DAI: DMIF Application Interface, ES: Elementary stream,
ESI: Elementary stream Interface, FlexMux: Flexible Multiplex.
IPMP: Intellectual Property Management and Protection,
OCI: Object Content Information, OD: Object descriptor,
SL: Synchronization layer
C.Roux & al. expires 14/08/01 5
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
7. The RTP packet
7.1 The RTP packet header
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC |M| PT | sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
: contributing source (CSRC) identifiers |
|=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| |
| |
| RTP Packet Payload |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1 - An RTP packet for MPEG-4 FlexMux stream
7.2 RTP header fields usage streams:
Payload Type (PT): The assignment of a particular RTP payload
type to this new packet format, is outside the scope of this
document, and is not specified here. If the dynamic payload
type assignment is used, it can be specified by some out of
band means (e.g. SDP, according to the syntax proposed in the
paragraph 9) that the MPEG-4 FlexMux payload format
is used for the corresponding RTP packets. The specification
of the usage of MPEG-4 FlexMux payload format can also include,
if needed, the specification of the usage of the MPEG-4 FlexMux
signaling format.
Marker (M) bit: set to zero.
Extension (X) bit: Defined by the RTP profile used.
Sequence Number:Increment by one for each RTP data packet sent.
It starts with a random initial value for security reasons.
Timestamp: it represents the target transmission time for the
C.Roux & al. expires 14/08/01 6
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
first byte of the RTP packet. Unless specified by an out of
band means (e.g. SDP), the resolution of the timestamp is set
to its default (90KHz).
SSRC, CC and CSRC fields are used as described in RFC 1889 [5].
7.3 The two RTP packet payloads
The MPEG-4 FlexMux payload format encompasses two payload
types:
the first one for MPEG-4 FlexMux packets and the second one for
MPEG-4 FlexMux signaling.
When the Payload type specifies the usage of the MPEG-4
FlexMux payload format, the RTP packet payload is built from
an integer number of complete, defined in [1].
When the Payload type specifies the usage of the MPEG-4
FlexMux signaling payload format, the RTP packet payload is
built from an integer number of descriptors, defined in the
following 7.4. paragraph.
7.3.1 payload for the MPEG-4 FlexMux payload format
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FlexMux Packet Header | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| FlexMux Packet Payload (byte aligned) |
|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FlexMux Packet Header | FlexMux |
|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| Packet |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload :...optional RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3 - An RTP Payload carrying MPEG-4 FlexMux packets
C.Roux & al. expires 14/08/01 7
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
7.3.2 payload for the MPEG-4 FlexMux signaling payload format
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MPEG-4 FlexMux signaling descriptor |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
| | MPEG-4 FlexMux signaling descriptor |
|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| signaling descriptor |...optional RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4 - An RTP Payload carrying MPEG-4 FlexMux signaling
descriptors
7.4 MPEG-4 FlexMux signaling descriptors
7.4.1 signaling descriptor usage:
They are used to describe in-band a FlexMux stream
characteristics. Their use is mostly static . Only one of these
descriptors (the FlexMux codetable entry descriptor) may have a
dynamic use, when the FlexMux stream characteristic is to be
dynamic.
They are built from a signaling descriptor header followed by a
signaling descriptor content.
The signaling descriptor header structure is identical for all
the signaling descriptors.
Their overall length is an integer number of bytes.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| HEADER | CONTENT |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5 - A signaling descriptor scheme
7.4.1 the signaling descriptor header
The signaling descriptor header is built from a tag field on
one byte, followed by a length field on one byte
C.Roux & al. expires 14/08/01 8
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TAG | LENGTH |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6 - A signaling descriptor header
TAG gives the type of the descriptor:
0x60 for FlexMux Channel Table descriptor
0x61 for FlexMux buffersize descriptor
0x62 for FlexMux timing descriptor
0x63 for FlexMux codetable entry descriptor
0x64 for FlexMux declaration descriptor
values from 0x00 to 0x5F, & value 0xFF are forbidden
values from 0x65 to 0xBF are ISO reserved
values from 0xC0 to 0xDF are IETF reserved
values from 0xE0 to 0xFE are Ad Hoc values
LENGTH - is the byte length of the descriptor's content that
follows.
7.4.2 FlexMux declaration descriptor
This descriptor is used to allow identification of some
different FlexMux streams within an MPEG-4 scene. This can
be the case when scalable coding is used.
When no FlexMux descriptor is used, the default TYPE is the
first FlexMux tool, and the default MODE is static.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MUXID | TYPE| MODE |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7 - declaration descriptor content
MUXID - is the identifier of the FlexMux stream
TYPE - is the type of the Multiplexing tool used to generate
the FlexMux stream. AS they are two FlexMux tools defined,
indicated type values shall be either 0 (for the first FlexMux
tool), 1 (for the second FlexMux tool).
C.Roux & al. expires 14/08/01 9
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
Values from 2 to 7 are IETF reserved, values 8 to 15 are Ad Hoc
values.
MODE û is the mode of management used by the Multiplexing tool,
to generate the FlexMux stream. Indicated mode values shall be
either 0 (static), 1 (dynamic). Values from 2 to 7 are IETF
reserved, values 8 to 15 are Ad Hoc values.
7.4.3 FlexMux timing descriptor
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FCR-ES-ID | FCRRESOLUTION |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FCRRESOLUTION (end) | FCRLENGTH | FMXRATELENGTH |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
Figure 8 - timing descriptor content
FCR-ES-ID û (16 bits)is the ES_ID associated to this clock
reference stream.
FCRRESOLUTION û(32 bits) is the resolution of the object time
base in cycles per second.
FCRLENGTH -(8 bits) is the length of the fmxClockReference
field in FlexMux packets with index = 238. A length of zero
shall indicate that no FlexMux packets with index = 238 are
present in this FlexMux stream. FCRLENGTH shall take values
between zero and 64.
FMXRATELENGTH û (8 bits)is the length of the fmxRate field
in FlexMux packets with index = 238. FMXRATELENGTH shall take
values between 1 and 32.
7.4.4 FlexMux Channel Table descriptor
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ES1 | M1 | ES2(start) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ES2 (end) | M2 | ...... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ........ | ESn | Mn |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 9 - FLexMuxChannel Table descriptor content
C.Roux & al. expires 14/08/01 10
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
<ES1>,..,<ESn> - these 16-bit fields specify the identifiers of
ISO/IEC 14496-1 SL-packetized streams defined in [1].
<M1>,..,<M2>,à This 8-bit fields specify the number of the
FlexMux channels used for these SL-packetized streams.
7.4.5 FlexMux codetable entry descriptor
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|MUXCODE|VERSION| SUBSTRUCTCNT | SLOTCNT |RCNT | M1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| NB1 | M2 | NB2 |.............. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ..............| SlotCNT |RCNT | Mi | NBi |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Mi+1 | NBi+1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 10 û FLexMuxcode table descriptor content
MUXCODE - the number through which this FlexMux codetable entry
is referenced
VERSION - the version of the FlexMux codetable entry.
Only the latest received version of a FlexMux codetable entry
is valid.
SUBSTRUCTCNT is the number of substructures of this FlexMux
codetable entry.
SLOTCNT - the number of slots with data from different FlexMux
channels that are described by this substructure
RCNT - indicates how often this substructure is to be repeated.
A zero indicates that this substructure is to be repeated
infinitely. Zero is only permitted in the last substructure
of a FlexMux codetable entry.
M1,..,Mi - the FlexMux channels to which the data in this slot
belongs.
NB1,..,NBi - the number of data bytes in this slot associated
to the FlexMux channel M1, Mi. This number of bytes corresponds
to one SL packet. SL packets are defined in [1].
C.Roux & al. expires 14/08/01 11
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
7.4.5 FlexMux buffersize descriptor
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DEFAULTSIZE | M1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SIZE1 | M2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SIZE2 | M3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SIZE3 |.............. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ............................................................. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 11 - FlexMux buffersize descriptor content
DEFAULTSIZE - the default size of multiplex buffers for each
individual channel in a FlexMux stream. FlexMux channels that
use a different buffer size may signal this using the following
Mi,SIZEi assignments
M1,M2,M3 - the FlexMux channels
SIZE1, SIZE2, SIZE3 - the exact sizes of FlexMux buffers, for
FlexMux channels M1,M2 and M3 of this FlexMux stream. Sizes are
expressed in bytes.
8. Flex Multiplexing
8.1 Some of the advantages :
1. Since a typical MPEG-4 session may involve a large number of
objects, that may be as many as a few hundred, transporting
each ES as an individual RTP session may not always be
practical. The use of one session per elementary stream cannot
be much cost effective, both on the server side and on the
client side in terms of performance, when the number of
elementary treams will increase within a scene.
2. The use of one single session for a multiplexed bitstream
enables to send a bunch of ESs that are tightly synchronized
together. Some of these ESs can themselves be Bifs and OD
ESs when a scene description is used with Audio-Visual ES,
and some other ESs can be OCI ES, and even IPMP ES when such
systems are involved.
C.Roux & al. expires 14/08/01 12
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
3. The FlexMultiplexing management supports embedding multiple
SL packets into one FlexMux packet, by the use of FlexMux
codetable entries.
4. If the multiplexing policy used is smoothing at most the
multiplexed SL streams, mutual synchronization between these
SL streams can be easily preserved when packet losses occur.
5. The use of the FlexMux technology enables possible
interconnection between Internet network and digital television
network, as MPEG normatively defines the use the MPEG-4 FlexMux
syntax to carry MPEG-4 over MPEG-2 transport channels[8].
6. The reconstruction of the correct timing of the FlexMux
stream is possible, if some QoS requirements are supported.
7. The overall MPEG-4 receiver buffer size is reduced, as
MPEG-4 compliant Flexmultiplexed streams, by the use of the
MPEG-4 timestamps, respect the MPEG-4 system decoder model.
8. The overall bandwidth management is easier. The FlexMux
syntax allows having piecewise constant bitrate FlexMux
bitstream, with an inband signaling mechanism.
9. Protection can be enhanced by means of repetitions of
vital SL packets.
10. Content providers are able to bundle together a single
stream with assurance that associated streams will be kept
together and synchronized.
8.2 Disadvantages:
The major disadvantage with the packetization of the MPEG-4
Flexmultiplexed streams is the added packet header overhead.
MPEG-4 does not support a reduction mechanism of the carried
MPEG-4 Flexmultiplexed streams packet headers. This issue needs
certainly be resolved using a mechanism similar to what was
proposed with [7]. During their transport, FlexMux packets
need not be compliant to the MPEG-4 standard, it is only when
they are delivered to the Flexdemultiplexer that the MPEG-4
compliance point is defined.
8.3. Transport of MPEG-4 FlexMux streams
C.Roux & al. expires 14/08/01 13
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
An MPEG-4 FlexMux stream is mapped directly onto the RTP
payload without any addition of extra header fields or removal
of any FlexMux packet header syntactic elements.
Each RTP packet will contain a timestamp derived from the
sender's clock reference. This clock is synchronized to the
FlexMux Clock Reference (FCR) and represents the target
transmission time of the first byte of the RTP packet payload.
On the receiving side, the RTP packet timestamp will not be
passed to the MPEG-4 Flexdemultiplexor.
This use of the timestamp is slightly different from the normal
use in RTP, in that it is not considered to be the media
display time-stamp. The first purpose of this RTP timestamp
will then be to reduce (after estimation) the network jitter,
and the relative time drift between the transmitter and the
receiver.
There are packetization restrictions due to the fact that no
synchronization pattern is part of the FlexMux packet header:
An RTP packet payload should start with the start of a FlexMux
packet. An RTP packet will contain an integer number of FlexMux
packets.
The FlexMux characteristics (declaration descriptor, timing
descriptor, Channel Table descriptor, codetable entry
descriptor, buffersize descriptor) may be provided by out of
band means (e.g. SDP), or by the inband signaling mechanism
supported by the use of the FLexMuxChannel signalling
descriptors.
The FlexMux declaration descriptor, FlexMux timing descriptor,
FlexMux Channel Table descriptor, FlexMux buffersize
descriptor, may only be used to define statically the FlexMux
stream characteristics.
The FlexMux codetable entry descriptor may change dynamically.
The IP packet marking facility may be needed. If this is the
case, as it is based on the 'degradationPriority' field
present in each FlexMux packet, all the FlexMux packets
grouped in the same RTP packet should have the
'degradationPriority' field filled with the same value.
The size of the FlexMux packets should be adjusted such that
the resulting RTP packet (embedding one or several FlexMux
packets) is not larger than the path-MTU.
Protection mechanisms for FlexMux streams within RTP packets
are outside of the scope of this specification.
C.Roux & al. expires 14/08/01 14
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
9. SDP syntax
9.1 attributes
New encoding names for the a = rtpmap attribute It is
recommended that, no matter what payload format is used,
each media stream be placed in a media section that is
appropriate. For example, a payload format which can carry
both video and audio streams may be used in sections of SDP
starting both with 'm=video' and 'm=audio'. The MIME name
for the payload format is thus registered under all applicable
branches.
a = rtpmap:<payload> <name>/<time scale>/<parameters>
<payload> is the dynamic payload number.
<name> when equal to mpeg4-flexmux indicates the encoding type
of the media, one MPEG-4 FlexMux stream. When equal to
mpeg4-flexsig indicates the encoding type of the media,
one MPEG-4 FlexMux signaling stream.
<time scale> is the time scale of the RTP time stamps.
<parameters> if used, can be a way to specialise <name>.
In order to be able to define either in the clear, or in binary
format, or to point out to some already defined characteristics
(MPEG-4 FlexMux descriptors) of a FlexMux stream, the following
attribute is defined
a = mpeg4-flexmuxinfo: <location of descriptor list>
<location of descriptor list> is a URL enclosed in double
quotes, that will supply the required flexmux list of
MPEG-4 FlexMux descriptors. If they are small, a
DATA:URL will probably suffice to carry them in-line.
If not, the URL should use a file-retrieval scheme
(e.g. HTTP). The data at the indicated URL consists of
some number of concatenated complete FlexMux
descriptors. These descriptors have an intrinsic length,
so simple concatenation suffices.
Examples:
a= mpeg4-flexmuxinfo:"http://example.com/FlexMuxdescriptor.dat"
a= mpeg4-flexmuxinfo:"http://example.com/FlexMuxdescriptor.xml"
a= mpeg4-flexmuxinfo:
"data:application/mpeg4- flexmuxinfo,7ab3742134bab347"
a= mpeg4-flexmuxinfo:"data:text/xml;<FlexMuxdescriptor.../>"
C.Roux & al. expires 14/08/01 15
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
The MPEG-4 FlexMux descriptors related to FlexMux description
are defined within the paragraph 7.4. The list of MPEG-4
descriptors cannot be empty. Ad Hoc descriptors can complete
it. The MIME name used for this data is defined below.
Other SDP attributes should, if used, carry values consistent
with those carried in MPEG-4 systems (for example, bit rate).
9.2 MIME Types
The historical approach for MPEG data is to declare it under
'video', and this approach is followed for MPEG-4. For
presentations with audio information and no visual aspect, the
'audio' top-level mime type may be used; otherwise, 'video' is
used.
When a FLexMux stream is served (e.g. over HTTP) or otherwise
must be identified by a MIME type, the type 'application/mpeg4-
flexmux' SHALL be used. These files consist of concatenated
FLexMux packets in transmission order.
In some cases, the information needed by a flexmux decoder
needs to be identified with a MIME type. In this case, the type
'application/mpeg4-flexmuxinfo' SHOULD be used.
The payload names used in an RTPMAP attribute within SDP, to
specify the mapping of payload number to its definition, also
come from the MIME namespace. Each of the RTP payload mappings
defined above has a distinct name. It is recommended that
visual streams be identified under 'video', and audio streams
be identified under 'audio', and otherwise 'application' be
used.
MIME media type name:application
MIME subtype name:mpeg4-iod, mpeg4-flexmux,
mpeg4-flexmuxinfo
Required parameters:none
Optional parameters:none
Encoding considerations:base64 generally preferred;
files are binary and should be transmitted without CR/LF
onversion, 7-bit stripping etc.
10. RTSP usage:
When RTSP is used as a session-control protocol:
RTP SHOULD be used as the transport protocol.
The initial DESCRIBE format SHOULD be SDP. If the SDP
information reveals that an IOD is needed, and the terminal
does not already have it, then a second DESCRIBE accepting an
C.Roux & al. expires 14/08/01 16
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
IOD SHOULD be performed.
Note that if an MPEG-4 FlexMux stream is closed (TEARDOWN) then
the RTSP session ID will be lost. The next (re-)opened FlexMux
stream will supply a new session ID. New DESCRIBEs may be
needed.
11. Security Considerations
RTP packets using the payload format defined in this
specification are subject to the security considerations
discussed in the RTP specification [5]. This implies that
confidentiality of the media streams is achieved by encryption.
Because the data compression used with this payload format is
applied end-to-end, encryption may be performed on the
compressed data so there is no conflict between the two
operations.
This payload type does not exhibit any significant
non-uniformity in the receiver side computational complexity
for packet processing to cause a potential denial-of-service
threat.
12. References
[1] ISO/IEC 14496-1:2000 MPEG-4 Systems October 2000
[2] ISO/IEC 14496-2:1999/Amd.1:2000 MPEG-4 Visual January 2000
[3] ISO/IEC 14496-3:1999/FDAM 1:20000 MPEG-4 Audio January 2000
[4] ISO/IEC 14496-6 FDIS Delivery Multimedia Integration
Framework,November 1998
[5] Schulzrinne, Casner, Frederick, Jacobson RTP: A
Transport Protocol for Real Time Applications RFC 1889,
Internet Engineering Task Force, January 1996.
[6] S. Bradner, Key words for use in RFCs to Indicate
Requirement Levels, RFC 2119, March 1997.
[7] Y. Kikuchi, T. Nomura, S. Fukunaga, Y. Matsui,
H. Kimata, RTP payload format for MPEG-4 Audio/Visual
streams, draft-ietf-avt-rtp-mpeg4-es-05.txt, September 2000.
[8] B. Thompson, T. Koren, D. Wing, Tunneling multiplexed
Compressed RTP ('TCRTP'), work in progress,
draft-ietf-avt-tcrtp-01.txt, July 2000.
C.Roux & al. expires 14/08/01 17
RTP payload format for MPEG-4 FlexMultiplexed streams 23/02/01
[9] D. Singer, Y Lim, A Framework for the delivery of MPEG-4
over IP-based Protocols, work in progress,
draft-singer-mpeg4-ip-01.txt,October 2000.
[10] Handley, Jacobson, SDP: Session Description Protocol,
RFC 2327, Internet Engineering Task Force, April 1998.
13. Authors' Addresses
Jan Van der Meer
Philips Digital Networks
Cederlaan 4
5600 JB Eindhoven
Netherlands
e-mail : jan.vandermeer@philips.com
C.Roux, D.Curet,
E.Gouleau, S.Relier
France Telecom
rue du Clos Courtel
35512 Cesson-Sevigne
France
e-mails: catherine.roux, dominique.curet
emmanuel.gouleau, stephanie.relier@rd.francetelecom.fr
Pierre Clement
Thomcast
40 r Bray
35510 Cesson-Sevigne
France
pierre.clement@thomcast.thomson-csf.com
Guy Cherry
nCUBE
110 Marsh Drive, Suite 200
Foster City, California 94404-1184
USA
guyc@ncube.com
C.Roux & al. expires 14/08/01 18