Audio/Video Transport Working Group H.M. Stokking
Internet Draft M.O. van Deventer
Intended status: Informational O.A. Niamut
Expires: March 2011 F.A. Walraven
R. van Brandenburg
TNO Netherlands
I. Vaishnavi
CWI Netherlands
September 24, 2010
RTCP XR Block Type for inter-destination media synchronization
draft-brandenburg-avt-rtcp-for-idms-00.txt
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on March 24, 2011.
Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
Brandenburg Expires March 24, 2011 [Page 1]
Internet-Draft RTCP for IDMS September 2010
carefully, as they describe your rights and restrictions with respect
to this document.
Abstract
This document specifies an RTCP XR Block Type for inter-destination
media synchronization (IDMS). The RTCP Block Type is used to collect
information on media play-out time of a specific RTP media stream,
and to distribute a summary of the collected information. This
information can be used to synchronize the play-out of media such as
audio, video and/or subtitles among multiple devices.
This document also specifies the associated SDP parameter. The SDP
parameter is used to signal the use of the RTCP XR block for inter-
destination media synchronization.
Typical applications for inter-destination media synchronization are
social TV, watching apart together and shared service control. Each
of these applications has two or more geographically separated users
watching a media stream on respective devices. RTCP-XR is intended as
a signaling mechanism to enable various media synchronization
algorithms amongst users. The presented RTCP-XR specification for
IDMS is independent of the used synchronization algorithm.
Table of Contents
1. Introduction.................................................3
1.1. Inter-destination Media Synchronization.................3
1.2. Applicability of RTCP to IDMS...........................3
1.3. Applicability of SDP to IDMS............................3
1.4. This document and ETSI TISPAN...........................4
2. Inter-destination media synchronization use cases............4
2.1. Watching Apart Together.................................4
2.2. Shared Service Control..................................4
3. Architecture for inter-destination media synchronization.....4
3.1. Media Synchronization Application Server (MSAS).........5
3.2. Synchronization Client (SC).............................5
3.3. Mixer...................................................5
4. RTCP XR Block Type for IDMS..................................5
5. SDP parameter for IDMS.......................................8
6. Security Considerations......................................9
7. IANA Considerations..........................................9
8. Conclusions..................................................9
9. References..................................................10
9.1. Normative References...................................10
9.2. Informative References.................................10
10. Acknowledgments............................................10
Brandenburg Expires March 24, 2011 [Page 2]
Internet-Draft RTCP for IDMS September 2010
1. Introduction
1.1. Inter-destination Media Synchronization
Inter-destination media synchronization (IDMS) refers to the play-out
of media streams at two or more geographically distributed locations
in a temporally synchronized manner. It can be applied to both
unicast and multicast media streams and can be applied to any type
and/or combination of streaming media, such as audio, video and text
(subtitles). [Boronat2009] provides an overview of technologies and
algorithms for IDMS.
IDMS requires the exchange of information on media receipt and
playout times. It may also require signaling for the initiation and
maintenance of IDMS sessions and groups.
1.2. Applicability of RTCP to IDMS
RTP and RTCP [RFC3550] are protocols that are typically used in
conjunction. RTP (Real-time Transport Protocol) provides end-to-end
network transport functions suitable for applications requiring real-
time data transport, such as audio, video or data, over multicast or
unicast network services.
The data transport is augmented by a control protocol (RTCP) to allow
monitoring of the data delivery in a manner that is scalable to large
multicast networks, and to provide minimal control and identification
functionality.
RTP receivers and - senders provide reception quality feedback by
sending out RTCP Receiver Report (RR) and Sender Report (SR) packets
[RFC3550], which may be augmented by eXtended Reports (XR) [RFC3611].
IDMS involves the collection, summarizing and distribution of RTP
packet arrival and play-out times.
RTCP is applicable to IDMS, as information on RTP packet arrival
times and play-out times can be considered reception quality feedback
information.
1.3. Applicability of SDP to IDMS
RTCP XR [RFC3611] defines the Extended Report (XR) packet type for
the RTP Control Protocol (RTCP), and defines how the use of XR
packets can be signaled by an application using the Session
Description Protocol (SDP).
Brandenburg Expires March 24, 2011 [Page 3]
Internet-Draft RTCP for IDMS September 2010
SDP signaling is used to set up and maintain a synchronization group
between Synchronization Clients (SCs).
1.4. This document and ETSI TISPAN
ETSI TISPAN [TS 183 063] has specified architecture and protocol for
IDMS using RTCP XR exchange and optional SDP signaling. This internet
draft is an excerpt of the IDMS part of that specification.
2. Inter-destination media synchronization use cases
Social TV is the combination of media content consumption by two or
more users at different devices and locations and real-time
communication between those users.
2.1. Watching Apart Together
Watching Apart Together is an example of Social TV, where two or more
users watch the same television broadcast at different devices and
locations, while communicating with each other using text, audio
and/or video.
A skew in the media play-out of the two or more users can have
adverse effects on their experience. A well-known use case here is
one friend experiencing a goal in a football match well before or
after other friend(s).Thus IDMS is required to provide play-out
synchronization.
2.2. Shared Service Control
Shared Service Control is an example of Social TV, where two or more
users experience some content-on-demand together, while sharing the
trick-play controls (play, pause, fast forward, rewind) of the
content on demand.
Similar to the previous use case, without IDMS, differences in play-
out speed and the effect of transit delay of trick-play control
signals would desynchronize content play-out.
3. Architecture for inter-destination media synchronization
The architecture for IDMS, which is based around a sync-maestro
architecture [Boronat2009], is sketched below. The SC and MSAS
entities are shown as additional functionality for the RTP receiver
and sender respectively.
It should be noted that a master-slave type of architecture is
supported by having one of the SC devices also act as an MSAS.
Brandenburg Expires March 24, 2011 [Page 4]
Internet-Draft RTCP for IDMS September 2010
+-----------------------+ +-----------------------+
| | | |
| Receiver | | Sender |
| | SR+XR | |
| +-----------------+ | <----- | +-----------------+ |
| | | | | | | |
| | Synchronization | | | | Media | |
| | Client | | | | Synchronization | |
| | (SC) | | | | Application | |
| | or | | | | Server | |
| | Mixer | | RR+XR | | (MSAS) | |
| | | | -----> | | | |
| +-----------------+ | | +-----------------+ |
| | | |
+-----------------------+ +-----------------------+
3.1. Media Synchronization Application Server (MSAS)
An MSAS collects RTP packet arrival times and play-out times from one
or more SC(s) in a synchronization group. The MSAS summarizes and
distributes this information to the SCs in the synchronization group,
e.g. by determining the SC with the most lagged play-out and using
its reported RTP packet arrival time and play-out time as a summary.
3.2. Synchronization Client (SC)
An SC reports RTP packet arrival times and play-out times of a media
stream. It can receive summaries of such information, and use that to
adjust its play-out buffer.
3.3. Mixer
A Mixer re-originates an RTP media stream. Therefore the SSRC and RTP
time stamps of an outgoing RTP media stream of a mixer are unrelated
to the SSRC and RTP time stamps of the incoming RTP stream. A Mixer
can report the correlation between SSCRs and RTP time stamps of
incoming and outgoing RTP media streams.
4. RTCP XR Block Type for IDMS
This section specifies the RTCP XR Block Type for reporting inter-
destination media synchronization information on an RTP media stream.
Its definition is based on [RFC5576]. The RTCP XR is used to report
information on receipt times and presentation times of RTP packets to
e.g. a Sender [RFC3611], a Feedback Target [RFC 3550] or a Third
Party Monitor [RFC3611]. The RTCP XR is also used to indicate
synchronization settings instructions to a receiver of the RTP media
stream.
Brandenburg Expires March 24, 2011 [Page 5]
Internet-Draft RTCP for IDMS September 2010
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P| Resrv | PT=XR=207 | length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of packet sender |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| BT=12 | SPST |Resrv|P| block length=7 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PT | Resrv |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Media Stream Correlation Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of media source |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Received NTP timestamp, most significant word |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Received NTP timestamp, least significant word |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Received RTP timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Presented NTP timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The first 64 bits form the header of the RTCP XR, as defined in [45].
The SSRC of packet sender identifies the sender of the specific RTCP
packet.
Block Type (BT): 8 bits. It identifies the block format. Its value
shall be set to 12.
Synchronization Packet Sender Type (SPST): 4 bits. This field
indentifies the role of the packet sender for this specific eXtended
Report. It can have the following values:
SPST=0 Reserved For future use.
SPST=1 The packet sender is an SC. It uses this XR to report
synchronization status information. Timestamps relate to the SC
input.
SPST=2 The packet sender is an MSAS. It uses this XR to report
synchronization settings instructions. Timestamps relate to the input
of a virtual SC, which acts as reference to which the SCs belonging
to this session are synchronized.
Brandenburg Expires March 24, 2011 [Page 6]
Internet-Draft RTCP for IDMS September 2010
SPST=3 The packet sender is a Mixer [RFC3550]. It uses this XR to
report synchronization correlation information related to its
incoming media stream. Timestamps relate to its input.
SPST=4 The packet sender is a Mixer [RFC3550]. It uses this XR to
report synchronization correlation information related to a specific
outgoing media stream of SC'. Timestamps relate to its input. (see
Note below)
SPST=5-15 Reserved For future use.
NOTE: Following the RTP/RTCP specification [RFC3611], RTP timestamps
relate to the arrival time of the first octet of an RTP packet. In
case of SPST=4 (Mixer output), there is not such an arrival time as
the media stream is re-originated at the Mixer. In this case, the
timestamp would relate to the arrival time of the equivalent octet
(representing e.g. the same video pixel or audio sample) of the
incoming media steam.
Reserved bits (Resrv): 3 bits. These bits are reserved for future use
and shall be set to 0.
Packet Presented NTP timestamp flag (P): 1 bit. Bit set to 1 if the
Packet Presented NTP timestamp contains a value, 0 if it is empty. If
this flag is set to zero, then the Packet Presented NTP timestamp
shall not be inspected.
Block Length: 16 bits. This shall be set to 7, as this RTCP Block
Type has a fixed length.
Payload Type (PT): 7 bits. This field identifies the format of the
media payload, according to [59]. The media payload is associated
with an RTP timestamp clock rate. This clock rate provides the time
base for the RTP timestamp counter.
Reserved bits (Resrv): 25 bits. These bits are reserved for future
use and shall be set to 0.
Media Stream Correlation Identifier: 32 bits. This identifier is used
to correlate synchronized media streams. The value 0 (all bits are
set "0") indicates that this field is empty. The value 2^32-1 (all
bits are set "1") is reserved for future use. If the RTCP Packet
Sender is an SC or an MSAS (SPST=1 or SPST=2), then the Media Stream
Correlation Identifier maps on the SyncGroupId. If the RTCP Packet
Sender is an Mixer (SPST=3 or SPST=4), related incoming and outgoing
media streams have the same Media Stream Correlation Identifier.
Brandenburg Expires March 24, 2011 [Page 7]
Internet-Draft RTCP for IDMS September 2010
SSRC: 32 bits. The SSRC of the media source shall be set to the value
of the SSRC identifier carried in the RTP header [44] of the RTP
packet to which the XR relates.
Packet Received NTP timestamp: 64 bits. This NTP timestamp [58] is
the arrival wall clock time of the first octet of the RTP packet to
which the XR relates. For SPST=2 it relates to a virtual SC to which
the other SCs in the synchronization group may synchronize. For
SPST=4 the SC' should calculate backwards when the content (video
frame, audio sample) associated with the first octet of the RTP
packet arrived. SCs shall be time-synchronized using e.g. NTP.
Packet Received RTP timestamp: 32 bits. This timestamp has the value
of the RTP time stamp carried in the RTP header [44] of the RTP
packet to which the XR relates.
Packet Presented NTP timestamp: 32 bits. This timestamp reflects the
NTP time when the data contained in the first octet of the associated
RTP packet is presented to the user. It consists of the least
significant 16 bits of the NTP seconds part and the most significant
16 bits of the NTP fractional second part. If this field is empty,
then it shall be set to 0 and the Packet Presented NTP timestamp flag
(P) shall be set to 0. It shall be empty for SPST=3 and SPST=4.
5. SDP parameter for IDMS
The SDP parameter sync-group is used to signal the use of the RTCP XR
block for inter-destination media synchronization. This SDP parameter
extends rtcp-xr-attrib as follows, using Augmented Backus-Naur Form
[RFC2234].
rtcp-xr-attrib = "a=" "rtcp-xr" ":" [xr-format *(SP xr-format)] CRLF
; Original definition from [RFC 3611], section 5.1
xr-format =/ grp-sync ; Extending xr-format for inter-destination
media synchronization
grp-sync = "grp-sync" [",sync-group=" SyncGroupId]
SyncGroupId = 1*DIGIT ; Numerical value from 0 till 4294967295
DIGIT = %x30-39
SyncGroupId is a 32-bit unsigned integer in network byte order and
represented in decimal. SyncGroupId identifies a group of SCs for
inter-destination media synchronization. It maps on the Media Stream
Correlation Identifier of Annex W.1 for SPST=1 and SPST=2. The value
Brandenburg Expires March 24, 2011 [Page 8]
Internet-Draft RTCP for IDMS September 2010
SyncGroupId=0 represents an empty SyncGroupId. The value 4294967295
(2^32-1) is reserved for future use.
The following is an example of the SDP attribute for inter-
destination media synchronization.
a=rtcp-xr:grp-sync,sync-group=42
6. Security Considerations
The specified RTCP XR Block Type in this document is used to collect,
summarize and distribute information on packet-receipt times and
play-out times of streaming media. The information may be used to
orchestrate the media play-out at multiple devices.
Errors in the information, either accidental or malicious, may lead
to undesired behavior. For example, if one device erroneously reports
a two-hour delayed play-out, then another device in the same
synchronization group could decide to delay its play-out by two hours
as well, in order to keep its play-out synchronized. A user would
likely interpret this two hour delay as a malfunctioning service.
Therefore, the application logic of both Synchronization Clients and
Media Synchronization Application Servers should check for
inconsistent information. Differences in play-out time of more than
e.g. ten seconds could be an indication of such inconsistent
information.
7. IANA Considerations
New block types for RTCP XR are subject to IANA registration. For
general guidelines on IANA considerations for RTCP XR, refer to
[RFC3611].
[TS 183 063] assigns the block type value 12 in the RTCP XR Block
Type Registry to "Inter-destination Media Synchronization Block". [TS
183 063] also registers the SDP [RFC4566] parameter "grp-sync" for
the "rtcp-xr" attribute in the RTCP XR SDP Parameters Registry.
8. Conclusions
This document specifies the RTCP XR and the associated SDP parameter
for inter-destination media synchronization.
Brandenburg Expires March 24, 2011 [Page 9]
Internet-Draft RTCP for IDMS September 2010
9. References
9.1. Normative References
[RFC2234] Crocker, D. and Overell, P.(Editors), "Augmented BNF for
Syntax Specifications: ABNF", RFC 2234, November 1997.
[RFC3550] Schulzrinne, H., "RTP: A Transport Protocol for Real-Time
Applications", RFC 3550, July 2003.
[RFC3611] Friedman, T. "RTP Control Protocol Extended Reports (RTCP
XR)", RFC 3611, November 2003.
[RFC5576] Lennox, J., "Source-Specific Media Attributes in the
Session Description Protocol (SDP)", RFC 5576, June 2009.
[TS 183 063] ETSI TISPAN, "IMS-based IPTV stage 3 specification",
TS 183 063 v3.4.1, June 2010.
9.2. Informative References
[Boronat2009] Boronat, F., et al, "Multimedia group and inter-stream
synchronization techniques: A comparative study", Elsevier
Information Systems 34 (2009), pp. 108-131
10. Acknowledgments
The authors thank TNO and KPN for each partially funding the work
behind this internet draft.
This document was prepared using 2-Word-v2.0.template.dot.
Brandenburg Expires March 24, 2011 [Page 10]
Internet-Draft RTCP for IDMS September 2010
Authors' Addresses
Hans M. Stokking
TNO
Brassersplein 2, Delft, the Netherlands
Phone: +31 15 28 57078
Email: hans.stokking@tno.nl
M. Oskar van Deventer
TNO
Brassersplein 2, Delft, the Netherlands
Phone: +31 15 28 57078
Email: oskar.vandeventer@tno.nl
Omar A. Niamut
TNO
Brassersplein 2, Delft, the Netherlands
Phone: +31 15 28 57078
Email: omar.niamut@tno.nl
Fabian A. Walraven
TNO
Brassersplein 2, Delft, the Netherlands
Phone: +31 15 28 57078
Email: fabian.walraven@tno.nl
Ray van Brandenburg
TNO
Brassersplein 2, Delft, the Netherlands
Phone: +31 15 28 57078
Email: ray.vanbrandenburg@tno.nl
Ishan Vaishnavi
CWI
Science Park 123, Amsterdam, the Netherlands
Phone: +31 20 592 4323
Email: i.vaishnavi@cwi.nl
Brandenburg Expires March 24, 2011 [Page 11]