Network Working Group E. Ivov
Internet-Draft SIP Communicator
Intended status: Informational E. Marocco
Expires: December 17, 2009 Telecom Italia
June 15, 2009
Delivering Conference Participant Sound Level Indicators in RTP Streams
draft-ivov-avt-slic-00
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on December 17, 2009.
Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents in effect on the date of
publication of this document (http://trustee.ietf.org/license-info).
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document.
Abstract
This document describes a mechanism for RTP-level mixers in audio
conferences to deliver information about the sound level information
Ivov & Marocco Expires December 17, 2009 [Page 1]
Internet-Draft Sound Level Indicators in Conferences June 2009
on the individual participants. Such sound level indicators are
transported in the same RTP packets as the audio data they pertain
to.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Protocol Operation . . . . . . . . . . . . . . . . . . . . . . 4
4. Header Format . . . . . . . . . . . . . . . . . . . . . . . . 6
5. Signaling Information . . . . . . . . . . . . . . . . . . . . 6
6. Security Considerations . . . . . . . . . . . . . . . . . . . 8
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8
8. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 9
9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 9
10. Appendix: An alternative approach . . . . . . . . . . . . . . 9
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11
11.1. Normative References . . . . . . . . . . . . . . . . . . 11
11.2. Informative References . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12
Ivov & Marocco Expires December 17, 2009 [Page 2]
Internet-Draft Sound Level Indicators in Conferences June 2009
1. Introduction
The Framework for Conferencing with the Session Initiation Protocol
(SIP) defined in RFC 4353 [RFC4353] presents an overall architecture
for multi-party conferencing. Among others, the framework borrows
from RTP [RFC3550] and extends the concept of a mixer entity
"responsible for combining the media streams that make up a
conference, and generating one or more output streams that are
delivered to recipients". Every participant would hence receive, in
a flat single stream, media originating from all the others.
Using such centralized mixer-based architectures simplifies support
for conference calls on the client side since they would hardly
differ from one-to-one conversations. However, the method also
introduces a few limitations. The flat nature of the streams that a
mixer would output and send to participants makes it difficult for
users to identify the original source of what they are hearing.
Mechanisms that allow the mixer to send to participants cues on
current speakers (e.g. the CSRC fields in RTP [RFC3550]) only work
for speaking/silent binary indications. There are, however, a number
of use cases where one would require more detailed information.
Possible examples include the presence of background chat/noise/
music/typing, someone breathing noisily in their microphone, or other
cases where identifying the source of the disturbance would make it
easy to remove it (e.g. by sending a private IM to the concerned
party asking them to mute their microphone). A more advanced
scenario could involve an intense discussion between multiple
participants that the user does not personally know. Sound level
information would help better recognize the speakers by associating
with them complex (but still human readable) characteristics like
loudness and speed for example.
One way of presenting such information in a user friendly manner
would be for a conferencing client to attach sound level indicators
to the corresponding participant related components in the user
interface as displayed in Figure 1.
Ivov & Marocco Expires December 17, 2009 [Page 3]
Internet-Draft Sound Level Indicators in Conferences June 2009
------------------------
| |
| 00:42 | Weekly Call |
| |
|------------------------|
| |
| Alice |====== | (S) |
| |
| Bob |= | |
| |
| Carol | | (M) |
| |
| Dave |=== | |
| |
|________________________|
Displaying detailed speaker information to the user by including
sound level for every participant.
Figure 1
Implementing a user interface like the above requires analysis of the
media sent from other participants. In a conventional audio
conference this is only possible for the mixer since all other
conference participants are generally receiving a single, flat audio
stream and have therefore no immediate way of determining individual
sound levels.
This document specifies an RTP extension header that allows such
mixers to deliver sound level information to conference participants
by including it directly in the RTP packets transporting the
corresponding audio data.
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
3. Protocol Operation
According to RFC 3550 [RFC3550] a mixer is expected to include in
outgoing RTP packets a list of identifiers (CSRC IDs) indicating the
sources that contributed to the resulting stream. The presence of
such CSRC IDs allows an RTP client to determine, in a binary way, the
Ivov & Marocco Expires December 17, 2009 [Page 4]
Internet-Draft Sound Level Indicators in Conferences June 2009
active speaker(s) in any given moment. RTCP also provides a basic
mechanism to map the CSRC IDs to user identities through the CNAME
field. More advanced mechanisms, may exist depending on the
signaling protocol used to establish and control a conference. In
the case of the Session Initiation Protocol [RFC3261] for example,
the Event Package for Conference State [RFC4575] defines a <src-id>
tag which binds CSRC IDs to media streams and SIP URIs.
This document describes an RTP header extension that allows mixers to
indicate the sound-level of every conference participant (CSRC) in
addition to simply indicating their on/off status. This new header
extension is based on the "General Mechanism for RTP Header
Extensions" [RFC5285].
Each instance of this header contains a list of one-octet sound level
values (see Section 4). Such values indicate sound level on a 0 to
255 scale where 0 is silence (i.e. same as omitting the corresponding
source id from the CSRC list) and 255 corresponds to a threshold
accepted by the mixer implementation as the maximum sound level that
a participant is likely to reach during a conference.
Every sound level value pertains to the CSRC identifier located at
the corresponding position in the CSRC list. In other words, the
first value would indicate the sound level of the conference
participant represented by the first CSRC identifier in that packet
and so forth. The number and order of these values MUST therefore
match the number and order of the CSRC IDs present in the same
packet.
When encoding sound level information, a mixer SHOULD include in a
packet information that corresponds to the audio data being
transported in that same packet. It is important that these values
follow the actual stream as closely as possible. Therefore a mixer
SHOULD also calculate the values after the original contributing
stream has undergone possible processing such as level normalization,
and noise reduction for example.
Note that in some cases a mixer may be sending an RTP audio stream
that only contains sound level information and no actual audio.
Updating a (web) interface conference module may be one reason for
this to happen.
It may sometimes happen that a conference involves more than a single
mixer. In such cases each of the mixers MAY choose to relay the CSRC
list and sound-level information they receive from peer mixers (as
long as the total CSRC count remains below 16). Given that the
maximum sound level is not precisely defined by this specification,
it is likely that in such situations average sound levels would be
Ivov & Marocco Expires December 17, 2009 [Page 5]
Internet-Draft Sound Level Indicators in Conferences June 2009
perceptibly different for the participants located behind the
different mixers.
4. Header Format
The sound level indicators are delivered to the receivers in-band
using the "General Mechanism for RTP Header Extensions" [RFC5285].
The payload of this extension (the transmitted list of sound level
values) is a sequence of 8-bit unsigned integers.
The form of the sound level indicators extension block is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID | len | level 1 | level 2 | level 3 ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The 4-bit len field is the number minus one of data bytes (i.e. sound
level values) transported in this header extension element following
the one-byte header. Therefore, the value zero in this field
indicates that one byte of data follows. A value of 15 is not
allowed by this specification and it MUST NOT be used as the RTP
header can carry a maximum of 15 CSRC IDs. The maximum value allowed
is therefore 14 indicating a following sequence of 15 sound level
values.
Note that use of the two-byte header defined in RFC 5285 [RFC5285]
follows the same rules the only change being the length of the ID and
len fields.
5. Signaling Information
The URI for declaring the sound level header extension in an SDP
extmap attribute and mapping it to a local extension header
identifier is "urn:ietf:params:rtp-hdrext:csrc-sound-level". There
is no additional setup information needed for this extension (i.e. no
extensionattributes).
An example attribute line in the SDP, for a conference might be:
a=extmap:7 urn:ietf:params:rtp-hdrext:csrc-sound-level
The above mapping will most often be provided per media stream (in
the media-level section(s) of SDP, i.e., after an "m=" line) or
globally if there is more than one stream containing sound level
Ivov & Marocco Expires December 17, 2009 [Page 6]
Internet-Draft Sound Level Indicators in Conferences June 2009
indicators in a session.
Presence of the above attribute in the SDP description of a media
stream indicates that some or all RTP packets in that stream would
contain the sound level information RTP extension header.
Conferencing clients that support sound level indicators and have no
mixing capabilities SHOULD always include the direction parameter in
the "extmap" attribute setting it to "recvonly". Conference focus
entities with mixing capabilities MAY omit the direction or set it to
"sendrecv" in SDP offers. Such entities SHOULD set it to "sendonly"
in SDP answers to offers with a "recvonly" parameter and to
"sendrecv" when answering other "sendrecv" offers.
The following Figure 2 and Figure 3 show two example offer/answer
exchanges between a conferencing client and a focus, and between two
conference focus entities.
v=0
o=alice 2890844526 2890844526 IN IP6 host.example.com
c=IN IP6 host.example.com
t=0 0
m=audio 49170 RTP/AVP 0 4
a=rtpmap:0 PCMU/8000
a=rtpmap:4 G723/8000
a=extmap:1/recvonly urn:ietf:params:rtp-hdrext:csrc-sound-level
v=0
i=A Seminar on the session description protocol
o=conf-focus 2890844730 2890844730 IN IP6 focus.example.net
c=IN IP6 focus.example.net
t=0 0
m=audio 52543 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=extmap:1/sendonly urn:ietf:params:rtp-hdrext:csrc-sound-level
A client-initiated example SDP offer/answer exchange negotiating an
audio stream with one-way flow of of sound level information.
Figure 2
Ivov & Marocco Expires December 17, 2009 [Page 7]
Internet-Draft Sound Level Indicators in Conferences June 2009
v=0
i=Un seminaire sur le protocole de description des sessions
o=fr-focus 2890844730 2890844730 IN IP6 focus.fr.example.net
c=IN IP6 focus.fr.example.net
t=0 0
m=audio 49170 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=extmap:1/sendrecv urn:ietf:params:rtp-hdrext:csrc-sound-level
v=0
i=A Seminar on the session description protocol
o=us-focus 2890844526 2890844526 IN IP6 focus.us.example.net
c=IN IP6 focus.us.example.net
t=0 0
m=audio 52543 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=extmap:1/sendrecv urn:ietf:params:rtp-hdrext:csrc-sound-level
An example SDP offer/answer exchange between two conference focus
entities with mixing capabilities negotiating an audio stream with
bidirectional flwo of sound level information.
Figure 3
6. Security Considerations
1. This document defines a means of attributing sound level to a
particular participant in a conference. An attacker may try to
modify the content of RTP packets in a way that would make sound
activity from one participant appear as coming from another.
2. Furthermore, the fact that sound level values would not be
protected even in an SRTP session may be of concern in some cases
where the activity of a particular participant in a conference is
confidential.
3. Both of the above are concerns that stem from the design of the
RTP protocol itself. It is therefore important that according to
the needs of a particular scenario, implementors and deployers
consider use of a lower level security and authentication
mechanism.
7. IANA Considerations
This document defines a new extension URI that, if approved, would
need to be added to the RTP Compact Header Extensions sub-registry of
the Real-Time Transport Protocol (RTP) Parameters registry, according
to the following data:
Ivov & Marocco Expires December 17, 2009 [Page 8]
Internet-Draft Sound Level Indicators in Conferences June 2009
Extension URI: urn:ietf:params:rtp-hdrext:csrc-sound-level
Description: Sound level indicators
Contact: emcho@sip-communicator.org
Reference: RFC XXXX
8. Open Issues
At the time of writing of this document the authors have no clear
view on how and if the following list of issues should be address
here:
1. Specific sound level mappings. The current version of this
specification treats sound level indicators as referable to any
scale chosen by the mixer. The only limitations consist in
making sure that the value of 0 should correspond to participant
inactivity/silence and the value 255x to a level that would
appear to users as loud but still attainable. It is however
possible to map specific levels (e.g. measured in dBm) with the
purpose of achieving cross-mixer uniformity of these values. An
obvious tradeoff here is the increased complexity of
implementation that would require mixers to convert sound level
to whatever specific unit they use for internal estimation, which
could be non-trivial in a number of cases.
2. Sound levels in video streams. This specification allows use of
sound level values in "silent" audio streams that don't otherwise
carry any payload thus allowing their delivery within systems
where the various focus/mixer components communicate with each
other as conference participants. The same train of thought may
very well justify sound level transport in video streams.
9. Acknowledgments
Roni Even, Ingemar Johansson, and several others provided helpful
feedback over the dispatch mailing list.
SIP Communicator's participation in this specification is funded by
the NLnet Foundation.
10. Appendix: An alternative approach
The problem statement [I-D.ivov-dispatch-slic-ps] preceding this
document originally favored a slightly different resolution approach
that the authors feel may still be relevant and therefore worth
publishing here.
A very simple way for a mixer to use the CSRC fields as a transport
Ivov & Marocco Expires December 17, 2009 [Page 9]
Internet-Draft Sound Level Indicators in Conferences June 2009
means for sound level indication would be to extend their meaning
over a series of packets rather than a single one. This way it could
be specified that the sound-level of a particular participant,
represented on a zero to ten scale, corresponds to the number of
occurrences of its CSRC identifier in the ten most recent RTP packets
received from the mixer.
For example, consider a conference call with four participants:
Alice, Bob, Carol, and Dave. At a certain point in time Alice has a
sound level of 6/10, Bob 1/10, Carol is silent or in other words 0/10
and Dave has a level of 3/10. In order to describe this state the
mixer could have sent the last ten RTP packets with the following
CSRC configuration:
+-------+----+-----+-----+-----+-----+-----+-----+-----+-----+------+
| | P1 | P2 | P3 | P4 | P5 | P6 | P7 | P8 | P9 | P10 |
+-------+----+-----+-----+-----+-----+-----+-----+-----+-----+------+
| Alice | + | + | + | + | + | + | | | | |
| Bob | | + | | | | | | | | |
| Carol | | | | | | | | | | |
| Dave | | | | | | | | + | + | + |
+-------+----+-----+-----+-----+-----+-----+-----+-----+-----+------+
A possible representation of a particular sound level configuration
through the presence/absence of CSRC IDs in subsequent RTP packets.
Table 1
The graphical interface of a user agent involved in such a conference
(like the one sketched in Figure 1) would then display correct sound
levels just showing for each participant as many ticks as were the
occurrencies of the respective CSRC in the previous ten RTP packets.
The algorithm for encoding sound level information this way is
relatively simple. In order to determine whether or not to include a
particular CSRC a mixer should:
o include the CSRC if the sound level of the participant in the
current packet is greater than the number of occurrencies of that
same CSRC in the nine previous packets;
o omit the CSRC if the sound level of the participant in the current
packet is lower than or equal to the number of occurrencies of
that same CSRC in the nine previous packets.
There are several advantages to using this approach, the most obvious
being its simplicity as well as the fact that sound level information
is transported together with the parts of the audio stream that it
actually concerns which should make synchronization straightforward.
Ivov & Marocco Expires December 17, 2009 [Page 10]
Internet-Draft Sound Level Indicators in Conferences June 2009
The technique would also work with other signaling protocols using
RTP such as XMPP's [RFC3920] Jingle extensions for example.
One of the first disadvantages that come to mind with this approach
is the fact that mixer would not be able to indicate level in a
single packet but would have to distribute it over a succession of up
to ten packets which would reduce the reactivity of the
representation.
It is probably worth mentioning, however, that a granularity that
allows switching from a level of zero to ten and back to zero again
in an instant manner is not of much use anyway since such UI updates
would be barely perceptible to the user. Still, this is a UI
decision and making it on a protocol level may bring some
inconveniences.
Another possible problem would come from implementations using CSRC
presence in a binary way to determine current speaker. When running
against a mixer that supports sound level indication such
implementations may appear to be jumpy as the participants that they
are designating as active may be changing status too rapidly.
11. References
11.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC5285] Singer, D. and H. Desineni, "A General Mechanism for RTP
Header Extensions", RFC 5285, July 2008.
11.2. Informative References
[I-D.ietf-mmusic-ice]
Rosenberg, J., "Interactive Connectivity Establishment
(ICE): A Protocol for Network Address Translator (NAT)
Traversal for Offer/Answer Protocols",
draft-ietf-mmusic-ice-19 (work in progress), October 2007.
[I-D.ivov-dispatch-slic-ps]
Ivov, E. and E. Marocco, "Dispatching Sound Level
Indicators in Conferences (Problem Statement)",
Ivov & Marocco Expires December 17, 2009 [Page 11]
Internet-Draft Sound Level Indicators in Conferences June 2009
draft-ivov-dispatch-slic-ps-00 (work in progress),
May 2009.
[RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
A., Peterson, J., Sparks, R., Handley, M., and E.
Schooler, "SIP: Session Initiation Protocol", RFC 3261,
June 2002.
[RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and
Video Conferences with Minimal Control", STD 65, RFC 3551,
July 2003.
[RFC3920] Saint-Andre, P., Ed., "Extensible Messaging and Presence
Protocol (XMPP): Core", RFC 3920, October 2004.
[RFC4353] Rosenberg, J., "A Framework for Conferencing with the
Session Initiation Protocol (SIP)", RFC 4353,
February 2006.
[RFC4575] Rosenberg, J., Schulzrinne, H., and O. Levin, "A Session
Initiation Protocol (SIP) Event Package for Conference
State", RFC 4575, August 2006.
Authors' Addresses
Emil Ivov
SIP Communicator
Strasbourg 67000
France
Email: emcho@sip-communicator.org
Enrico Marocco
Telecom Italia
Via G. Reiss Romoli, 274
Turin 10148
Italy
Email: enrico.marocco@telecomitalia.it
Ivov & Marocco Expires December 17, 2009 [Page 12]