XRBLOCK Working Group Alan Clark
Internet Draft Telchemy
Intended status: Standards Track Martin Kastner
Expires: May 17, 2012 Telchemy
Geoff Hunt
Unaffiliated
November 14, 2011
RTCP XR Report Block for QoE Metrics Reporting
draft-clark-xrblock-rtcp-xr-qoe-00
Abstract
This document defines an RTCP XR Report Block that allows the
reporting of QoE metrics for use in voice, audio and video
services.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 17, 2012.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Clark & Kastner [Page 1]
RTCP XR QoE Metrics November 2011
1. Introduction
1.1. QoE Metrics Report Block
This draft defines a new block types to augment those defined
in RFC3611 for use in reporting QoE metrics. QoE metrics consider
the impact of a range of transmission and payload (content)
related impairments on the quality of a service from the user
viewpoint.
1.2. RTCP and RTCP XR Reports
The use of RTCP for reporting is defined in RFC3550 [2].
RFC3611 [3] defined an extensible structure for reporting using
an RTCP Extended Report (XR). This draft defines
a new Extended Report block that MUST be used as defined in
RFC3550 and RFC3611.
1.3 Performance Metrics Framework
The Performance Metrics Framework [9] provides guidance on the
definition and specification of performance metrics. Metrics
described in this draft either reference external definitions
or define metrics generally in accordance with the guidelines
in [9].
1.4 Applicability
This memo applies to any application of RTP for which QoE
measurement algorithms are defined.
2. Definitions
2.1 QoE Metrics
A QoE ("Quality of Experience") metric is intended to provide a
measure that is indicative of the user's view of a service.
This is commonly expressed as a MOS ("Mean Opinion Score") which
usually (but not always) is a 1.0-5.0 numerical scale in which
a 1.0 represents "Unacceptable" and 5.0 represents "Excellent".
True MOS scores are obtained using subjective testing, and tend
vary from test to test. Subjective testing is also not
suitable for measuring the quality of operational services and
hence it is common practice to use objective algorithms to
estimate subjective quality. During the development of such QoE
algorithms, there is extensive comparison against both subjective
test data and data from other "trusted" objective test tools.
Clark & Kastner [Page 2]
RTCP XR QoE Metrics November 2011
ITU-T Recommendation P.564 defines a methodology for verifying
the performance of QoE estimation algorithms for Voice over IP
services. There is standardization work underway related to
QoE metrics for video and audio. The continuous progression of
work in this area means that new algorithms may be defined in
the future, hence this memo does make provision for new
algorithms. Implementors are advised that IPR disclosures
have been made in respect of most known QoE estimation algorithms
and they should check the IPR disclosure databases and policies of
the relevant standards organizations (for example ITU and ETSI).
ITU-T Recommendation P.800.1 describes terminology that should be
use for MOS scores used to describe Speech quality. This uses the
abbreviations LQ and CQ for Listening and Conversational Quality
respectively, and extends these using O for Objective, E for
Estimated and S for Subjective. Hence an objectively measured
listening quality MOS score would be denoted MOS-LQO.
MOS scores typically use a common scale of 1 to 5 and are scaled
for comparison with subjectively measured MOS. MOS scores for
narrowband speech and wideband speech, or for low resolution
video and high resolution video are typically placed into the
same range. This occurs because a subjective test is usually
a comparitive test amongst similar codecs or devices. Hence a
high quality AMR-WB or G.722 wideband voice call may have a lower
MOS score than a narrowband G.729 call, even though the quality is
higher. Similary, a video subjective test typically uses devices
with similar resolution and hence a high definition system may
have the same MOS score as a standard definition system.
ITU-T P.800.1 addressed this issue of MOS scaling through the use
of an additional N or W qualifier to denote Narrowband or Wideband.
So a MOS-LQON score is an objectively measured listening quality
MOS for narrowband (8kHz sample rate) conditions. Some codecs
are able to switch dynamically between narrowband and wideband,
which is addressed by the the "M" or mixed qualifier.
The issue for audio video MOS is very similar to that of speech.
This is addressed by recent work in ITU-T [11] which introduced
the idea of Absolute and Relative MOS. Absolute MOS "does"
include the effects of image resolution whereas Relative MOS does
"not". This draft presumes that ITU-T will adopt similar
terminology to P.800.1 for video MOS. [Editors note, will need
updating as ITU update relevant standards]
Two cases of MOS-CQ have been treated separately in this draft.
The first of these is MOS-CQEN, which is an Estimated (not
measured) MOS based on ITU-T G.107. The MOS value is calculated
by first calculating an R (or RCQ) value and then converting
Clark & Kastner [Page 3]
RTCP XR QoE Metrics November 2011
this to a MOS. This conversion leads to a MOS score that is
typically higher than current subjective test data (4.45 vs
4.2), which can lead to difficulty interpreting the values.
The second case is MOS-CQEN-TTC which is related to a Japanese
national standard - TTC JJ201.01. JJ201.01 is based on G.107
however Japanese MOS scores are typically much lower than those
in other countries and a MOS score for G.711 would be 3.8 in
Japan versus 4.2 for a typical subjective test and 4.45 for
G.107.
It is extremely important that the correct MOS is referenced.
For example a MOS of 3.6 would represent a small degree of
degration (0.2) using the Japanese JJ201.01 scaling but a very
large degradation ( 0.85) using G.107 scaling.
2.2 Channel
Certain types of encoder (for example stereo audio codecs)
incorporate multiple audio or video channels into a single encoded
stream which is then packetized and carried in RTP or MPEG
Transport. Within the scope of this memo, the term "channel"
applies to this definition only - if multiple audio or video
streams are carried either in separate RTP sessions (identified
by an SSRC) or MPEG Transport program streams (identified by a
PID) then the Measurement Identifier block MUST be used to
identify the stream to which metrics apply.
3. QoE Metrics Block
3.1 Report Block Structure
0 1 2 3
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| BT=N | I | Tag | block length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Chan | Type | Calc alg | QoE Metric |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
..........
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Chan | Type | Calc alg | QoE Metric |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
3.2 Definition of Fields in QoE Metric Report Block
block type (BT): 8 bits
A QoE Report Block is identified by the constant
QOEX.
[Note to RFC Editor: please replace QOEX with the IANA provided RTCP
XR block type for this block.]
Clark & Kastner [Page 4]
RTCP XR QoE Metrics November 2011
Measurement Type Indication (I): 2 bits
This field is used to indicate whether the QoE Metrics are
Sampled, Interval or Cumulative metrics, that is, whether the
reported values applies to the most recent measurement interval
duration between successive metrics reports (I=10) (the Interval
Duration), to the accumulation period characteristic of
cumulative measurements (I=11) (the Cumulative Duration) or is
a sampled instantaneous value (I=01).
Numerical values for interval or duration are provided in the
Measurement Identifier block referenced by the tag field below.
Measurement Identifier association (tag): 6 bits
This field is used to identify the Measurement Identifier block
which describes this measurement. The relevant Measurement
Identifier block has the same tag value as the QoE block
Note that there may be more than one Measurement Identifier block
per RTCP packet.
Block length: 16 bits
The length of this report block in 32-bit words minus one.
Channel
The channel number of the audio or video stream to which this
metric applies
Type
0000000 - 0011111 Speech QoE Scores
0100000 - 0111111 Audio QoE Scores
1000000 - 1011111 Video QoE Scores
1100000 - 1111111 Other application QoE Scores
Clark & Kastner [Page 5]
RTCP XR QoE Metrics November 2011
Speech QoE Scores (see ITU-T P.800.1 [10] for definitions)
0000000 MOS-LQON - Listening Quality MOS (Narrowband)
0000001 MOS-LQOW - Listening Quality MOS (Wideband)
0000010 MOS-LQOU - Listening Quality MOS (Ultra wideband)
0000011 MOS-LQOM - Listening Quality MOS (Mixed)
0000100-0000111 - Reserved
0001000 MOS-CQON - Conversational Quality MOS (Narrowband)
0001001 MOS-CQOW - Conversational Quality MOS (Wideband)
0001010 MOS-CQOU - Conversational Quality MOS (Ultra wideband)
0001011 MOS-CQOM - Conversational Quality MOS (Mixed)
0001100 MOS-CQEN - Conversational Quality MOS (Narrowband)
Scaled per ITU-T G.107
0001101 MOS-CQEN-TTC - Conversational Quality MOS (Narrowband)
Scaled per TTC JJ201.01 [8] (Japan)
0001110-0001111 - Reserved
0010000 MOS-TQON - Talking Quality MOS (Narrowband)
0010001 MOS-TQOW - Talking Quality MOS (Wideband)
0010010 MOS-TQOU - Talking Quality MOS (Ultra wideband)
0010011 MOS-TQOM - Talking Quality MOS (Mixed)
0010100 - 0010111 - Reserved
0011000 R-LQ - R Factor - Listening Quality
0011001 R-CQ - R Factor - Conversational Quality [6]
0011010 - 0011111 - Reserved
Audio QoE Scores (see ITU-T P.??? and [11])
0100000 Absolute MOS-AQOA - Audio Quality MOS, absolute scaling
0100001 Relative MOS-AQOR - Audio Quality MOS, relative scaling
Video and Multimedia QoE Scores (see ITU-T P.??? and [11])
1000000 Absolute MOS-VQOA - Video Quality MOS, absolute scaling
1000001 Relative MOS-VQOR - Video Quality MOS, relative scaling
1000100 Absolute MOS-AQOA - Audio-Video Quality MOS, absolute
1000101 Relative MOS-AQOR - Audio-Video Quality MOS, relative
Other application QoE Scores
1100000 - 1111111 Reserved for other interactive applications
that use RTP for communication
Clark & Kastner [Page 6]
RTCP XR QoE Metrics November 2011
Calculation Algorithm
0 - ITU-T P.564 Compliant Algorithm [5] (Voice)
1 - G.107 [6] (Voice)
2 - G.107 / ETSI TS 101 329-5 Annex E [6,7] (Voice)
3 - TTC JJ201.01 [8] (Japan)
4 - Reserved for ITU-T P.NAMS
5 - Reserved for ITU-T P.NBAMS
255 - Indicated via SDP
QoE Metric
A 8:8 integer scaled representation of the QoE metric value.
This allows values in the range 0.0 to 255.996 to be represented.
4. SDP Signaling
RFC3611 [3] defines the use of SDP (Session Description Protocol)
[4] for signaling the use of XR blocks. XR blocks MAY be used
without prior signaling.
This section augments the SDP [4] attribute "rtcp-xr" defined in
RFC3611[3] by providing a "xr-format" to signal the use of the report
block defined in this document.
rtcp-xr-attrib = "a=" "rtcp-xr" ":" [xr-format *(SP xr-format)]
CRLF (defined in RFC3611)
xr-format = xr-format /
qoe-metrics
qoe-metrics = "qoe-metrics" [EQUAL word]
DIGIT = %x30-39
format-ext = non-ws-string
non-ws-string = 1*(%x21-FF)
CRLF = %d13.10
5. IANA Considerations
This document creates a new block type within the IANA "RTCP XR Block
Type Registry" called the QoE Metrics, and a new [new-xrblock]
parameter within the "RTCP XR SDP Parameters Registry".
Clark & Kastner [Page 7]
RTCP XR QoE Metrics November 2011
6. Security Considerations
RTCP reports can contain sensitive information since they can provide
information about the nature and duration of a session established
between two or more endpoints.
7. Contributors
8. References
Normative
[1] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[2] Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson,
"RTP: A Transport Protocol for Real-Time Applications", STD 64,
RFC 3550, July 2003.
[3] Friedman, T., Caceres, R., and A. Clark, "RTP Control Protocol
Extended Reports (RTCP XR)", RFC 3611, November 2003.
[4] Handley, M. and V. Jacobson, "SDP: Session Description
Protocol", RFC 4566, July 2006.
[5] ITU-T Recommendation P.564, Conformance testing for narrowband
Coice over IP transmission quality assessment models
[6] ITU-T Recommendation G.107, "The E Model, a computational model
for use in transmission planning"
[7] ETSI TS 101 329-5, QoS Measurement for Voice over IP
[8] TTC 201.01 (Japan) A method for speech quality assessment
for Coice over IP
[9] Clark A., Claise B. "Guidelines for Considering New Performance
Metrics Development", RFC6390, October 2011
[10] ITU-T P.800.1 "Mean Opinion Score (MOS) terminology"
Informative
[11] ITU-T TD483 "Interpretation of MOS in different contexts",
January 2011
Clark & Kastner [Page 8]
RTCP XR QoE Metrics November 2011
Author's Addresses
Alan Clark
Telchemy Incorporated
2905 Premiere Parkway, Suite 280
Duluth, GA 30097
USA
Email: alan.d.clark@telchemy.com
Martin Kastner
Telchemy Incorporated
2905 Premiere Parkway, Suite 280
Duluth, GA 30097
USA
Email: martin.kastner@telchemy.com
Geoff Hunt
Unaffiliated
Clark & Kastner [Page 9]