[Search] [txt|pdfized|bibtex] [Tracker] [Email] [Nits]
Versions: 00                                                            
XRBLOCK Working Group                                   Alan Clark
Internet Draft                                            Telchemy
Intended status: Standards Track                    Martin Kastner
Expires: May 17, 2012                                     Telchemy
                                                        Geoff Hunt
                                                 November 14, 2011

      RTCP XR Report Block for QoE Metrics Reporting


   This document defines an RTCP XR Report Block that allows the
   reporting of QoE metrics for use in voice, audio and video

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on May 17, 2012.

Copyright Notice

   Copyright (c) 2011 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Clark & Kastner                                                [Page  1]

RTCP XR QoE Metrics                                        November 2011

1. Introduction

1.1. QoE Metrics Report Block

   This draft defines a new block types to augment those defined
   in RFC3611 for use in reporting QoE metrics.  QoE metrics consider
   the impact of a range of transmission and payload (content)
   related impairments on the quality of a service from the user

1.2. RTCP and RTCP XR Reports

   The use of RTCP for reporting is defined in RFC3550 [2].
   RFC3611 [3] defined an extensible structure for reporting using
   an RTCP Extended Report (XR).  This draft defines
   a new Extended Report block that MUST be used as defined in
   RFC3550 and RFC3611.

1.3 Performance Metrics Framework

   The Performance Metrics Framework [9] provides guidance on the
   definition and specification of performance metrics.  Metrics
   described in this draft either reference external definitions
   or define metrics generally in accordance with the guidelines
   in [9].

1.4 Applicability

   This memo applies to any application of RTP for which QoE
   measurement algorithms are defined.

2. Definitions

2.1 QoE Metrics

   A QoE ("Quality of Experience") metric is intended to provide a
   measure that is indicative of the user's view of a service.
   This is commonly expressed as a MOS ("Mean Opinion Score") which
   usually (but not always) is a 1.0-5.0 numerical scale in which
   a 1.0 represents "Unacceptable" and 5.0 represents "Excellent".

   True MOS scores are obtained using subjective testing, and tend
   vary from test to test.  Subjective testing is also not
   suitable for measuring the quality of operational services and
   hence it is common practice to use objective algorithms to
   estimate subjective quality.  During the development of such QoE
   algorithms, there is extensive comparison against both subjective
   test data and data from other "trusted" objective test tools.

Clark & Kastner                                                [Page  2]

RTCP XR QoE Metrics                                        November 2011

   ITU-T Recommendation P.564 defines a methodology for verifying
   the performance of QoE estimation algorithms for Voice over IP
   services.  There is standardization work underway related to

   QoE metrics for video and audio.  The continuous progression of
   work in this area means that new algorithms may be defined in
   the future, hence this memo does make provision for new
   algorithms.  Implementors are advised that IPR disclosures
   have been made in respect of most known QoE estimation algorithms
   and they should check the IPR disclosure databases and policies of
   the relevant standards organizations (for example ITU and ETSI).

   ITU-T Recommendation P.800.1 describes terminology that should be
   use for MOS scores used to describe Speech quality.  This uses the
   abbreviations LQ and CQ for Listening and Conversational Quality
   respectively, and extends these using O for Objective, E for
   Estimated and S for Subjective.  Hence an objectively measured
   listening quality MOS score would be denoted MOS-LQO.

   MOS scores typically use a common scale of 1 to 5 and are scaled
   for comparison with subjectively measured MOS.  MOS scores for
   narrowband speech and wideband speech, or for low resolution
   video and high resolution video are typically placed into the
   same range.  This occurs because a subjective test is usually
   a comparitive test amongst similar codecs or devices. Hence a
   high quality AMR-WB or G.722 wideband voice call may have a lower
   MOS score than a narrowband G.729 call, even though the quality is
   higher.  Similary, a video subjective test typically uses devices
   with similar resolution and hence a high definition system may
   have the same MOS score as a standard definition system.

   ITU-T P.800.1 addressed this issue of MOS scaling through the use
   of an additional N or W qualifier to denote Narrowband or Wideband.
   So a MOS-LQON score is an objectively measured listening quality
   MOS for narrowband (8kHz sample rate) conditions.  Some codecs
   are able to switch dynamically between narrowband and wideband,
   which is addressed by the the "M" or mixed qualifier.

   The issue for audio video MOS is very similar to that of speech.
   This is addressed by recent work in ITU-T [11] which introduced
   the idea of Absolute and Relative MOS.  Absolute MOS "does"
   include the effects of image resolution whereas Relative MOS does
   "not".  This draft presumes that ITU-T will adopt similar
   terminology to P.800.1 for video MOS. [Editors note, will need
   updating as ITU update relevant standards]

   Two cases of MOS-CQ have been treated separately in this draft.
   The first of these is MOS-CQEN, which is an Estimated (not
   measured) MOS based on ITU-T G.107. The MOS value is calculated
   by first calculating an R (or RCQ) value and then converting

Clark & Kastner                                                [Page  3]

RTCP XR QoE Metrics                                        November 2011

   this to a MOS.  This conversion leads to a MOS score that is
   typically higher than current subjective test data (4.45 vs
   4.2), which can lead to difficulty interpreting the values.
   The second case is MOS-CQEN-TTC which is related to a Japanese
   national standard - TTC JJ201.01.  JJ201.01 is based on G.107
   however Japanese MOS scores are typically much lower than those
   in other countries and a MOS score for G.711 would be 3.8 in
   Japan versus 4.2 for a typical subjective test and 4.45 for

   It is extremely important that the correct MOS is referenced.
   For example a MOS of 3.6 would represent a small degree of
   degration (0.2) using the Japanese JJ201.01 scaling but a very
   large degradation ( 0.85) using G.107 scaling.

2.2 Channel

   Certain types of encoder (for example stereo audio codecs)
   incorporate multiple audio or video channels into a single encoded
   stream which is then packetized and carried in RTP or MPEG
   Transport. Within the scope of this memo, the term "channel"
   applies to this definition only - if multiple audio or video
   streams are carried either in separate RTP sessions (identified
   by an SSRC) or MPEG Transport program streams (identified by a
   PID) then the Measurement Identifier block MUST be used to
   identify the stream to which metrics apply.

3. QoE Metrics Block

3.1 Report Block Structure

    0               1               2               3
    0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
   |     BT=N      | I |   Tag     |         block length          |
   |Chan |     Type    |  Calc alg |         QoE Metric            |
   |Chan |     Type    |  Calc alg |         QoE Metric            |

3.2 Definition of Fields in QoE Metric Report Block

   block type (BT): 8 bits

      A QoE Report Block is identified by the constant
   [Note to RFC Editor: please replace QOEX with the IANA provided RTCP
   XR block type for this block.]

Clark & Kastner                                                [Page  4]

RTCP XR QoE Metrics                                        November 2011

   Measurement Type Indication (I): 2 bits

      This field is used to indicate whether the QoE Metrics are
      Sampled, Interval or Cumulative metrics, that is, whether the
      reported values applies to the most recent measurement interval

      duration between successive metrics reports (I=10) (the Interval
      Duration), to the accumulation period characteristic of
      cumulative measurements (I=11) (the Cumulative Duration) or is
      a sampled instantaneous value (I=01).

      Numerical values for interval or duration are provided in the
      Measurement Identifier block referenced by the tag field below.

   Measurement Identifier association (tag): 6 bits

      This field is used to identify the Measurement Identifier block
      which describes this measurement.  The relevant Measurement
      Identifier block has the same tag value as the QoE block
      Note that there may be more than one Measurement Identifier block
      per RTCP packet.

   Block length: 16 bits

      The length of this report block in 32-bit words minus one.


      The channel number of the audio or video stream to which this
      metric applies


     0000000 - 0011111 Speech QoE Scores

     0100000 - 0111111 Audio QoE Scores

     1000000 - 1011111 Video QoE Scores

     1100000 - 1111111 Other application QoE Scores

Clark & Kastner                                                [Page  5]

RTCP XR QoE Metrics                                        November 2011

   Speech QoE Scores (see ITU-T P.800.1 [10] for definitions)

     0000000  MOS-LQON - Listening Quality MOS (Narrowband)
     0000001  MOS-LQOW - Listening Quality MOS (Wideband)
     0000010  MOS-LQOU - Listening Quality MOS (Ultra wideband)
     0000011  MOS-LQOM - Listening Quality MOS (Mixed)
     0000100-0000111 - Reserved

     0001000  MOS-CQON - Conversational Quality MOS (Narrowband)
     0001001  MOS-CQOW - Conversational Quality MOS (Wideband)
     0001010  MOS-CQOU - Conversational Quality MOS (Ultra wideband)
     0001011  MOS-CQOM - Conversational Quality MOS (Mixed)
     0001100  MOS-CQEN - Conversational Quality MOS (Narrowband)
                        Scaled per ITU-T G.107
     0001101  MOS-CQEN-TTC  - Conversational Quality MOS (Narrowband)
                        Scaled per TTC JJ201.01 [8] (Japan)
     0001110-0001111 - Reserved

     0010000  MOS-TQON - Talking Quality MOS (Narrowband)
     0010001  MOS-TQOW - Talking Quality MOS (Wideband)
     0010010  MOS-TQOU - Talking Quality MOS (Ultra wideband)
     0010011  MOS-TQOM - Talking Quality MOS (Mixed)
     0010100 - 0010111 - Reserved

     0011000  R-LQ     - R Factor - Listening Quality
     0011001  R-CQ     - R Factor - Conversational Quality [6]
     0011010 - 0011111 - Reserved

   Audio QoE Scores (see ITU-T P.??? and [11])

     0100000  Absolute MOS-AQOA - Audio Quality MOS, absolute scaling
     0100001  Relative MOS-AQOR - Audio Quality MOS, relative scaling

   Video and Multimedia QoE Scores (see ITU-T P.??? and [11])

     1000000  Absolute MOS-VQOA - Video Quality MOS, absolute scaling
     1000001  Relative MOS-VQOR - Video Quality MOS, relative scaling
     1000100  Absolute MOS-AQOA - Audio-Video Quality MOS, absolute
     1000101  Relative MOS-AQOR - Audio-Video Quality MOS, relative

   Other application QoE Scores

     1100000 - 1111111  Reserved for other interactive applications
                       that use RTP for communication

Clark & Kastner                                                [Page  6]

RTCP XR QoE Metrics                                        November 2011

   Calculation Algorithm

     0      -   ITU-T P.564 Compliant Algorithm [5] (Voice)
     1      -   G.107 [6] (Voice)
     2      -   G.107 / ETSI TS 101 329-5 Annex E [6,7] (Voice)
     3      -   TTC JJ201.01 [8] (Japan)
     4      -   Reserved for ITU-T P.NAMS
     5      -   Reserved for ITU-T P.NBAMS
     255    -   Indicated via SDP

   QoE Metric

     A 8:8 integer scaled representation of the QoE metric value.
     This allows values in the range 0.0 to 255.996 to be represented.

4. SDP Signaling

   RFC3611 [3] defines the use of SDP (Session Description Protocol)
   [4] for signaling the use of XR blocks.  XR blocks MAY be used
   without prior signaling.

   This section augments the SDP [4] attribute "rtcp-xr" defined in
   RFC3611[3] by providing a "xr-format" to signal the use of the report
   block defined in this document.

     rtcp-xr-attrib = "a=" "rtcp-xr" ":" [xr-format *(SP xr-format)]
           CRLF (defined in RFC3611)

     xr-format = xr-format /

     qoe-metrics   = "qoe-metrics" [EQUAL word]
     DIGIT          = %x30-39
     format-ext     = non-ws-string
     non-ws-string  = 1*(%x21-FF)
     CRLF           = %d13.10

5. IANA Considerations

   This document creates a new block type within the IANA "RTCP XR Block
   Type Registry" called the QoE Metrics, and a new [new-xrblock]
   parameter within the "RTCP XR SDP Parameters Registry".

Clark & Kastner                                                [Page  7]

RTCP XR QoE Metrics                                        November 2011

6. Security Considerations

   RTCP reports can contain sensitive information since they can provide
   information about the nature and duration of a session established
   between two or more endpoints.

7. Contributors

8. References


   [1] Bradner, S., "Key words for use in RFCs to Indicate
   Requirement Levels", BCP 14, RFC 2119, March 1997.

   [2]  Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson,
   "RTP: A Transport Protocol for Real-Time Applications", STD 64,
   RFC 3550, July 2003.

   [3] Friedman, T., Caceres, R., and A. Clark, "RTP Control Protocol
   Extended Reports (RTCP XR)", RFC 3611, November 2003.

   [4] Handley, M. and V. Jacobson, "SDP: Session Description
   Protocol", RFC 4566, July 2006.

   [5] ITU-T Recommendation P.564, Conformance testing for narrowband
   Coice over IP transmission quality assessment models

   [6] ITU-T Recommendation G.107, "The E Model, a computational model
   for use in transmission planning"

   [7] ETSI TS 101 329-5, QoS Measurement for Voice over IP

   [8] TTC 201.01 (Japan) A method for speech quality assessment
   for Coice over IP

   [9] Clark A., Claise B. "Guidelines for Considering New Performance
   Metrics Development", RFC6390, October 2011

   [10] ITU-T P.800.1 "Mean Opinion Score (MOS) terminology"


   [11] ITU-T TD483 "Interpretation of MOS in different contexts",
   January 2011

Clark & Kastner                                                [Page  8]

RTCP XR QoE Metrics                                        November 2011

Author's Addresses

   Alan Clark
   Telchemy Incorporated
   2905 Premiere Parkway, Suite 280
   Duluth, GA  30097

   Email: alan.d.clark@telchemy.com

   Martin Kastner
   Telchemy Incorporated
   2905 Premiere Parkway, Suite 280
   Duluth, GA  30097

   Email: martin.kastner@telchemy.com

   Geoff Hunt

Clark & Kastner                                                [Page  9]