[Search] [txt|xml|pdfized|bibtex] [Tracker] [Email] [Nits]
Versions: 00                                                            
Internet Engineering Task Force                           N. Wilson, Ed.
Internet-Draft                                               RealVNC Ltd
Intended status: Informational                        September 24, 2018
Expires: March 28, 2019

Codec-Layer Feedback for the Real-time Transport Control Protocol (RTCP)


   This document specifies an extension to the messages defined in the
   Audio-Visual Profile with Feedback (AVPF), allowing for an RTP codec
   to send codec-specific feedback via RTCP.  Currently, RTCP feedback
   messages have been defined only for generic capabilities which may be
   defined in the context of individual codecs, and some codecs have
   their own RTCP feedback messages.  There is a need for a feedback
   message at the codec-layer, allowing other codecs to send codec-
   specific feedback.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on March 28, 2019.

Copyright Notice

   Copyright (c) 2018 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must

Wilson                   Expires March 28, 2019                 [Page 1]

Internet-Draft        Codec-Layer Feedback for RTCP       September 2018

   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Definitions . . . . . . . . . . . . . . . . . . . . . . . . .   3
     2.1.  Glossary  . . . . . . . . . . . . . . . . . . . . . . . .   3
     2.2.  Requirements Language . . . . . . . . . . . . . . . . . .   3
   3.  Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . .   3
   4.  Protocol Overview . . . . . . . . . . . . . . . . . . . . . .   4
   5.  Format of RTCP Feedback Message . . . . . . . . . . . . . . .   5
   6.  Semantics of the RTCP Feedback Message  . . . . . . . . . . .   5
   7.  SDP Definition  . . . . . . . . . . . . . . . . . . . . . . .   6
     7.1.  Extension of the rtcp-fb attribute  . . . . . . . . . . .   6
     7.2.  Offer/Answer  . . . . . . . . . . . . . . . . . . . . . .   7
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   7
   9.  Security Considerations . . . . . . . . . . . . . . . . . . .   8
   10. References  . . . . . . . . . . . . . . . . . . . . . . . . .   8
     10.1.  Normative References . . . . . . . . . . . . . . . . . .   8
     10.2.  Informative References . . . . . . . . . . . . . . . . .   9
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .   9

1.  Introduction

   The Extended RTP Profile for RTCP-Based Feedback (RTP/AVPF,
   [RFC4585]) introduced the concept of feedback messages, which may be
   used by RTP peers to exchange information controlling an RTP stream.
   The framework provided by RTP/AVPF divides such feedback into
   Transport-Layer (RTPFB) and Payload-Specific feedback (PSFB)
   messages.  These include generic capabilities such as "Slice Loss
   Indication" (SLI), which are widely applicable to many different
   codecs, yet require interpretation in the context of each codec,
   since the notion of "slice" is codec-specific.  Since then, further
   Codec-Control messages have been defined in the same framework, which
   further expand the generic messages defined for the RTP/AVPF control
   layer, for example "Full Intra Request" from [RFC5104], which follows
   the same model as SLI.

   However, these generic messages do not enable truly codec-specific
   feedback to be exchanged.  There is currently no payload-specific
   feedback number assigned for codec-defined feedback which is not
   covered by the existing mechanisms (SLI, FIR, and others).  Some
   codecs have resorted to defining their own top-level feedback
   messages (such as the H.271 Video Back Channel Message, [RFC5104]),
   yet the space of assigned numbers for PSFB messages is scarce, and it

Wilson                   Expires March 28, 2019                 [Page 2]

Internet-Draft        Codec-Layer Feedback for RTCP       September 2018

   is not reasonable to allocate further numbers for messages that are
   strongly tied to specific codecs.

   Some applications may make use of the Application-Layer Feedback
   capability from [RFC4585], however this is unsatisfactory for
   widespread use, since a codec may be used by more than one
   application, at which point collisions may occur in the use of the
   application-layer feedback.

   This document therefore requests that a PSFB number be reserved for
   codec-layer feedback.  Such feedback is specific to an individual
   codec, as identified by the payload type identifier carried in the
   feedback message.  The contents and semantics of such a feedback
   message are entirely defined by the codec, in the same way that
   codecs have free control over the contents of the RTP payload.

2.  Definitions

2.1.  Glossary

   AVPF -  The extended RTP profile for RTCP-based feedback

   CCM  -  Codec Control Message [RFC5104]

   CLF  -  Codec-Layer Feedback

   FCI  -  Feedback Control Information [RFC4585]

   FIR  -  Full Intra Request

   PSFB -  Payload-Specific Feedback [RFC4585]

   SLI  -  Slice Loss Indication

2.2.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   document are to be interpreted as described in [RFC2119].

3.  Use Cases

   We briefly outline here some use-cases for codec-specific feedback
   messages.  In each case, there is no clear existing RTCP message
   which may be used.

   Codec-specific quality control.  Some codecs may wish to allow
      control of the quality level via feedback from the receiver.  Such

Wilson                   Expires March 28, 2019                 [Page 3]

Internet-Draft        Codec-Layer Feedback for RTCP       September 2018

      quality control would not fit well into a generic message, since
      the codec's control parameters (quantization tables, dithering
      level, text quality threshold) may differ widely from the control
      parameters for other codecs.  The ability for the codec to define
      its own PSFB messages, without requiring a codec-specific
      allocation of a new PSFB message, would be helpful.

   Codec-specific picture control.  Some codecs may wish to allow the
      receiver to specify to the sender a preferred color palette to use
      for the transmitted picture, perhaps to ensure the image is usable
      in a constrained viewing environment.  Different codecs however
      encode color data in different ways (sRGB, YUV, YDbDr, and many
      more).  In the case of a codec designed to support palettized
      encodings for constrained viewing environments, it would be
      helpful for the codec to be able to define its own feedback
      messages to provide such a capability.

   Codec-specific decoder control.  There are codecs in which the
      decoder requires access to certain control data in order to begin
      decoding, for example the RTP embedding of H.264 [RFC6184] has
      special considerations in the transmission of Picture Parameter
      Set (PPS) NALUs, with the recommendation that these units be
      exchanged reliably over an out-of-band channel (such as SDP
      exchange).  However, for other codecs, the decoder control data
      may be less easily exchanged over such a channel: either because
      it is larger in size, or changes more frequently, or may be re-
      ordered with respect to the RTP stream (for codecs where the
      decoder does not need to block decoding until the very latest
      control information is received).  In these cases, it may be
      preferable to exchange the decoder-control data over RTCP, so that
      it does not participate in the generic RTP control such as
      retransmissions.  For a codec with specific decoder-control data,
      it may be helpful to enable the codec to make use of RTCP as well
      as RTP messages.

4.  Protocol Overview

   This document extends the RTCP feedback messages defined in the RTP/
   AVPF [RFC4585] by defining an RTCP Codec-Layer Feedback (CLF)
   message.  The RTCP CLF message contains a header identifying the
   codec by means of the payload type identifier, and the remaining
   contents and semantics of the message are defined by the codec.

   A CLF RTCP message may be sent from the sender to the receiver, or
   from the receiver to the sender, depending on the codec.

Wilson                   Expires March 28, 2019                 [Page 4]

Internet-Draft        Codec-Layer Feedback for RTCP       September 2018

5.  Format of RTCP Feedback Message

   As specified by section 6.1 of [RFC4585], Payload-Specific FB
   messages are identified by the RTCP packet type value PSFB (206).

   This memo defines the CLF message, which is identified by RTCP packet
   type values PT=PSFB and FMT=XXX.

   The Feedback Control Information (FCI) for the CLF is of variable-
   length, as determined by the codec which defines the CLF message.  At
   least four octets of FCI must be provided.  The length of the CLF
   feedback message MUST be set to 2+N/4, where N is the length in
   octets of the CLF message.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   |P| Payload Type|    CLF data, defined by the codec...          |
   |                         ...   | Padding (opt) | Padding count |

   Syntax of the Codec-Layer Feedback (CLF) message

   P: 1 bit
      Indicates whether the CLF message is padded.  All CLF messages
      must be a multiple of 4 octets long; if the P bit is set to 1,
      then the final octet contains a padding count of between 1 and 3,
      indicating the number of final padding octets in the message
      (including the padding count).

   Payload Type:  7 bits
      Indicates the RTP payload type in the context of which the native
      CLF data MUST be interpreted.

6.  Semantics of the RTCP Feedback Message

   Within the common packet header for feedback messages (as defined in
   Section 6.1 of [RFC4585]), the "SSRC of packet sender" field
   indicates the source of the feedback message, and the "SSRC of media
   source" indicates the SSRC of the sender end of the stream.

   A CLF message MUST NOT be used for codecs that do not define the
   contents and semantics of the CLF message.

   A CLF message MUST NOT be sent in a direction which is not defined by
   the codec: a CLF message MUST NOT be sent by the sender to the
   receiver unless the codec defines the CLF message in that direction,

Wilson                   Expires March 28, 2019                 [Page 5]

Internet-Draft        Codec-Layer Feedback for RTCP       September 2018

   and a CLF message MUST NOT be sent by the receiver to the sender
   unless the codec defines the CLF message in that direction.

   CLF messages are sent according to timing rules defined by the codec.
   A mixer or translator MAY choose to forward the message, but SHOULD
   ensure that such forwarded messages are not sent at a rate greater
   than the receiver's bandwidth (for example, a mixer which receives
   feedback messages from a unicast peer, and forwards those messages to
   a multicast session may be forced to drop packets); such a mixer
   SHOULD understand the semantics of the CLF message for the payload
   being forwarded, to ensure that the dropped packets do not prevent
   correct operation of the codec.  A mixer SHOULD NOT drop CLF packets
   which it does not understand.  A mixer MAY choose not to forward CLF
   messages at all, by declaring that it does not support CLF messages
   (for example, by using an SDP declaration to state that it does not
   accept CLF messages for the given codec).

   Since mixers may not support CLF messages for any particular codec,
   and may not support CLF messages at all, codecs SHOULD NOT mandate
   that CLF support is required for the correct operation of the codec.

7.  SDP Definition

   Section 4 of [RFC4585] defines a new SDP [RFC4566] attribute, rtcp-
   fb, that may be used to negotiate the capability to handle specific
   AVPF commands and indications.  The ABNF for rtcp-fb is described in
   section 4.2 of [RFC4585].  In this section, we extend the rtcp-fb
   attribute to include the CLF message.

7.1.  Extension of the rtcp-fb attribute

   AVPF specifies that the rtcp-fb attribute must only be used as a
   media level attribute and must not be provided at session level.  All
   the rules described in [RFC4585] for rtcp-fb attribute relating to
   payload type and to multiple rtcp-fb attributes in a session
   description also apply to the new feedback message defined in this

   In [RFC5104], a feedback value "ccm" is defined, which indicates the
   support of codec control using RTCP feedback messages.  The "ccm"
   feedback value is used with a new parameter defined in this memo,
   "clf", to indicate support for Codec-Layer Feedback (CLF).

   In the ABNF for rtcp-fb-ccm-param defined in [RFC5104], there is a
   placeholder token to allow defining new CCM message parameters.
   "clf" is defined as a new CCM parameter in this document.

Wilson                   Expires March 28, 2019                 [Page 6]

Internet-Draft        Codec-Layer Feedback for RTCP       September 2018

   Inclusion of the CLF message type in an SDP description indicates
   support for sending and receiving the CLF message, depending on
   whether the codec defines the messages in each direction.

   The "clf" parameter SHOULD NOT be used with rtcp-fb-pt set to "*", to
   indicate support for the CLF message for all codecs; instead support
   for CLF messages SHOULD be declared for each supported codec

   For example, to indicate support for receiving CLF messages, the
   sender may declare the following in SDP:

   o=alice 12345 12345 IN IP4 host.example.com
   s=Media with CLF feedback
   t=0 0
   c=IN IP4 host.example.com
   m=video 23456 RTP/AVPF 98
   a=rtpmap:98 urn:codec-name/90000
   a=rtcp-fb:98 ccm clf

7.2.  Offer/Answer

   The Offer/Answer [RFC3264] implications for codec layer feedback
   feedback messages are similar to those described in [RFC5104].  The
   offerer MAY indicate the capability to support selected CLF messages.
   The answerer MUST remove all CLF parameters corresponding to the CLF
   message and codec combinations that it does not wish to support in
   this particular media session (for example, because it does not
   implement the CLF message for the codec in question).  The answerer
   MUST NOT add new clf parameters in addition to what has been offered.

   As in [RFC5104], only if CLF is jointly declared for a codec by both
   offer and the answer, may CLF messages be used.

   Including a CLF parameter in an offer or answer indicates that the
   party (offerer or answerer) is at least capable of receiving the
   corresponding CLF message and acting upon it.  It is not mandated to
   send CLF messages of any negotiated type, unless the acknowledgement
   or response is a mandatory part of receiving the CLF message.

8.  IANA Considerations

   A new value "clf" must be registered with IANA in the "Codec Control
   Messages" registry, located at time of publication at:

   The new value is as follows:

Wilson                   Expires March 28, 2019                 [Page 7]

Internet-Draft        Codec-Layer Feedback for RTCP       September 2018

      Value name: clf
      Long name: Codec-Layer Feedback
      Usable with: ccm
      Reference: RFC-realvnc-codec-layer-feedback

   The following value must be registered as a FMT value in the "FMT
   Values for PSFB Payload Types" registry located at the time of
   publication at: http://www.iana.org/assignments/rtp-parameters

   PSFB range

   Name Long Name            Value Reference
   ---- -------------------- ----- ----------------------------------
   CLF  Codec Layer Feedback  XXX  [RFC-realvnc-codec-layer-feedback]

9.  Security Considerations

   The CLF message is defined within the context of the existing
   security model for RTP/RTCP sessions.  As such, it does not enable
   significant changes to the existing security model.

   Since the behaviour of CLF messages is undefined by this
   specification, detailed considerations must be provided by each codec
   specification which defines such a message.

   To guard against malicious injection of CLF data, the CLF messages
   should be treated as untrusted external data, in the same way that an
   RTP payload itself would be.  Authentication and integrity checks may
   be used where appropriate to ensure that third-parties may not inject
   or modify CLF messages, for example using SRTP and the SAVFP profile
   to provide security.

   The CLF message format itself is not believed to carry security risks
   additional to those already incurred by RTP/RTCP peers.

10.  References

10.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,

   [RFC3264]  Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
              with Session Description Protocol (SDP)", RFC 3264,
              DOI 10.17487/RFC3264, June 2002,

Wilson                   Expires March 28, 2019                 [Page 8]

Internet-Draft        Codec-Layer Feedback for RTCP       September 2018

   [RFC4566]  Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
              Description Protocol", RFC 4566, DOI 10.17487/RFC4566,
              July 2006, <https://www.rfc-editor.org/info/rfc4566>.

   [RFC4585]  Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
              "Extended RTP Profile for Real-time Transport Control
              Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
              DOI 10.17487/RFC4585, July 2006,

   [RFC5104]  Wenger, S., Chandra, U., Westerlund, M., and B. Burman,
              "Codec Control Messages in the RTP Audio-Visual Profile
              with Feedback (AVPF)", RFC 5104, DOI 10.17487/RFC5104,
              February 2008, <https://www.rfc-editor.org/info/rfc5104>.

10.2.  Informative References

   [RFC6184]  Wang, Y., Even, R., Kristensen, T., and R. Jesup, "RTP
              Payload Format for H.264 Video", RFC 6184,
              DOI 10.17487/RFC6184, May 2011,

Author's Address

   Nicholas Wilson (editor)
   RealVNC Ltd
   Betjeman House, 104 Hills Road
   Cambridge  CB2 1LQ

   Phone: +44 1223 310411
   Email: ncw@realvnc.com

Wilson                   Expires March 28, 2019                 [Page 9]