Audio/Video Transport Working Group Shigeru Fukunaga Internet Draft Noriyuki Sato Document:draft-fukunaga-low-delay-rtcp-01.txt Oki November 24, 2000 Expires: May 24, 2001 Koichi Yano FastForward Networks Akihiro Miyazaki Koichi Hata Rolf Hakenberg Carsten Burmeister Matsushita Low Delay RTCP Feedback Format Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1. Abstract Low delay feedback in RTP has gained a lot of interest recently in order to enhance the performance in RTP sessions with two or only a small number of participants. A recent document describes rules that enhance the functionality of the existing RTP timing and allow low delay feedback. While that document describes when it is allowed to send data, we describe here what kind of low delay feedback may be sent. Therefore we define a general feedback behavior as well as three general RTCP feedback message types. These message types can either be used as is or they can be extended in payload or application specific ways. Low Delay RTCP Feedback Format November 24, 2000 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [2]. 3. Introduction RTP and its profile define strict rules on the timing behavior of RTCP feedback. It is very important for large multicast groups to define such rules, because the traffic on the feedback channel might explode otherwise. However for small multicast groups or even unicast, a frequently used application of RTP, the rules can be less strict. A new feedback timing behavior for RTP is defined in [8]. The document describes in certain scenarios when it is allowed to send feedback. These new timing rules scale from unicast connections, where feedback is possible anytime immediately to small multicast groups, where feedback can be sent with low delay according to a credit based scheduling scheme. While [8] describes the behavior when the receiver is allowed to send feedback, this document describes a framework what is going to be sent. Section 4 will first describe general items that define the behavior of the RTP receiver, in order to allow frequent low delay feedback. Section 5 describes three general feedback packet types. General formats and semantics are given, but room is left for the extension to specific feedback messages. 4. General Feedback Behavior [3] sets rules for the usage of RTCP packets. A rough summary of the restrictions is given below: The interval between two RTCP packets from the same sender or receiver should be at least five seconds. RTCP control traffic should be limited to a small fraction of the overall session bandwidth. It is suggested that RTCP traffic from receivers should be limited to 3.75% and RTCP traffic from the sources to 1.25% of the session bandwidth. RTCP packets are sent in compound packets The above described rules prevent the efficient usage of RTCP for several purposes that need frequent low delay feedback, e.g. some functionality of predictive video codings. Each of the items are evaluated in the following sections according to their usefulness in Low Delay RTCP Feedback Format November 24, 2000 unicast or small multicast environments. Extensions are defined that allow the efficient usage of RTCP for low delay feedback, while not colliding with the intent of the RTP specification. 4.1 RTCP transmission interval As described above, [3] sets the minimum interval between two consecutive RTCP packets from the same sender or receiver to be at least five seconds. For several applications that could use RTCP for transporting feedback, this is too much delay. Therefore the RTP profile [3] was extended by [8]. [8] defines new delay values according to small multicast groups or unicast sessions. With these extensions it is possible to send immediate or at least low delay feedback. For details see [8]. 4.2 RTCP control traffic bandwidth share [3] defines the total amount of control traffic to be not more than 5% of the total session bandwidth. This share is divided between control traffic from the senders and receivers. The senders should get at least 1/4 of the 5%, which leaves a maximum of 3.75% of the total session bandwidth for receiver originated control traffic. [3] sets the value of 5% as a recommendation. The absolute value could be changed, however it says that the value should be fixed and equal for all participants. For small multicast groups, and also for unicast sessions, this limit should not be exceeded. It is important to control the traffic originated by the receivers. Besides under normal circumstances it is possible to enable efficient feedback without exceeding this threshold. Usually the feedback packets are much smaller than the data packets, which allows a frequent usage of feedback and the receiver still does not need more than the allowed share of the bandwidth. 4.3 RTCP compound packets [3] says that all RTCP data must be sent in compound packets, which consist of: - optional encryption prefix - sender or receiver report - optional additional receiver reports - source description - optional application defined packets - optional BYE packet Low Delay RTCP Feedback Format November 24, 2000 Beside the above listed packets, also the packets that contains the information that is to be transmitted as the reason for the low delay feedback has to be added. The components of the compound packet are investigated according to their relevance an necessity in small multicast and unicast sessions in the following subsections. 4.3.1 Packet compound components 4.3.1.1 Encryption prefix If the packet is to be encrypted this prefix still has its justification and must be used. 4.3.1.2 Sender or Receiver Report [3] justifies the sending of a sender or receiver report in every compound packet by the fact that statistical information should be sent as often as bandwidth constraints allow, to maximize the resolution of the statistics. While this is true for large multicast groups, where the interval between consecutive feedback packets must be large, it is not necessary for the low delay feedback case. In this case feedback is sent much more often and thus sending statistics in every packet is not needed necessarily. The resolution of the statistics should be reasonable. The amount highly depends on the environment. E.g. for unicast sessions, where feedback may be sent for nearly every data packet, complete receiver statistics might only be useful every few seconds. According to [3] feedback from one receiver can be sent at maximum every five seconds. Hence the resolution of the statistics at the sender for one receiver is at maximum once every five seconds. The receiver should send receiver reports not less frequently than it would sent them if the low delay feedback extensions was not used. 4.3.1.3 Source Description SDES components should be sent if the SSRC identifier changed. 4.3.1.4 Application defined packets Application defined packets might be included before the optional BYE packet. Low Delay RTCP Feedback Format November 24, 2000 4.3.1.5 BYE packet If a bye packet is to be sent, it still must be the last packet of the compound. 4.3.1.6 Low delay feedback packet The packets that build this part of the compound packet are described with their general syntax and semantics in section 5. 4.3.2 Low delay feedback compound packet Feedback packets should be sent in compound packets to allow the receiver of the feedback packet to easily parse the packet and verify it. Feedback, even low delay feedback, is not expected to be very frequently, hence the possibility to send a compound packet is given and should be used, while the bandwidth requirements can be kept. 5 Low Delay Feedback Packets In this document we define a framework of low delay RTCP feedback messages. Therefore three different types of messages are defined, which are categorized as follows: - Transport Layer Feedback Messages - Payload Specific Feedback Messages - Application Layer Feedback Messages Transport Layer Feedback Messages are used to transmit protocol related information from the receiver to the sender. This information could be a general acknowledgement (ACK) or negative- acknowledgement (NACK). It could be thought of other types of Transport Layer Feedback Messages, which can be registered later then. Payload Specific Feedback Messages transport information that is specific to certain payloads. The format and type of information is defined either directly within an RTP Payload Format or in additional feedback format documents. Application Layer Feedback Messages are used to transport application specific feedback directly from the receiver's application to the sender's application. These data that is sent between the applications is usually defined in the application's specification, e.g. NEWPRED messages are defined in MPEG-4 Low Delay RTCP Feedback Format November 24, 2000 specification. The data can be identified by the application and therefore needs no additional external information. The general syntax and semantics for the three packet types is described in the following subsections. 5.1 Transport Layer Feedback Messages This feedback messages are used to inform the sender's RTP protocol entity about certain events that the receiver detected. These events could be e.g. the reception of a packet or the detection of a packet loss. The generic format of the packet is as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|E| FMT | PT=RTPFB | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of packet sender | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : FCI : : : The various fields V, P, SSRC and length are defined in the RTP specification [3]. We summarize the meaning of all of the fields below: version (V): 2 bits This field identifies the RTP version. The current version is 2. padding (P): 1 bit If set, the padding bit indicates that the packet contains additional padding octets at the end which are not part of the control information but are included in the length field. early packet (E): 1 bit This bit is set if the packet is sent early. A detailed description can be found in [8]. Low Delay RTCP Feedback Format November 24, 2000 feedback message type (FMT): 4 bits This field identifies the type of the feedback. The packet can be used for different purposes. The following types are defined (up to now): 0: forbidden 1: General NACK 2: General ACK 3-15: reserved packet type (PT): 8 bits This is the RTCP packet type which identifies the packet as being an RTP Feedback Message. length: 16 bits The length of this packet in 32-bit words minus one, including the header and any padding. This conforms with the definition of the length field used in RTCP sender and receiver reports [3]. SSRC of packet sender: 16 bits The synchronization source identifier for the originator of this packet. SSRC of media source: 16 bits The synchronization source identifier of the media source that this feedback is related to. feedback control information (FCI): variable length Format and semantic of the FCI varies for the different message types. See sections 5.1.1 and 5.1.2 for the definition of the syntax and the semantic of this field for the general NACK and ACK packets. 5.1.1 General NACK In this section the general NACK packet is described. As said above, the general NACK packet is of the type Transport Layer Feedback Message. The basic header of section 5.1 applies for this packet, where FMT is set to 1 (General NACK). The general NACK packet is used to indicate the loss of one or several RTP packets. Therefore the lost packet(s) are identified by the means of a packet identifier and a bitmask. Low Delay RTCP Feedback Format November 24, 2000 The Feedback control information (FCI) field has the following syntax: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PID | BLP | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Packet ID (PID): 16 bits The PID field is used to specify a lost packet. RTP Payload Formats may decide how to identify a packet. Typically RTP sequence number is used for PID as the default format. bitmask of following lost packets (BLP): 15 bits The BLP allows for reporting losses of any of the 15 RTP packets immediately following the RTP packet indicated by the PID. The BLP's definition is identical to that given in [RFC2032]. Denoting the BLP's least significant bit as bit 1, and its most significant bit as bit 15, then bit i of the bitmask is set to 1 if the sender has not received RTP packet number PID+i (modulo 2^16) and the receiver decides this packet is lost; bit i is set to 0 otherwise. Note that the sender MUST NOT assume that a receiver has received a packet because its bitmask was set to 0. For example, the least significant bit of the BLP would be set to 1 if the packet corresponding to the PID and the following packet have been lost. However, the sender cannot infer that packets PID+2 through PID+15 have been received simply because bits 2 through 15 of the BLP are 0; all the sender knows is that the receiver has not reported them as lost at this time. 5.1.2 General ACK The general ACK packet is also of the type Transport Layer Feedback Message and therefore uses the header as defined in section 5.1 with FMT=2 (general ACK). The general ACK packet is used to indicate that one or several RTP packets are received correctly. Therefore the received packet(s) are identified by the means of a packet identifier and a bitmask. Acking of a range of consecutive packets is also possible. Low Delay RTCP Feedback Format November 24, 2000 The Feedback control information (FCI) field has the following syntax: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 1st PID |R| BLP/PID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Packet ID (1st PID): 16 bits This PID field is used to specify a correctly received packet. RTP Payload Formats may decide how to identify a packet. Typically RTP sequence number is used for PID as the default format. Range of ACKs (R): 1 bit The R-bit indicates that a range of consecutive packets are received correctly. The 1st PID field specifies the first packet of that range and the next field (BLP/PID) will carry the last PID of the range that is acknowledged. bitmask of following lost packets / Packet ID (BLP/PID): 15 bits The semantic of this field changes according to the value of the R- field. If R=1, this field is used to identify the last packet of the acknowledged packet range, as described above. If R=0, this field carries a bitmask. The BLP allows for reporting reception of any of the 15 RTP packets immediately following the RTP packet indicated by the PID. The BLP's definition is identical to that given in [RFC2032]. Denoting the BLP's least significant bit as bit 1, and its most significant bit as bit 15, then bit i of the bitmask is set to 1 if the sender has received RTP packet number PID+i (modulo 2^16) and the receiver decides to ACK this packet; bit i is set to 0 otherwise. 5.2 Payload Specific Feedback Messages Packets of the type Payload Specific Feedback Messages are used to transport information from the receiver to the sender that is directly related to the payload of the forward stream. Messages of this type include e.g. Picture or Slice Loss Indications. Low Delay RTCP Feedback Format November 24, 2000 The basic header format is described below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|E| FMT | PT=PSFB | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of packet sender | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |-| RTP PT | : +-+-+-+-+-+-+-+-+ FCI : : : The various fields V, P, SSRC and length are defined in the RTP specification [3]. We summarize the meaning of all of the fields below: version (V): 2 bits This field identifies the RTP version. The current version is 2. padding (P): 1 bit If set, the padding bit indicates that the packet contains additional padding octets at the end which are not part of the control information but are included in the length field. early packet (E): 1 bit This bit is set if the packet is sent early. A detailed description can be found in [8]. feedback message type (FMT): 4 bits This field identifies the type of the feedback. Every RTP Payload Format has its own FMT number space, i.e. a certain FMT number can have different meanings for different payload types. It is therefore expected that either the RTP Payload Format documents itself or additional feedback format documents (related to one or several Payload Formats) define the FMT numbers and their meanings. FMT=0 is forbidden. packet type (PT): 8 bits This is the RTCP packet type which identifies the packet as being an Payload Specific Feedback Message. length: 16 bits The length of this packet in 32-bit words minus one, including the header and any padding. This conforms with the definition of the length field used in RTCP sender and receiver reports [3]. SSRC of packet sender: 16 bits Low Delay RTCP Feedback Format November 24, 2000 The synchronization source identifier for the originator of this packet. SSRC of media source: 16 bits The synchronization source identifier of the media source that this feedback is related to. RTP payload type (RTP PT): 7 bits This field indicates the payload format to which the feedback is related. It is important to use that field, because the session could be used with several payload formats on the forward direction and the semantics of the packet is different for different payload formats. feedback control information (FCI): variable length Format and semantic of the FCI varies for the different message types. The type of FCI that is used is indicated by the combination of FMT and RTP PT. The mapping of FMT numbers to FCIs is expected to be described in additional payload specific documents or the RTP Payload Formats. 5.2.1 Example This section describes a short example how the payload specific feedback messages can be defined and used. As an example we consider a predictive coded video application. If packet loss occurs, it is important that the sender gets this information, because it will otherwise send data, which the receiver might not be able to use. Therefore an additional document can describe a feedback message, which works as a picture or slice loss indication. If the sender receives this message, it might start with a new intra coded frame, which the receiver would be able to use properly. The document must specify the format of the FCI-field, one or several RTP payload formats, for which this should be used and a FMT number that is not allocated in all of these formats. A different possibility is to define the FMT number and FCI syntax directly in a video RTP Payload Format. 5.3 Application Layer Feedback Messages These messages are used to transport application defined data directly from the receiver's to the sender's application. The data that is transported is not identified by the feedback message. Low Delay RTCP Feedback Format November 24, 2000 Therefore the application must be able to identify the messages payload. Usually applications define their own set of messages, e.g. NEWPRED messages in MPEG-4 [9] or feedback messages in H.263/Annex N,U [10]. These messages do not need any additional information from the RTCP message and thus this message has the following simple format: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|E| FMT | PT=AFB | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of packet sender | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ : Application Message : : : The various fields V, P, SSRC and length are defined in the RTP specification [3]. We summarize the meaning of all of the fields below: version (V): 2 bits This field identifies the RTP version. The current version is 2. early packet (E): 1 bit This bit is set if the packet is sent early. A detailed description can be found in [8]. padding (P): 1 bit If set, the padding bit indicates that the packet contains additional padding octets at the end which are not part of the control information but are included in the length field. The last octet of the padding is a count of how many padding octets should be ignored. feedback message type(FMT): 4 bits This field identifies the type of the feedback. The packet can be used for different purposes. No type is defined up to now: 0: forbidden 1-15: reserved packet type (PT): 8 bits This is the RTCP packet type which identifies the packet as being an Application Feedback Message. Low Delay RTCP Feedback Format November 24, 2000 length: 16 bits The length of this packet in 32-bit words minus one, including the header and any padding. This conforms with the definition of the length field used in RTCP sender and receiver reports [3]. SSRC of packet sender: 16 bits The synchronization source identifier for the originator of this packet. SSRC of media source: 16 bits The synchronization source identifier of the media source that this feedback is related to. Application Message (FCI): variable length This field contains the original application message that should be transported from the receiver to the source. The format is application dependent. The length of this field is variable. If the application data is not four-byte aligned, padding must be added. 5.3.1 Example An example usage of the Application Feedback Message is NEWPRED for MPEG-4 [9]. The NEWPRED messages are defined in the MPEG-4 specification. The MPEG-4 entity at the receiver sends the message, which is transported in the application feedback packet. It is then given to the MPEG-4 entity at the sender, which can identify the information as a NEWPRED message and knows what to do with it. Normally one NEWPRED message is mapped onto one AFM-RTCP packet, and several messages could be continuously mapped onto one AFM-RTCP packet. In the case several messages are mapped onto one AFM-RTCP packet, padding should only be required on the last individual message. One message SHALL NOT be fragmented into different AFM-RTCP packets. 6. Security Considerations Security is not considered in this draft. 7. References 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. 2 Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 Low Delay RTCP Feedback Format November 24, 2000 3 Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson, "RTP: A Transport Protocol for real-time applications", RFC 1889, January 1996. 4 Postel, J.,"User Datagram Protocol", STD 6, RFC 768, August 1980. 5 Postel, J., "Internet Protocol", STD 5, RFC 791, September 1981. 6 Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", RFC 2460, December 1998. 7 Yano, K., Podolsky, M., McCanne, S., "RTP Profile for RTCP-based Retransmission Request for Unicast session", IETF Internet Draft, work in progress, July 2000. 8 Wenger, S., Ott, J., "RTCP-based Feedback for Predictive Video Coding", IETF Internet Draft, work in progress, July 2000. 9 ISO/IEC 14496-2:1999/Amd.1:2000, "Information technology - Coding of audio-visual objects - Part2: Visual", July 2000. 10 ITU-T Recommendation H.263, "Video Coding for Low Bit Rate Communication," November 2000. 8. Author's Addresses Shigeru Fukunaga Oki Electric Industry Co., Ltd. 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan Tel. +81 6 6949 5101 Fax. +81 6 6949 5108 Email: fukunaga444@oki.co.jp Noriyuki Sato Oki Electric Industry Co., Ltd. 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan Tel. +81 6 6949 5101 Fax. +81 6 6949 5108 Email: sato652@oki.co.jp Koichi Yano FastForward Networks, 75 Hawthorne St. #601 San Francisco, CA 94105 415.430.2500 Low Delay RTCP Feedback Format November 24, 2000 Akihiro Miyazaki Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Tel. +81-6-6900-9192 Fax. +81-6-6900-9193 Mail akihiro@isl.mei.co.jp Koichi Hata Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Tel. +81-6-6900-9192 Fax. +81-6-6900-9193 Mail fukusima@isl.mei.co.jp Rolf Hakenberg Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-162 Fax. +49-(0)6103-766-166 Mail hakenberg@panasonic.de Carsten Burmeister Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-263 Fax. +49-(0)6103-766-166 Mail burmeister@panasonic.de Full Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.