Audio/Video Transport Working Group A.Miyazaki Internet Draft H.Fukushima draft-ietf-avt-rtp-selret-03.txt K.Hata Expires: May 2002 T.Wiebke R.Hakenberg C.Burmeister Matsushita N.Takatori S.Okumura T.Ohno Mitsubishi RTP Payload Formats to Enable Multiple Selective Retransmissions Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document describes new RTP payload formats to enable multiple and optional selective retransmissions in RTP. These are especially applicable to environments where enhanced RTCP feedback is available, such as described in [4]. These payload formats can be used to separate the media stream according to prioritization of packets or according to the status of the transmission (i.e. transmission or retransmission). The formats serve as a retransmission framework, where we give examples how these can be used for different retransmission schemes. Miyazaki et al. Expires May 2002 1 Multiple Selective RTP Retransmissions November 2001 Changes from draft-ietf-avt-rtp-selret-02.txt The main changes since the previous version of the draft were a new section about how to indicate the payload format usage in SDP, many clarifications regarding how to use the headers of the payload formats, especially regarding combination of headers and two new examples retransmission schemes including examples how to describe those with SDP. The following list describes the detailed changes: All sections: - editorial changes Section 3, Requirements for Retransmissions in RTP - new section, highlighting identified requirements Section 4, General Use of the Payload Formats - clarification about having several headers in one packet - list of possible header combinations - general rules of how to use the payload formats Section 5, Retransmission Payload Format - changes in the header format to include the possibility to signal the complete original sequence number or the least significant bits of the delta sequence number Section 6, Selective Payload Format - changes in the header format to include the possibility to signal the complete referenced sequence number Section 9, Relation to SDP - new section, describing how to use SDP to indicate the usage of the payload formats Section 10, Example Usages of the Payload Formats - adding SDP descriptions to all examples - two new example usages: - retransmissions on a different stream - retransmissions in layered transmissions Miyazaki et al. Expires May 2002 2 Multiple Selective RTP Retransmissions November 2001 Table of Contents Status of this Memo Abstract Changes from draft-ietf-avt-rtp-selret-02.txt 1. Conventions used in this document 2. Introduction to the problem 3. Requirements for Retransmissions in RTP 4. General Usage of the Payload Formats 5. Retransmission Payload Format 6. Selective Payload Format 7. Feedback 8. Congestion Control 9. Relation to SDP 9.1 Indicating RTX Usage in SDP 9.2 Indicating SEL Usage in SDP 9.3 Indicating RTX and SEL Usage in SDP 10. Example Usages of the Payload Formats 10.1 A Multiple Retransmission Algorithm 10.2 A Selective Retransmission Algorithm 10.3 Sending Retransmissions on a Different Stream 10.4 Using Retransmissions in Layered Transmissions Security Considerations Acknowledgements Intellectual property considerations References AuthorÆs Addresses Miyazaki et al. Expires May 2002 3 Multiple Selective RTP Retransmissions November 2001 1. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [1]. 2. Introduction to the problem The Real-time Transport Protocol (RTP)[3] was designed as an Internet protocol to transmit real-time (or near real-time) data over large multicast. Even though the multicast capability is a strong feature of RTP, a common use of it is non-interactive unicast or small multicast transmission of media streams. These applications, further referred to as media streaming, has lower delay requirements. A constant offset delay, in which already received packets are stored at the client, is tolerable and used for different purposes, such as delay jitter compensation. Typically RTP is run over the User Datagram Protocol (UDP). While the unreliable UDP performs good for reliable channels, the quality of the received media stream is bad, when transmitted over error- prone links. Especially compressed streams, such as MPEG video-data, are highly susceptible to packet loss, which will lead to a bad video quality at the clients. E.g. in a wireless environment with an unstable transmission quality (bit error rate (BER) about 10E-3 .. 10E-6) a sufficient quality of video and voice transmitted by RTP cannot be achieved in most of the cases. The scenario for such a media streaming over an error-prone link might be the one in Figure 1. Media files are stored on a media server which is connected via an internet link to a mobile network. The wireless link between the mobile network and the mobile terminal, which works as the client in this scenario, is generally unreliable and subject to bit errors. Media Mobile Mobile Server Network Terminal (Client) *** | +----+ ** ** +--+ | | * * | | | | -------------- * * ~ ~ ~ ~ ~ ~ ~ | | +----+ * * +--+ ** ** Internet *** Wireless Link Link Figure 1 : Scenario for media streaming Miyazaki et al. Expires May 2002 4 Multiple Selective RTP Retransmissions November 2001 To achieve a better quality of the received media stream, especially when transmitted over error-prone links, retransmissions of lost packets of the stream seem to be a possible solution. 3. Requirements for Retransmissions in RTP There are several issues that have to be considered for retransmissions in RTP. We describe in this section some of those, which lead to a set of requirements for the retransmission scheme. As the RTP sender SHOULD employ a congestion control algorithm if the session is transmitted over an best effort environment (see [3] and [5]), the sending rate is typically limited. Lost packets might also decrease the allowed sending rate, if these are interpreted as a congestion signal. What is more the bandwidth of a mobile channel is a very scarce resource and has to be used with great care. Thus RTP senders ( especially if transmitting data over wireless channels) are often confronted with strictly limited available bit rates. Hence it is in most cases not useful or even not allowed to send retransmissions of all lost packets additionally to the original stream (note however that there are scenarios of Quality of Service (QoS) controlled networks, where it can be possible to reserve enough bandwidth in advance to transmit retransmission of all lost packets in addition to the original stream). The media data is typically coded for compression before transmission. The reduction of redundancy leads to the streams being more susceptible to errors. Another property of encoded media streams is that they typically consist of data of different importance. Some parts of the data might be absolutely vital for displaying the media, while other parts may only enhance the quality of the displayed stream. A common mechanism to achieve different QoS for different parts of the data is to split the data in layers and transmit these layers in different RTP sessions. The layers are typically hierarchical ordered, which means that one layer (i.e. the base layer) is independently decodable. The next layer (i.e. the first enhancement layer) enhances the quality of the received stream, but depends on the base layer to be decoded. The next layer depends on the first enhancement layer etc. As the layers are transmitted in different sessions, the client can choose how many layers it would like to receive, according to available bandwidth or other criterions. While this mechanism is general, scalable and backwards compatible, it introduces some additional overhead and complexity in the server. So having other means to differentiate the services for the data within the stream can be beneficial. However, as this layered Miyazaki et al. Expires May 2002 5 Multiple Selective RTP Retransmissions November 2001 encoding and transmission is an accepted technique, especially for multicast transmissions, to achieve different QoS levels within one media stream or to perform congestion control in multicast, it has to be ensured that the retransmission scheme is able to support this. Thereby it should be possible to do retransmissions of each layer separately. In order to keep the service differentiation between the layers, it should be possible to differentiate between retransmissions of different streams. Because of the strictly limited bandwidths, which a server might be confronted with (e.g. in mobile networks or other best effort environments), it has to be taken care that only important data is retransmitted. It might be possible to skip the transmission of non- important data at all, for the benefit of transmitting important data twice. With a strategy of this kind, the bandwidth requirements may be kept, while the quality of the received stream can be increased. Therefore a retransmission scheme that retransmits all or selected packets and attempts to retransmit these packets one or several times without introducing avoidable delay is needed. What is more layered encoding have to be supported. We see already that there is the need for retransmission schemes, with sophisticated functionalities. It might not be possible to design one scheme that fits all requirements in an optimum way. Therefore we define in this document two new RTP payload formats. With these formats it is possible to design retransmission schemes (where the intelligence is at the server or at the client side) that fulfill the requirements written above. We will give examples of how to use the payload formats to enable different retransmission schemes at the end of this document. 4. Usage of the Payload Formats The payload formats presented in this document are meant as a general solution to enable retransmissions in RTP. Based on this scheme we present examples how the payload formats could be used to realize different retransmission schemes. Thereby the focus may lay on bandwidth efficiency within the media stream or on the back channel or the need for supporting layered encoding. The payload formats are not meant to replace a media RTP payload format, but are used additionally. This means that an RTP packet using one or both of the formats defined below, can have several encapsulated headers, where the outer header always indicates the payload type of the next inner header (see examples at the end of this section). Miyazaki et al. Expires May 2002 6 Multiple Selective RTP Retransmissions November 2001 4.1 General Usage Rules The sender of RTP packets, using the retransmission capabilities has different possibilities at transmitting RTP packets. For packets that are transmitted for the first time, the sender MAY use the Selective (SEL) payload format. If so, it has the possibility to signal the packet's priority and the sequence number of the last high priority packet in-band. This can be used at the receiver of the packets to detect if the lost packet was high or low priority. If the SEL payload format is specified for an RTP session, e.g. a payload type number is dynamically assigned, the client SHOULD NOT send a retransmission request, if it is sure it has only lost low priority packet. However the client MAY send a retransmission request for several packets, including low priority packets (using the BLP field of the generic NACK packet, see section 7), as long as there is either at least one high priority packet lost or the client cannot judge if there might be a high priority packet lost within the lost packets. If the RTX payload format is specified for an RTP session, e.g. a payload type number is dynamically assigned, all retransmissions SHOULD be sent using the RTX payload type. Thereby the header encapsulation scheme, which is described in section 4.2 SHOULD be used. 4.2 Header Usage and Encapsulation For the encapsulation of retransmissions in the RTX packet the following rule applies: 1. The sender of the retransmission SHOULD strip off the RTP header of the original RTP packet, that is to be retransmitted, but store the values of the header fields temporarily for use in this packet's header fields (see step 2 and 4). It is important to note that only the RTP header is stripped off, payload format specific headers are kept as they are. Hence especially RTX and SEL specific headers SHOULD NOT be stripped off, but SHOULD be transmitted together with the headers that are generated in the next steps. 2. An RTX header is generated and placed before the packet. Therefore the difference of the RTP sequence number used for this RTP packet to the sequence number used for the original transmission MAY be computed and the least significant bits MAY be inserted in the header. If this is not done (e.g. because the sequence number difference is too high), the complete sequence number of the original RTP transmissions MUST be included in the header. 3. A SEL header MAY be appended, if the sender wishes to signal the priority of the packet in-band. Miyazaki et al. Expires May 2002 7 Multiple Selective RTP Retransmissions November 2001 4. The RTP header is generated. Thereby the fields of the header are filled as usual, e.g. as described in the payload format that is used for the media to be transported. However the RTP sequence number MUST be increased by one to the previous RTP packet and the timestamp and marker bit MUST be set to the values of the original transmission's RTP header. 4.3 Header Examples Following we show qualitative some example RTP packets using the RTX and/or SEL payload format: 1. RTP packet, using the SEL format to signal the priority of the packet in-band: +---------------------------------------------+ | RTP header with PT=SEL | | | +---------------------------------------------+ | SEL Payload Type specific header | | with PT=media_pt | +---------------------------------------------+ : Media Payload Type specific header : : : +---------------------------------------------+ | | | Media Payload | : : 2. Retransmission of an RTP packet, using the RTX format for the retransmission: +---------------------------------------------+ | RTP header with PT=RTX | | | +---------------------------------------------+ | RTX Payload Type specific header | | with PT=media_pt | +---------------------------------------------+ : Media Payload Type specific header : : : +---------------------------------------------+ | | | Media Payload | : : Miyazaki et al. Expires May 2002 8 Multiple Selective RTP Retransmissions November 2001 3. Retransmission of an RTP packet, using the SEL format to signal the priority of the packet in-band and the RTX format for the retransmission: +---------------------------------------------+ | RTP header with PT=RTX | | | +---------------------------------------------+ | RTX Payload Type specific header | | with PT=SEL | +---------------------------------------------+ | SEL Payload Type specific header | | with PT=media_pt | +---------------------------------------------+ : Media Payload Type specific header : : : +---------------------------------------------+ | | | Media Payload | : : 4. Retransmission of a retransmitted RTP packet, using the RTX format twice, conform to the rules given above: +---------------------------------------------+ | RTP header with PT=RTX | | | +---------------------------------------------+ | RTX Payload Type specific header | | with PT=RTX | +---------------------------------------------+ | RTX Payload Type specific header | | with PT=media_pt | +---------------------------------------------+ : Media Payload Type specific header : : : +---------------------------------------------+ | | | Media Payload | : : The previous examples show that there might be quite some headers involved in sending a single RTP packet. There are some points to note with this. First the header encapsulation is straight forward regarding server side implementation of retransmissions and buffering of transmitted packets. Second the overhead because of the retransmission headers is quite small. Usually only one RTX header is used for retransmissions and thus the additional overhead is two or four bytes. Retransmitted retransmissions have two RTX headers, Miyazaki et al. Expires May 2002 9 Multiple Selective RTP Retransmissions November 2001 which sums up to 4 to 8 bytes of additional overhead. Multiple retransmissions can happen and enhance the performance, but still these are quite rare cases. Considering that RTP is designed for real-time or near-real-time services, several retransmission attempts will soon run out of time. 5. Retransmission Payload Format Retransmissions in RTP SHOULD be sent with the RTX payload format. This enables the receiver to reconstruct exactly the original RTP header and all its containing information. Additionally multiple retransmissions are enabled. The syntax of the payload type specific header is as follows: 0 1 2 3 4 5 6 7 +---+---+---+---+---+---+---+---+ | X | PT of original packet | +---+---+---+---+---+---+---+---+ |LSB (Delta) RTP Sequence Number| +---+---+---+---+---+---+---+---+ : original RTP Sequence Number : if X=1 +...............................+ Extended Sequence Number (X): 1 bit This bit indicates whether a Delta RTP Sequence Number is used or the complete original RTP sequence number. For X=0 the least significant 8 bits of the Delta RTP Sequence Number are transmitted, for X=1 the original RTP sequence number is inserted in the header. Payload Type of original packet: 7 bit This field contains the original payload type information, i.e. the payload type of the payload that is transported in this packet. This might be the payload type of the media itself or of another encapsulating payload type, e.g. the SEL payload type. LSB (Delta) RTP Sequence Number: 8 bit The value of this field is dependent on the setting of the X-bit (see above). If the X-bit is set to zero, this field contains the eight least significant bits of the difference of this packet's RTP sequence number to the RTP sequence number of the original transmission of this packet's payload, calculated modulo 2^32, i.e. Delta SN = (SN(this) û SN(original)) mod 2^32. If the number of bits that are needed to encode the difference exceeds 8, X=1 MUST be set and the complete original RTP sequence number is transmitted in the following two bytes, i.e. covering this field and the following MSB RTP Sequence Number field. Miyazaki et al. Expires May 2002 10 Multiple Selective RTP Retransmissions November 2001 MSB RTP Sequence Number: 8 bit For X=1 this field is used together with the previous eight bits to carry the complete original RTP sequence number (see also "(LSB (Delta) RTP Sequence Number"). 6. Selective Payload Format In certain scenarios, e.g. environments with a very low bit rate feedback channel, it is necessary to limit the feedback that is sent. Therefore some intelligence has to be placed in the receiver, in order to enable it to detect the if a retransmission request should be sent for a lost packet or not. This can be done by the means of this selective (SEL) payload type. If packets of a low priority are lost, the receiver does not need to send feedback. Still these packets are needed to detect the loss of high priority packets, and therefore they contain the sequence number of the last important packet that was sent. If the receiver gets a low priority packet, it has only to verify if it has received the indicated last important packet. Only for these packets retransmission requests should be sent. The syntax of the packet is as follows: 0 1 2 3 4 5 6 7 +---+---+---+---+---+---+---+---+ |DTI| PT of original packet | +---+---+---+---+---+---+---+---+ |PRI| X | (Delta) SN | +---+---+---+---+---+---+---+---+ : : + complete SN + if X==1 : : +...............................+ : : + Delta Time + if DTI==1 : : +...............................+ DTI (Delta Time Indicator): 1 bit DTI=1 indicates that the delta time field is present. DTI=0 means that the Delta Time field is not used. Payload Type of original packet: 7 bit This field contains the original payload type information, i.e. the payload type of the payload that is transported in this packet. PRI (Priority): 1 bit This field indicates the priority of the packet. PRI=1 means a high priority packet, where PRI=0 is sent for low priority packets. Miyazaki et al. Expires May 2002 11 Multiple Selective RTP Retransmissions November 2001 X (Extended RTP Sequence Number): 1 bit This field indicates whether a short "Delta SN" is used to indicate the last high priority packet (X=0) or the complete RTP sequence number of the last high priority packet is transmitted (X=1). Delta SN (Delta Sequence Number): 6 bit This filed is only present in case of X=0. In this case it contains the offset in sequence number space to the previous important packet. It is calculated as follows: DSN = SN_this - SN_important. If X=1 this field MUST be ignored. Complete SN (complete referenced RTP Sequence Number): 16 bit This filed is present if X=1. In this case it contains the complete RTP sequence number of the last high priority packet. Delta Time: 15 bit This field indicates the difference between the time stamp of the last important RTP packet (that should be retransmitted if lost) and that of the current RTP packet. It can be used by the receiver to compute the timestamp of a lost important packet. The default unit of this field is [ms] and by default the following table is used to encode the Delta Time values. Delta Time [ms] | bits -----------------+-------------------- more than 32766 | 0111 1111 1111 1111 32766 | 0111 1111 1111 1110 32765 | 0111 1111 1111 1101 : | : 2 | 0000 0000 0000 0010 1 | 0000 0000 0000 0001 0 | 0000 0000 0000 0000 -1 | 1111 1111 1111 1111 -2 | 1111 1111 1111 1110 : | : -32766 | 1000 0000 0000 0010 -32767 | 1000 0000 0000 0001 less than -32767 | 1000 0000 0000 0000 It is possible to define other units or coding tables, however, how to specify and negotiate this is outside the scope of this document. 7. Feedback Feedback is an important part of any retransmission scheme. To be efficient, also under near real-time conditions, it might be needed to receive this feedback with the lowest delay that is possible. The basic RTP sets constraints about feedback timing rules, which might Miyazaki et al. Expires May 2002 12 Multiple Selective RTP Retransmissions November 2001 not be suitable for a retransmission scheme under real-time or near real-time conditions. To enable low delay feedback a new extended RTP profile for RTCP feedback was defined, which redefines these timing rules according to the group size of the multicast session (see [4] for details). In unicast sessions it is possible to send feedback virtually immediately. Hence this profile provides the means to do low delay feedback which is needed for the retransmission schemes. In the Low Delay Feedback profile RTCP feedback messages are defined, which are suited for retransmission requests or acknowledgements. Generally a simple feedback message, called general NACK is suitable to be used as a retransmission request. It is defined in [4] as follows: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|E| FMT=1 | PT=RTPFB | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of packet sender | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PID | BLP | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The PID is used to indicate a lost packet for which the retransmission is requested and the BLP might be used to indicate several lost packets in one NACK message. A detailed description of this and other low delay feedback messages can be found in [4]. 8. Congestion Control In general retransmission schemes contradict congestion control. If packets are lost, which is generally accepted to be an indication of congestion, the sender increases the network load by sending additional retransmissions. While this behavior cannot be accepted in best effort environments such as the Internet, it is reasonable in quality of service (QoS) controlled networks, e.g by reserving a certain bit rate for the RTP session. However the used bit rate MUST be monitored to control that the maximum rate is by no means exceeded. Even though retransmission algorithms normally contradict congestion control they could enhance the quality of service, in environments where losses occur not only due to congestion but also (or mainly) due to bit errors on the link. Miyazaki et al. Expires May 2002 13 Multiple Selective RTP Retransmissions November 2001 Taking this information into account the following rule MUST be followed, if the retransmissions are used in an best effort environment (e.g. the Internet). At reception of a retransmission request or detecting packet loss by other means, the sender MUST NOT increase the sending bit rate on mid term timescales. Retransmissions MAY be sent within the current bit rate, e.g. by skipping or delaying the transmission of other scheduled packets. The rules for congestion control of [3], [4] and [5] MUST be followed. 9. Relation to SDP The payload formats, described in this document can be indicated using SDP. This section specifies the usage of SDP in relation to the RTX and SEL payload format. 9.1 Indicating RTX Usage in SDP RTX packets are retransmissions of RTP packets. The original packets have assigned a payload type value, dynamically or statically. There is no static payload type assignment for RTX, so dynamic payload type numbers MUST be used. The binding to the number is indicated by an rtpmap attribute. The name used in this binding is "rtx". The payload type number is indicated in the "m" line of the media as an (additional) payload type. The retransmissions, using the RTX payload type, can be sent to either the same transport address and port in the same RTP session, or to a different port or address in a different RTP session. In the former case the streams are separated using the payload type field, in the latter by the different addresses. It is important to note, that the RTX packets and original packets would not share the same sequence number space in case of different RTP sessions. We define an optional fmtp attribute for usage in SDP to indicate the maximum time a server will try to retransmit a lost packet. It is defined as follows: a=fmtp:<number> <rtx-time> <number> indicates the assigned payload type number (indicated in the "m" line). <rtx-time> indicates the time in [ms], measured from the first sending of a packet until the server will not try to retransmit the packet again. This is a maximum time, the client can decrease this (e.g. because of limited memory resources), by not sending retransmission requests more early. Miyazaki et al. Expires May 2002 14 Multiple Selective RTP Retransmissions November 2001 No fmtp attribute present for a retransmission stream means that the maximum retransmission time is not defined, but MAY be negotiated by other means. A sample indication of RTX in SDP, where the retransmissions are sent in the same RTP session as the original data and thus share one sequence number space, is as follows: m = video 8000 RTP/AVPF 98 99 a = rtpmap:98 MP4V-ES/90000 a = rtpmap:99 rtx/90000 An additional line a = fmtp:99 3500 will indicate that the server will try to retransmit packets only 3.5 seconds, after which it will not try again. If the retransmissions should not share the same sequence number space and thus should be sent in a different session and on a different transport address or port, the SDP description could loook like: m = video 8000 RTP/AVPF 98 a = rtpmap:98 MP4V-ES/90000 m = video 8002 RTP/AVPF 99 a = rtpmap:99 rtx/90000 An additional line a = fmtp:99 3500 will indicate that the server will try to retransmit packets only 3.5 seconds, after which it will not try again. 9.2 Indicating SEL Usage in SDP SEL packets are RTP packets with an additional SEL specific header, which enables a priority detection (also of lost packets). It is expected that the SEL packets are typically sent either instead of all other media packets, or in combination with other media packets. E.g. all media packets might be encapsulated in SEL packets, or only one importance level is encapsulated, where the other packets are sent with their original payload type. There is no static payload type assignment for SEL, so dynamic payload type numbers MUST be used. The binding to the number is indicated by an rtpmap attribute. The name used in this binding is Miyazaki et al. Expires May 2002 15 Multiple Selective RTP Retransmissions November 2001 "sel". The payload type number is indicated in the "m" line of the media as an additional payload type to the original one. The priority indicating packets, using the SEL payload type, are sent to the same transport address and port as the media stream is sent or would have been sent. A sample indication of SEL in SDP is as follows: m = video 8000 RTP/AVPF 98 99 a = rtpmap:98 sel/90000 a = rtpmap:99 MP4V-ES/90000 9.3 Indicating RTX and SEL Usage in SDP Both payload formats, SEL and RTX, MAY be used together. The following is an example of how to indicate the usage of both formats: m = video 8000 RTP/AVPF 98 99 100 c = IN IP4 192.129.35.13 a = rtpmap:98 sel/90000 a = rtpmap:99 MP4V-ES/90000 a = rtpmap:100 rtx/90000 An additional line a = fmtp:100 3500 will indicate that the server will try to retransmit packets only 3.5 seconds, after which it will not try again. 10. Example Usages of the Payload Formats In section 3 we investigated the requirements for a retransmission scheme to be used for RTP. We saw that different applications and environments have different requirements on a retransmission scheme, which are not necessarily compatible. Thus there is a need for different retransmission schemes, depending on the environment and application that it should be used for. In this section we show four examples how the two payload formats, which are presented in the previous sections, can be used to build efficient retransmission schemes. Note that it is not necessarily needed that client and server have negotiated the same retransmission scheme. The rules of how to use the payload formats, see section 5 for details, describe the general behavior that the client (or the server) can expect from the server (or the client). However certain algorithms and behaviors of clients or servers may Miyazaki et al. Expires May 2002 16 Multiple Selective RTP Retransmissions November 2001 enhance the retransmission scheme or make it efficient in certain environments. These kind of schemes are described below, where no standardized coordinated behavior of client and server is needed, but both enhance the functionality rather independently from each other. First we consider a quite simple retransmission algorithm, suitable for the most common applications and environments. With this it is possible to retransmit lost RTP packets, detect the loss of retransmissions efficiently and thus retransmit lost packet several times, if needed. The second example uses the SEL payload format additionally to signal the need for a retransmission request from the server to the client in-band. This leads to the client being able to judge the priority of lost packets and conditionally request retransmissions. Thus bandwidth on the feedback channel can be saved. The third example shows a retransmission scheme, where the retransmissions are not sent in the same stream as the original data. Thus the original RTP stream is untouched. This scheme might be usable for applications that require a fixed mapping between RTP sequence number and the contained payload. The third example shows how retransmissions can be performed for a layered encoded stream. Different layers are sent in different RTP sessions and retransmissions in this case are done per RTP session. Thus the retransmissions fit well into the layered principle of the coding. 10.1 A Multiple Retransmission Algorithm This section gives an examples how the RTX payload format is used to enable multiple retransmissions in RTP. A sample indication in SDP could be: m = video 8000 RTP/AVPF 98 99 a = rtpmap:98 MP4V-ES/90000 a = rtpmap:99 rtx/90000 An additional line a = fmtp:99 3500 would indicate also that the server will try to retransmit packets for 3.5 seconds, after which the packets are deleted from the retransmission buffer and never be sent again. The sender transmits the media data in normal RTP packets using an RTP payload format for that media (e.g. MPEG-4 as described in [6]). Miyazaki et al. Expires May 2002 17 Multiple Selective RTP Retransmissions November 2001 If packets get lost the receiver will detect that by the gap in the sequence number. To signal lost packets the NACK message, described in section 8 is used. The PID field is used to specify a lost packet. The default method, which is to use the RTP sequence number to indicate the lost packet, is used. The BLP field can be used to indicate several lost packets in one NACK message. The detailed semantic is described in [4]. At receiving a retransmission request the sender decides, according to the priority of the packet, the network conditions (see section 8) and maybe other factors, if the packet is to be retransmitted. The maximum transmission time (e.g. 3.5 seconds as indicated with the fmtp attribute above) SHOULD be taken into account if it was signaled, because the client MAY size its playout buffer accordingly. Other negotiations, which are not in the scope of this document, MAY define a different value for a maximum retransmission time. In this case the negotiated value SHOULD be used. In case of a positive decision, the server uses the RTX payload format (see section 5) to retransmit the packet. The sequence number of the RTP packet is increased by one over the previous packet sent. This enables the receiver to detect also a lost retransmission. Example of a retransmission packet: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC=0 | PT=RTX | RTP sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP timestamp = timestamp of original transmission | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of media source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| PT=media_pt |0| DSN=12 | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + : : : Media Payload : : : Note that the client is able to detect the loss of a retransmission by a gap in the sequence number. For the client this looks just like another lost packet, and thus a retransmission request will be sent. If the server, after receiving the request, decided that the packet is still to be retransmitted another time, an additional RTX header is appended. The client, after receiving the packet, can then detect to which retransmission request(s) the packet belongs. Miyazaki et al. Expires May 2002 18 Multiple Selective RTP Retransmissions November 2001 10.2 A Selective Retransmission Algorithm A special scenario, where the previously described multiple retransmission algorithm does not work well, is if the feedback channel has a very limited capacity. Possible scenarios are small multicast groups, where it is according to the defined timing rules for RTCP feedback seldom possible to send immediate feedback, or the feedback channel is mainly used for other purposes (e.g. wireless control channel) and has a very low bit rate available for RTCP feedback. In this scenarios it is very important to limit the feedback to the very minimum, which would mean that feedback is only sent in form of retransmission requests for really important data. To solve the above described requirements on the retransmission scheme, it is necessary that the receiver knows the priority of the lost packets, to decide if a retransmission request should be sent or not. To do so at least two solutions apply: First the important and non-important packets are sent in different streams, using a different sequence number space. Second the selective packet format is used. There are scenarios where the first solution is applicable and thus should be used, because it uses less overhead. However there are drawbacks with this solution. Given that only very few high priority packets are sent, e.g. once every 2 seconds, the time between loosing a packet and detection of this packet loss is at least this inter-packet time. Hence in scenarios as described above, where feedback can be sent only very infrequently, and thus only really important packets are retransmitted, a solution which uses the selective payload format might be the only possible solution at all. The mechanism is signaled in SDP for example as follows: m = video 8000 RTP/AVPF 98 99 100 a = rtpmap:98 sel/90000 a = rtpmap:99 MP4V-ES/90000 a = rtpmap:100 rtx/90000 a = fmtp:100 1500 The sender sends all media data in RTP packets using the SEL payload type plus additional the media payload type. The priority field is set according to the packet's priority. In all packets the Delta SN field is used to signal the delta to the sequence number of the previous high priority packet. The receiver can use this information to check if the last high priority packet was received correctly or not. If this packet was not received or he cannot reliable detect the priority of lost packets (if many packets were lost in a row), a retransmission request is sent. The receiver can also reconstruct the timestamp of the lost packet if the sender used the Delta Time field. From this time information Miyazaki et al. Expires May 2002 19 Multiple Selective RTP Retransmissions November 2001 it is possible to calculate the time after which the packet cannot be used any more (display time). In this case the client should maintain a round trip time (RTT) measurement and compare the calculated display time with the estimated time when the retransmission could arrive. If the retransmission would arrive already outdated, a retransmission request should not be sent. At receiving a retransmission request from the receiver, the server has to make the same decision as described in the previous section, i.e. it has to judge according to the packets priority (which is most probably quite high, because a retransmission request for it was sent), network status etc. if the packet should be retransmitted. If so it uses the retransmission packet type, to enable multiple retransmissions as described in the previous section. However the SEL payload type is additionally used to enable the priority detection at the client. 10.3 Sending Retransmissions on a Different Stream Transmitting the retransmissions in the same stream and on the same address and port as the original data is very efficient. Loss of retransmissions can be detected more or less immediately by the sequence number. One RTCP session controls original and retransmission stream. The RTCP loss fraction value would be correctly computed by considering the retransmissions as well etc. However the mapping between RTP sequence number and the payload is not strictly linear, because of the retransmissions using the same sequence number space. This artifact might lead to confusion in the applications, if these use the RTP sequence number for their purposes in a way that does not allow this non-linear mapping. If so, the retransmissions MAY be sent on a different stream. This implies setting up a new RTP session, which can be indicated in SDP as follows: m = video 8000 RTP/AVPF 98 a = rtpmap:98 MP4V-ES/90000 m = video 8002 RTP/AVPF 99 a = rtpmap:99 rtx/90000 a = fmtp:99 3500 In this case the server will transmit the original video data to port 8000. If the client detects a packet loss, it will request a retransmissions, by sending a generic NACK RTCP message. At reception of this message, the server will decide if the packet should be retransmitted. The same decision finding as in the first two solutions have to be made. In case of a positive retransmission decision, the server will send the packet on the second video stream. As it is a new RTP session, it will use a new RTP sequence number. Because the sequence number is randomly initialized, there Miyazaki et al. Expires May 2002 20 Multiple Selective RTP Retransmissions November 2001 is no mapping or relation between this sequence number and the sequence number of the original video stream. Therefore the full original RTP sequence number has to be transmitted in the RTX header. It has to be noted, that the loss of a retransmission is much more difficult to detect in this case. Waiting for another retransmission, to detect the packet loss by the means of the sequence number would probably introduce a too large delay. Hence timers have to be used for each retransmission. 10.4 Using Retransmissions in Layered Transmissions As described above in section 4, layered encoding and transmission is a well accepted mechanisms to perform rate control, especially in multicast. In this section we will show a sample mechanism how the retransmission framework interacts with layered transmissions. Note that retransmissions in multicast is still an open research topic. We do not intend to solve this and/or specify a retransmission scheme for multicast. However we specify how the retransmission framework can support a layered retransmission scheme syntactically. The loss detection and decisions when to retransmit a packet to which receiver group is clearly outside the scope of this document. As packets of different priorities or importance are sent in different RTP streams, the retransmissions should be sent in the corresponding layer. The rational for this is that the retransmission will probably have the same importance over time and thus have the same importance as at the original transmission. An SDP indication would be: m = video 8000 RTP/AVPF 98 99 a = rtpmap:98 MP4V-ES/90000 a = rtpmap:99 rtx/90000 a = fmtp:99 3500 c = 224.2.1.1/127/3 for multicast or m = video 8000/3 RTP/AVPF 98 99 a = rtpmap:98 MP4V-ES/90000 a = rtpmap:99 rtx/90000 a = fmtp:99 3500 for unicast. Thus three addresses/ports are allocated, each transmitting the original data of the corresponding layer. After a loss detection the receiver(s) signal the lost packet to the server. The server has to Miyazaki et al. Expires May 2002 21 Multiple Selective RTP Retransmissions November 2001 make a retransmission decision, which can be based, beside the arguments given in the previous examples, on the layer, where the packet was lost. In case of a positive retransmission decision, the server uses the RTX payload format to retransmit the packet on its corresponding layer and within the corresponding sequence number space. With this it is immediately possible for the receivers to detect a lost retransmission. Security Considerations T.B.D. Acknowledgements For the work on this retransmission framework, we received a lot of help and advice from the AVT working group and the chairs, Colin Perkins and Stephen Casner. Especially the mechanism of sending retransmissions with a new sequence number in the same sequence number space, using a different payload type was initiated by Colin Perkins. Intellectual property considerations Matsushita has filed patent applications that might possibly have technical relation to this contribution. If part(s) of the contribution by Matsushita employee is (are) included in a standard and Matsushita has patents and/or patent application(s) that are essential to implementation of such included part(s) in said standard, Matsushita is prepared to grant - on the basis of reciprocity (grantback) - a license on such included part(s) on reasonable, non-discriminatory terms and conditions (in according with paragraph 10.3.3 of the RFC 2026). Miyazaki et al. Expires May 2002 22 Multiple Selective RTP Retransmissions November 2001 References 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. 2 Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 3 Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson, "RTP: A Transport Protocol for real-time applications", RFC 1889, January 1996. 4 Ott,J. et al., "Extended RTP Profile for RTCP-based Feedback", IETF Internet Draft, draft-ietf-avt-rtcp-feedback-00, work in progress, July 2001. Miyazaki et al. Expires May 2002 23 Multiple Selective RTP Retransmissions November 2001 AuthorÆs Addresses Akihiro Miyazaki Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Mail akihiro@isl.mei.co.jp Hideaki Fukushima Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Mail fukusima@isl.mei.co.jp Koichi Hata Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Mail hata@isl.mei.co.jp Thomas Wiebke Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Mail wiebke@panasonic.de Rolf Hakenberg Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Mail hakenberg@panasonic.de Carsten Burmeister Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Mail burmeister@panasonic.de Norihito Takatori Mitsubishi Electric Corporation Information Technology R&D Center 5-1-1 Ofuna, Kamakura, Kanagawa, 247-8501, Japan Mail takatori@isl.melco.co.jp Seiji Okumura Mitsubishi Electric Corporation Information Technology R&D Center 5-1-1 Ofuna, Kamakura, Kanagawa, 247-8501, Japan Mail okumura@isl.melco.co.jp Tsugihiko Ohno Mitsubishi Electric Corporation Information Technology R&D Center 5-1-1 Ofuna, Kamakura, Kanagawa, 247-8501, Japan Mail tohno@isl.melco.co.jp Miyazaki et al. Expires May 2002 24
Datatracker
Report a datatracker bug
draft-ietf-avt-rtp-selret-03
Internet-Draft
| Document | Document type |
This is an older version of an Internet-Draft whose latest revision state is "Expired".
Expired & archived
|
|
|---|---|---|---|
| Select version | |||
| Compare versions | |||
| Author | |||
| RFC stream |
|
||
| Other formats | |||
| Additional resources | Mailing list discussion |