Audio/Video Transport Working Group Akihiro Miyazaki Internet Draft Hideaki Fukushima Document: draft-miyazaki-avt-rtp-selret-00.txt Thomas Wiebke March 01, 2000 Rolf Hakenberg Expires: September 01, 2000 Carsten Burmeister Matsushita RTP Payload Type Format to Enable Selective Retransmissions Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. 1. Abstract This document describes a new RTP payload type and a new RTCP packet format to enable selective retransmissions of generic media data. While RTP is widely used for media streaming applications over the Internet, using it over unreliable links needs some further modifications to achieve a good quality of the media stream at the receiver. This paper describes a mechanism to use the lower delay requirements in streaming applications to make RTP more reliable, by the means of selective retransmissions. Therefore a new RTP payload type is defined as well as a new RTCP packet. An example algorithm to use the new payload type fields and the new RTCP packet is introduced and an additional section shows results of a performance evaluation. The evaluation is done by simulating a video streaming application over an mobile communication system. Miyazaki Expires September 01, 2000 1 Selective Retransmissions for RTP March 01, 2000 Table of Contents 1. Abstract 2. Conventions used in this document 3. Introduction 4. Proposed solution 5. Required RTP header additions 6. Required additional RTCP packet 7. An example use of the payload type 7.1 Selective retransmission 7.2 Multiple retransmission attempts 7.3 Retransmission with time limit based on client judgement 8. Performance evaluation of the proposed solution 8.1 Application 8.2 Environment 8.3 Performance results 8.4 Conclusions 9. Security considerations 10. Intellectual property consideration 11. References 12. Author's addresses 2. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [2]. 3. Introduction The Real-Time Transport Protocol (RTP)[3] was designed as an Internet protocol to transmit real-time (or near real-time) data over multicast or unicast. Even though the multicast capability is a strong feature of RTP, a common use of it is non-interactive unicast transmission of media streams. This application, further referred to as media streaming, has lower delay requirements. A constant offset Miyazaki Expires September 01, 2000 2 Selective Retransmissions for RTP March 01, 2000 delay, in which already received packets are stored at the client, is tolerable. Typically RTP is run over the User Datagramm Protocol (UDP)[4]. While the unreliable UDP performs good for reliable channels, the quality of the received media stream is bad, when transmitted over error-prone links. Especially compressed streams, such as MPEG video-data, are highly susceptible to packet loss, which will lead to a bad video quality at the client. E.g. in a wireless environment with a unstable transmission quality (bit error rate (BER) about 10E-3 .. 10E-6) a sufficient quality of video and voice transmitted by RTP cannot be achieved. The scenario for such a media streaming over an error-prone link might be the one in Figure 1. Media files are stored on a media server which is connected via an internet link to a mobile network. The wireless link between the mobile network and the mobile terminal, which works as the client in this scenario, is generally unreliable and subject to bit errors. Media Mobile Mobile Server Network Terminal (Client) *** | +----+ ** ** +--+ | | * * | | | | -------------- * * ~ ~ ~ ~ ~ ~ ~ | | +----+ * * +--+ ** ** Internet *** Wireless Link Link Figure 1 : Scenario for media streaming To achieve a better quality of the received media stream, especially when transmitted over error-prone links, retransmissions of lost packets of the stream seems to be a possible solution. However the quite low delay requirements of media streaming applications and the bandwidth limitations of the wireless link have to be considered. In wireless systems bandwidth is a scarce resource and efficient use of this resource is of vital importance. Hence reliability for the complete media stream might not be achievable. Furthermore in compressed media streams, e.g. MPEG video stream, not every packet is equal important. Some packets contain data that other data depends upon, which makes these packets more important. Miyazaki Expires September 01, 2000 3 Selective Retransmissions for RTP March 01, 2000 Current protocols applying retransmission for reliable transmission (e.g. TCP) make no use of this characteristic of compressed media streams. 4. Proposed solution The solution presented in this document is based on the concept of selective retransmission of data packets in case of packet loss. As mentioned before, compressed media streams generally consist of data packets of high importance and data packets of lower importance. In the proposed solution data packets of high importance are retransmitted in case of packet loss and by this the quality of the received media stream is increased significantly. However single retransmission of lost data packets, which are transmitted over links with high bit error rates are still far from being reliable. Even though no 100% reliability is desired, the delay requirements and bandwidth limitations might allow more than one retransmission of important parts of the media stream. Therefore mechanisms for multiple retransmission attempts are included in our proposal. On the other hand by limiting the retransmissions on only the data packets of high importance and transmitting the data packets of lower importance the usual unreliable way the delay requirements of media streaming can still be meet. Additionally the total bandwidth requirements for transmission and retransmission of the media stream data are kept at a low level suitable and advantageous for usage over bandwidth limited, wireless links. To keep also the bandwidth requirements on the link back from the client to the media server at a minimum level, means for a judgement at the client, whether a retransmission would still arrive in time (retransmission judgement), is proposed. With our solution it is possible to calculate the time when the lost packet has to be received at the client at latest. This time is compared against the round trip time. Only if this calculation predicts that a retransmission of the lost packet will be received in time, a retransmission is generated at the client and send to the server. Therefore no bandwidth is wasted for unnecessary retransmission requests and subsequent retransmissions. In summary we propose the following enhancements/extensions to RTP: 1) Selective retransmission Retransmission of lost packets, but limited to lost packets of high importance. 2) Multiple retransmission attempts Miyazaki Expires September 01, 2000 4 Selective Retransmissions for RTP March 01, 2000 In case retransmission of a lost data packet fails, further attempts can be requested. 3) Retransmission with time limit based on client judgement Means to calculate the time when a lost packet has to be received at the client at latest to perform a efficient client based retransmission judgement. 5. Required RTP header additions in a new RTP payload type In order achieve the solution described in section 3, we propose to extend the RTP header by the following fields. 0 1 2 3 4 5 6 7 +---+---+---+---+---+---+---+---+ 0 |PA | PT | +---+---+---+---+---+---+---+---+ 1 |PR |DTI| SNHP | +---+---+---+---+---+---+---+---+ 2 : Diff Time Stamp : +...............................+ 3 : NULL padding : +...............................+ Padding (PA): 1bit If PA field is set to 1 padding bits (NULL) are added to the end of RTP header extension so that the total size of it can be divided by 4 bytes. Payload Type (PT): 7 bits This fields identifies the payload type that follows the header additions and is used as described in [3]. Priority (PR): 1 bit This field indicates the priority of the RTP packet. If set it indicates the presence of an data packet with high priority. Diff Time Stamp Indicator(DTI): 1 bit If DTI field is set diff time stamp field is included in the RTP header extension. For further information please refer to description of diff time stamp field. Sequence Number of RTP packet with High Priority (SNHP): 6 bits If priority field (PR) is set this field indicates the sequence number of the RTP packet with high priority. In other words this number will only be incremented if a high priority packet is sent. Miyazaki Expires September 01, 2000 5 Selective Retransmissions for RTP March 01, 2000 If the priority field is not set SNHP indicates the sequence number of the latest packet with high priority. Diff Time Stamp This field indicates the difference between the time stamp of the last RTP packet with high priority and that of the current RTP packet. NULL Padding Null Pading bits to make the total size of the proposed RTP header extension divisible by 4 bytes. 6. Required new RTCP packet type The retransmission request, which is sent from the client to the server to request the retransmission of a lost packet, has a format as described below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P| RXP | PT = RReq | length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |C1 | SNHP1 |C2 : SNHP2 : +-+-+-+-+-+-+-+-+................ version(V), padding(P), length, SSRC: as described in [3], Section 6.7. retransmission protocol(RXP): This field identifies the specific retransmission protocol to be used. The specification of the retransmission protocols is out of the scope of this document, however an example protocol, that uses the new RTCP packet is described in section 7. control(C): Indicates the number of lost packets: 00: single packet 01: two consecutive packets 10: three consecutive packets 11: start & end marker indicate more than three consecutive packets SNHP: Indicates the SNHP value of the requested retransmission Miyazaki Expires September 01, 2000 6 Selective Retransmissions for RTP March 01, 2000 7. An example use of the payload type In section 4 we proposed a solution to enhance the quality of media stream, transmitted over an unreliable link. How this solution can be achieved by using the header fields as described in section 5 and 6 is shown in form of an example in this section. 7.1 Selective retransmission As described before a media stream contains more or less important parts. In section 4 we introduced a priority bit (PR) which SHOULD be used to mark packets, that contain the important parts of the media stream. With this it is possible to distinguish between two priority levels, and to allow a retransmission of only the important packets. 7.2 Multiple retransmission attempts If a retransmission fails, further attempts to retransmit the lost packet MAY be requested. While the detection of a lost packet is normally straight forward (e.g. by checking the sequence number), detecting a lost retransmission requires further mechanisms. This is done by the means of the SNHP field. The SNHP value MUST be incremented by one for every high priority packet sent, regardless whether it is a retransmission or sent for the first time. By comparing the actual received with the most recent SNHP value, the client will detect the loss of a high priority packet immediately (i.e. if the SNHP value was increased, but no high priority packet was received, it must have been lost). This enables the client to send another retransmission request and the server to do multiple retransmissions. 7.3 Retransmission with time limit based on client judgement As described in section 3, bandwidth might be a very scarce resource. Therefore requesting the retransmission or retransmitting packets that will not be received in time anyway, SHOULD be avoided. Our solution proposes the client to estimate the arrival time of the requested lost packet and the time after which it will become useless. Therefore the RTP timestamp of the lost packet is needed, which is achieved by using the diff times stamp field. In these fields the difference between the actual packet's time stamp and the most recent high priority packet's timestamp is calculated. At detecting a lost high priority packet, it is possible to calculate the time stamp of the lost packet by the means of this field, because the received packet contains its own timestamp and the difference to the lost packet's timestamp. Miyazaki Expires September 01, 2000 7 Selective Retransmissions for RTP March 01, 2000 The knowledge of the lost packet's time stamp enables the client to judge if a retransmission request would still make sense or if the bandwidth should be saved. 8. Performance evaluation of the proposed solution The previous sections introduced a solution, how to enhance the capabilities of media streaming application over unreliable links. The solution implies a new RTP payload type and a new RTCP packet as described in sections 5 and 6. How these fields can be used in general is described in an example in section 7. This section will show the increased performance that is possible with our solution. This is done by simulating a video streaming application over a unreliable wireless link. 8.1 Application In this performance evaluation we simulate a MPEG4 video streaming application. MPEG4 video data is streamed from a server to a client (i.e. unicast). In the client the received frames are buffered before they are displayed at their scheduled playing time. Therefore the delay requirements are not exactly real-time. The scheduled playing time includes an additional buffering delay. We consider a transmission of a real MPEG4 video stream, with a total play time of about 120s and an average bit rate of 33100 bps. The frame rate is 10 fps. The frame length is variable between 100byte and 2000byte. The additional allowed buffering delay is set in our simulations to 2 seconds. The MPEG4 video stream consists of Intra coded (I-)frames and Predictive coded (P-)frames. While the I-frames are completely independent and can be displayed all by themselves, the P-frames contain only the difference to the previous frame and hence need the previous frame to be received correctly to be displayed. This leads to I-frames having a higher importance than P-frames and with this to I-frame having priority one and P-frames having priority zero assigned. 8.2 Environment A mobile environment is considered as a typical representation for unreliable, bandwidth limited links. Figure 1 illustrates the general scenario that is evaluated. The used protocols for the evaluation are shown in Figure 2. Miyazaki Expires September 01, 2000 8 Selective Retransmissions for RTP March 01, 2000 Media Server Mobile Network Mobile Client +-----------+ +-----------+ | Video- | | Display- | |Application| |Application| +-----------+ +-----------+ | RTP | | RTP | +-----------+ +-----------+ | UDP | | UDP | +-----------+ +-----------+ +-----------+ | IP |<--------->| IP | | IP | +-----------+ +-----------+ +-----------+ | W-CDMA |<--------->| W-CDMA | +-----------+ +-----------+ Internet Wireless Link Link Figure 2 The RTP layer implies the newly defined RTP payload type and RTCP packet as described in sections 5 and 6 and the use of the additional fields as given in the example in section 7. Additional to the client's retransmission judgement, a similar mechanism is used on the server side, to avoid unnecessary retransmissions. The Internet link is modeled as error-free, loss-less and with a constant delay of 50 ms. For the wireless link a W-CDMA channel is considered. W-CDMA, as a third generation mobile communication system with high Quality-of- Service (QoS) capabilities, is a well suited network for this kind of applications in a mobile use. The W-CDMA system is standardized at the 3rd Generation Partnership Project (3GPP). The standardization is still under progress and no final specifications are finished. For the simulations the specifications of December 1999 are considered. The wireless link and its link layer protocols are simulated according to this specifications (see [7] for details). A single user is considered for the mobile channel, hence the bandwidth is not shared between different users here. The channel is simulated with the fading model, as described in [8]. The fading process has a mean signal-to-noise ratio per received symbol of Es/N0. Different mean Es/N0 values are simulated, which reflect different channel conditions. The back channel in our simulations is modeled as error-free and therefore loss-less. Miyazaki Expires September 01, 2000 9 Selective Retransmissions for RTP March 01, 2000 The propagation delay of the mobile channel, is strongly dependent on the length of the packet that is to be sent. The minimum delay of 50ms applies for all transmitted frames, due to the coding, interleaving etc. in the physical layer. 8.3 Performance results This section shows some results of the simulations as described above. To measure the quality of the received video stream, the number of displayed frames, relative to the number of sent frames, is considered. The next figure shows the amount of received frames for RTP and the extended RTP for different channel conditions. It can be seen from this figures, that both applications suffer from bad channel conditions. In states, where Es/N0 is smaller than 5dB, not even half of the video frames can be displayed, using normal RTP. In those conditions many errors occur, which leads to a loss of many frames. Additionally, if an I-frame gets lost, all following P-frames, even if they would be received correctly, are discarded. The application using the extended RTP as described in section 7, performs much better. Even though it suffers from bad channel conditions as well, the repetition of lost I-frames, leads to an increased performance. The next figure shows the amount of displayed P-frames for different channel conditions. Miyazaki Expires September 01, 2000 10 Selective Retransmissions for RTP March 01, 2000 Even though the lost P-frames are not retransmitted, more frames are displayed, using the extended RTP. This is due to the fact, that fewer correctly received frames are discarded, because of a lost I- frame. The next figure shows the amount of received I-frames for different channel conditions. Miyazaki Expires September 01, 2000 11 Selective Retransmissions for RTP March 01, 2000 From this figure the effect of the extended RTP is visible best. Nearly all I-frames are displayed at the receiver in this case, while only half of the frames can be displayed with normal RTP in bad channel conditions. 8.4 Conclusions It was shown in the previous sections that repetition of important media data, leads to a better media quality at the receiver. These repetitions are possible, if the lower delay requirements in streaming applications are used wisely. To do so, a newly defined RTP payload type and RTCP packet are introduced, which enable a media server to do multiple, selective retransmissions. 9. Security Considerations Security is not considered in this draft. 10. Intellectual property considerations Matsushita has filed patent applications that might possibly have technical relation to this contribution. 11. References 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. 2 Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997 3 Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson, "RTP: A Transport Protocol for real-time applications", RFC 1889, January 1996. 4 Postel, J.,"User Datagram Protocol", STD 6, RFC 768, August 1980. 5 Postel, J., "Internet Protocol", STD 5, RFC 791, September 1981. 6 Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", RFC 2460, December 1998. 7 Homepage of 3GPP: http://www.3gpp.org 8 "Procedure for Evaluation of Transmission Technologies for FPLMTS", ITU-R TG8-1, 8-1/TEMP/233-E, September 1995. Miyazaki Expires September 01, 2000 12 Selective Retransmissions for RTP March 01, 2000 12. Author's Addresses Akihiro Miyazaki Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Tel. +81-6-6900-9192 Fax. +81-6-6900-9193 Mail akihiro@isl.mei.co.jp Hideaki Fukushima Matsushita Electric Industrial Co., Ltd 1006, Kadoma, Kadoma City, Osaka, Japan Tel. +81-6-6900-9192 Fax. +81-6-6900-9193 Mail fukusima@isl.mei.co.jp Thomas Wiebke Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-161 Fax. +49-(0)6103-766-166 Mail wiebke@panasonic.de Rolf Hakenberg Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-162 Fax. +49-(0)6103-766-166 Mail hakenberg@panasonic.de Carsten Burmeister Panasonic European Laboratories GmbH Monzastr. 4c, 63225 Langen, Germany Tel. +49-(0)6103-766-263 Fax. +49-(0)6103-766-166 Mail burmeister@panasonic.de Miyazaki Expires September 01, 2000 13 Selective Retransmissions for RTP March 01, 2000 Full Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into Miyazaki Expires September 01, 2000 14