Internet Draft Adam Li draft-ietf-avt-ulp-08.txt Editor October 26, 2003 UCLA Expires: April 26, 2004 An RTP Payload Format for Generic FEC STATUS OF THIS MEMO This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC 2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at anytime. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as work in progress. The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. ABSTRACT This document specifies a payload format for generic Forward Error Correction (FEC) for media data encapsulated in RTP. It is based on the exclusive-or (parity) operation, and it is a generalized algorithms that includes Uneven Level Protection (ULP). The payload format described in this draft allows end systems to apply protection using arbitrary protection lengths and levels, in addition to using arbitrary protection group sizes. It enables complete recovery or partial recovery of the critical payload and RTP header fields depending on the packet loss situation. This scheme is completely backward compatible with non-FEC capable hosts. Those receivers that do not know about FEC can simply ignore the protection data. This draft will obsolete RFC 2733 and RFC 3009. Adam H. Li, et al. [Page 1]
I-Draft An RTP Payload Format for Generic FEC October 2003 Table of Contents 1. Introduction .................................................. 3 2. Terminology ................................................... 4 3. Basic Operation ............................................... 5 4. Parity Codes .................................................. 6 5. RTP Media Packet Structure .................................... 6 6. FEC Packet Structure .......................................... 7 6.1. Baseline Mode FEC ........................................... 7 6.2. Extended Mode FEC ........................................... 8 6.3. RTP Header of FEC Packets ................................... 8 6.4. FEC Header of FEC packets ................................... 9 6.5. ULP Level Header of FEC Packets ............................ 10 7. Protection Operation ......................................... 11 7.1. Protection Operation in the Baseline Mode .................. 12 7.2. Protection Operation in the Extended Mode .................. 12 7.2.1. Extended Mode - Protection Level 0 ....................... 12 7.2.2. Extended Mode - Protection Level 1 and Higher ........... 13 8. Recovery Procedure ........................................... 13 8.1. Reconstruction in the Baseline Mode ........................ 15 8.2. Reconstruction in the Extended Mode ........................ 15 8.2.1. Extended Mode - Reconstruction of Level 0 ................ 15 8.2.1. Extended Mode - Reconstruction of Level 1 and Higher ..... 15 9. Examples ..................................................... 15 9.1. A Baseline Mode Example .................................... 16 9.2. An Extended Mode Example With one Protection Level ......... 17 9.3. An Extended Mode Example With Two Protection Levels ........ 19 10. Security and Congestion Considerations ...................... 22 11. Indication FEC Usage in SDP ................................. 23 11.1. FEC as a Separate Stream .................................. 23 11.2. Use with Redundant Encoding ............................... 24 11.3. Usage with RTSP ........................................... 25 12. MIME Registrations .......................................... 26 12.1. Registration of audio/ulpfec .............................. 26 12.2. Registration of video/ulpfec .............................. 27 12.3. Registration of text/ulpfec ............................... 28 12.4. Registration of application/ulpfec ........................ 29 13. Application Statement ....................................... 30 14. Acknowledgements ............................................ 32 15. Bibliography ................................................ 32 16. Author's Address ............................................ 33 Adam H. Li [Page 2]
I-Draft An RTP Payload Format for Generic FEC October 2003 1. Introduction Because of the real-time nature of many applications, they usually have more stringent delay requirements than normal data transmissions. As a result, retransmission of the lost packets is generally not a valid option for such applications. In these cases, a better method to attempt recovery of information from packet loss is through Forward Error Correction (FEC). FEC has been one of the main methods used to protect against packet loss over packet switched networks [1,2]. In particular, the use of traditional error correcting codes, such as parity, Reed-Solomon, and Hamming codes, has attracted attention. To apply these mechanisms, protocol support is required. This document defines a payload format for RTP [3] which allows for generic forward error correction of real time media. In this context, generic means that the FEC protocol is (1) independent of the nature of the media being protected, be it audio, video, or otherwise, (2) flexible enough to support a wide variety of FEC mechanisms, (3) designed for adaptivity so that the FEC technique can be modified easily without out of band signaling, and (4) supportive of a number of different mechanisms for transporting the FEC packets. Furthermore, in many cases the bandwidth of the network connections is a very limited resource. On the other hand, most of traditional FEC schemes are not designed for optimal utilization of the limited bandwidth resource. A more efficient way to utilize the limited bandwidth would be to use unequal error protection to provide different levels of protection for different parts of the data stream which vary in importance. The unequal error protection schemes can usually make more efficient use of the bandwidth to provide better overall protection of the data stream against the loss. Proper protocol support is essential for realizing these unequal error protection mechanisms. However, the application of most of the unequal error protection schemes requires the knowledge of the importance for different parts of the data stream. Most of such schemes are designed for a particular type of media according to the structure of the media protected, and as a result, are not generic. This document defines an extended mode FEC algorithm and protocol that allows for generic forward error correction with unequal error protection for real-time media. It is called the Uneven Level Protection (ULP). The payload data are protected by one or more protection levels. Lower protection levels provide greater protection by using smaller group sizes (compared to higher protection levels) for generating the FEC packet. As we will discuss below, audio/video applications would generally benefit from an unequal error protection scheme that gives more protection to the beginning part of each packet. So in the ULP algorithm, the data Adam H. Li [Page 3]
I-Draft An RTP Payload Format for Generic FEC October 2003 that are closer to the beginning of the packet are protected by lower protection levels because these data are in general more important, and they tend to carry more information than the data further behind in the packet. In many multimedia streams, the more important parts of the data are always at the beginning of the data packet. This is the common practice for most codecs since the beginning of the packet is closer to the re-synchronization marker at the header and thus is more likely to be correctly decoded. In additional, almost all media formats have the frame headers at the beginning of the packet, which is the most vital part of the packet. For video streams, most modern formats have optional data partitioning modes to improve error resilience in which the video macroblock header data, the motion vector data, and DCT coefficient data are separated into their individual partitions. In ITU-T H.263 version 3, there is the optional data partitioned syntax of Annex V. In MPEG-4 Visual Simple Profile, there is the optional data partitioning mode. When these modes are enabled, the video macroblock (MB) header and motion vector partitions (which are much more important to the quality of the video reconstruction) are transmitted in the partition(s) at the beginning of the video packet while residue DCT coefficient partitions (which are less important) are transmitted in the partition close to the end of the packet. Because the data is arranged in descending order of importance, it would be beneficial to provide more protection to the beginning part of the packet in transmission. For audio streams, the bitstreams generated by many of the new audio codecs also contain data with different classes of importance. These different classes are then transmitted in order of descending importance. Thus, applying more protection to the beginning of the packet would also be beneficial in these cases. Even for uniform- significance audio streams, special stretching techniques can be applied to the partially recovered audio data packets. In cases where audio redundancy coding is used, more protection should be applied to the original data located in the first half of the packet. The rest of the packet containing the redundant copies of the data, does not need the same level of protection. It is clear that audio/video applications would generally benefit from the FEC algorithms specified in this document. And with the extended mode ULP FEC, more efficient protection of the media payload can be potentially achieved. This document specifies the protocol and algorithm for applying the generic FEC to the RTP media payloads. 2. Terminology The following terms are used throughout this document: Adam H. Li [Page 4]
I-Draft An RTP Payload Format for Generic FEC October 2003 Media Payload: The raw, un-protected user data that are transmitted from the sender. The media payload is placed inside of an RTP packet. Media Header: The RTP header for the packet containing the media payload. Media Packet: The combination of a media payload and media header is called a media packet. FEC Packet: The FEC algorithms at the transmitter take the media packets as an input. They output both the media packets that they are passed, and newly generated packets called FEC packets, which contain redundant media data used for error correction. The FEC packets are formatted according to the rules specified in this document. FEC Header: The header information contained in an FEC packet. FEC Payload: The payload of an FEC packet. Associated: A FEC packet is said to be "associated" with one or more media packets (or vice versa) when those media packets are used to generate the FEC packet (by use of the exclusive-or operation). In case of extended mode FEC, this refers to only those packets used to generate the Level 0 FEC payload, if not explicitly stated otherwise. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [4]. 3. Basic Operation The payload format described here is used whenever the sender in an RTP session would like to protect the media stream it is sending with generic parity FEC. The FEC supported by this format is based on the simple exclusive-or (XOR) parities operation. The sender takes the packets from the media stream requiring protection and determines the operation needed, and in case of the extended mode, also the protection levels for these packets and the protection length for each level. The data are grouped as described below in Section 6. An XOR operation is applied across the payload to generate the FEC information. The result based on the procedures defined here is an RTP packet containing FEC information. This packet can be used at the receiver to recover the packets or parts of the packets used to generate the FEC packet. By using unequal error protection provided by the extended mode, this scheme can make more efficient use of the channel bandwidth, and provide more Adam H. Li [Page 5]
I-Draft An RTP Payload Format for Generic FEC October 2003 efficient error resilience for transmission over error prone channels. The payload format contains information that allows the sender to tell the receiver exactly which media packets are protected by the FEC packet, and the protection levels and lengths for each of the levels. Specifically, each FEC packet contains an offset mask m(k) for each protection level k. If the bit i in the mask m(k) is set to 1, then media packet number N + i is protected by this FEC packet at level k. N is called the sequence number base, and is sent in the FEC packet as well. When the extended mode is used, the amount of data that are protected at level k is indicated by L(k), which is also sent in the FEC packet. The protection length, offset mask and payload type are sufficient to signal the forward error correction schemes based on arbitrarily defined parity protection with little overhead. A set of rules is described in Section 5.3 that defines how the mask should be set for different protection levels, with examples in Section 8. This document also describes procedures on transmitting all the protection operation parameters in-band. This allows the sender great flexibility; the sender can adapt the code to current network conditions and be certain the receivers can still make use of the ULP FEC for recovery. At the receiver, the FEC and original media are received. If no media packets are lost, the FEC packets can be ignored. In the event of a loss, the FEC packets can be combined with other received media to recover all or part of the missing media packets. 4. Parity Codes For brevity, we define the function f(x,y,..) to be the XOR (parity) operator applied to the packets x,y,... The output of this function is another packet, called the parity packet. For simplicity, we assume here that the parity packet is computed as the bitwise XOR of the input packets. The exact procedure is specified in section 6. Recovery of data packets using parity codes is accomplished by generating one or more parity packets over a group of data packets. To be effective, the parity packets must be generated by linearly independent combinations of data packets. The particular combination is called a parity code. One class of codes takes a group of k data packets, and generates n-k parity packets. There are a large number of possible parity codes for a given n,k. The payload format does not mandate a particular code. For example, consider a parity code which generates a single parity packet over two data packets. If the original media packets are a,b,c,d, the packets generated by the sender are: Adam H. Li [Page 6]
I-Draft An RTP Payload Format for Generic FEC October 2003 a b c d <-- media stream f(a,b) f(c,d) <-- FEC stream where time increases to the right. In this example, the error correction scheme (we use the terms scheme and code interchangeably) introduces a 50% overhead. But if b is lost, a and f(a,b) can be used to recover b. 5. RTP Media Packet Structure The formatting of the media packets is unaffected by FEC. If the FEC is sent as a separate stream, the media packets are sent as if there was no FEC. This scheme leads to a very efficient encoding. When little or no FEC is used, the transmitted stream contains mostly media packets. The overhead for using the FEC scheme is only present in FEC packets, and can be easily monitored and adjusted by tracking the amount of FEC in use. 6. FEC Packet Structure The FEC packets has two modes: Baseline Mode and Extended Mode. The baseline mode protects each media packet to its full length with equal weight to all the parts of the packet. The extended mode provides additional flexibility by allowing different protection to be applied to different parts of the packets. The packets of the two modes of FEC are distinguished by the Extension bit carried in-bound in the FEC header. External signal is not needed. Mode can be changed on the fly in the same stream during a session. 6.1. Baseline Mode FEC In the baseline mode FEC, an FEC packet is constructed by placing an FEC header and FEC payload in the RTP payload, as shown in Figure 1: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FEC Header (12 or 16 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FEC Payload | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1: Baseline mode FEC Packet Structure Adam H. Li [Page 7]
I-Draft An RTP Payload Format for Generic FEC October 2003 Please note that baseline mode FEC packets contain only one payload section, and do not have any level headers. 6.2. Extended Mode FEC In the extended mode FEC (which is also called ULP FEC), a FEC packet is constructed by placing an FEC header and one or more levels of FEC header and payload into the RTP payload, as shown in Figure 2: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | RTP Header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FEC Header (12 or 16 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FEC Level 0 Header (2 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FEC Level 0 Payload | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FEC Level 1 Header (4 or 8 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | FEC Level 1 Payload | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Cont. | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 2: Extended mode FEC Packet Structure 6.3. RTP Header of FEC Packets The version field is set to 2. The SSRC value will generally be the same as the SSRC value of the media stream it protects, though it MAY be different if the FEC stream is being demultiplexed via the SSRC value. The sequence number has the standard definition: it MUST be one higher than the sequence number in the previously transmitted FEC packet. The timestamp MUST be set to the value of the media RTP clock at the instant the FEC packet is transmitted. Thus, the TS value in FEC packets is always monotonically increasing. The payload type for the FEC packet is determined through dynamic, out of band means. According to RFC 1889 [3], RTP participants that cannot recognize a payload type must discard it. This provides backwards compatibility. The FEC mechanisms can then be used in a multicast group with mixed FEC-capable and FEC-incapable receivers. Adam H. Li [Page 8]
I-Draft An RTP Payload Format for Generic FEC October 2003 In such a case, the FEC stream will have a payload type which is not recognized by the FEC-incapable receivers, and will thus be disregarded. 6.4. FEC Header of FEC Packets This header is 12 or 16 octets, depending on whether the extension bit is set. The format of the header is shown in Figure 2 and consists of extension flag (E bit), long-mask flag (L bit), P recovery field, X recovery field, CC recovery field, M recovery field, PT recovery field, SN base field, TS recovery field, and the mask field. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |E|L|P|X| CC |M| PT recovery | SN base | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TS recovery | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | length recovery | mask | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | mask cont. (present only when L = 1) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3: FEC Header Format The E bit indicates the mode of the current FEC packet. The E bits for the baseline mode packets MUST be set to 0. The E bits for the extended mode packets MUST be set to 1. The L bit indicates whether the long mask is used. When the L bit is not set, the mask is 16-bit long. When the L bit is set, the mask is then 48-bit long. The P recovery field, the X recovery field, the CC recovery field, the M recovery field, and the PT recovery field are obtained via the protection operation applied to the P, X, CC, M, and PT values of the media packets associated with the FEC packet. The SN base field MUST be set to the minimum sequence number of those media packets protected by FEC (at all levels for the extended mode). This allows for the FEC operation to extend over any string of at most 16 packets when the L bit is not set or 48 packets when the L bit is set. The TS recovery field is computed via the protection operation applied to the timestamps of the media packets associated with this FEC packet. This allows the timestamp to be completely recovered. Adam H. Li [Page 9]
I-Draft An RTP Payload Format for Generic FEC October 2003 The length recovery field is used to determine the length of any recovered packets. It is computed via the protection operation applied to the unsigned network-ordered 16 bit representation of the sums of the lengths (in bytes) of the media payload, CSRC list, extension and padding of media packets associated with this FEC packet (in other words, the CSRC list, RTP extension, and padding of the media payload packets, if present, are "counted" as part of the payload). This allows the FEC procedure to be applied even when the lengths of the media packets are not identical. For example, assume an FEC packet is being generated by xor'ing two media packets together. The length of the two media packets are 3 (0b011) and 5 (0b101) bytes, respectively. The length recovery field is then encoded as 0b011 xor 0b101 = 0b110. The mask field in the FEC header indicates which packets are associated (or associated at level 0 if extended mode is used) with the FEC packet. It is either 16 bits or 48 bits depending on whether the L bit is set. If bit i in the mask is set to 1, then the media packet with sequence number N + i is associated with this FEC packet, where N is the SN Base field in the FEC packet header. The least significant bit of the mask corresponds to i=0, and the most significant to i=15 when the L bit is not set or i=47 when the L bit is set. 6.5. ULP Level Header of FEC Packets The ULP Level Header is 2 octets for ULP level 0, and 4 or 8 octets (depending on whether the L bit is set in the FEC header) for ULP level 1 and higher. The formats of the headers are shown in Figure 3 and Figure 4. Figure 3 shows the ULP FEC level header with level 0. It consists of only one field for the protection length. The Protection Length field is 16 bits. It indicates the protection length provided by the ULP FEC for Level 0 (i.e., the payload length after the header). 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Protection Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3: ULP Level Header Format (Level 0) Figure 4 shows the ULP FEC level header with level 1 and higher. It consists of a Protection Length field and a mask field (for level 1 and higher headers). The protection length field is 16-bit long. The mask field is 16-bit long (when the L bit is not set) or 48-bit long (when the L bit is set). Its meaning is the same as the mask field Adam H. Li [Page 10]
I-Draft An RTP Payload Format for Generic FEC October 2003 in the main FEC header, except it now indicates which packets are protected by the FEC at the current level. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Protection Length | mask | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | mask cont. (present only when L = 1) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4: ULP Level Header Format (Level 1 and higher) The setting of mask field when the extended mode FEC is used shall follow the following rules: a. A media packet can only be protected at each protection level once. b. For a media packet to be protected at level p, it must also be protect at level p-1. c. If an ULP FEC packet contains protection at level p, it must also contain protection at level p-1. The payload of the ULP FEC packet of each level is the protection operation applied to the concatenation of the CSRC list, RTP extension, media payload, and padding of the media packets associated with the ULP FEC packet. Details are described in the next section on the protection operation. Note that it's possible for the FEC packet to be slightly larger than the media packets it protects (due to the presence of the FEC header). This could cause difficulties if this results in the FEC packet exceeding the Maximum Transmission Unit size for the path along which it is sent. 7. Protection Operation The protection operation involves copying the payload, padding it with zeros, and computing the parity (XOR) across the resulting bit strings. The resulting bit string is used to generate the ULP FEC packet. The following procedure MAY be followed for the protection operation. Other procedures MAY be used, but the end result MUST be identical to the one described here. Adam H. Li [Page 11]
I-Draft An RTP Payload Format for Generic FEC October 2003 The protection operation can be performed in one of the two modes: the baseline mode and the extended mode. 7.1. Protection Operation in the Baseline Mode The protection operation in the baseline mode is performed as the following. For each media packet to be protected, a bit string is generated by concatenating the following fields together in the order specified: o The first 64 bits of the RTP header (64 bits) o Unsigned network-ordered 16 bit representation of the sum of the lengths of the CSRC List, length of the padding, length of the extension, and length of the media packet (16 bits) o if CC is nonzero, the CSRC List (variable length) o if X is 1, the Header Extension (variable length) o the payload (variable length) o Padding, if present (variable length) If the lengths of the bit strings are not equal, each bit string that is shorter than length of the longest, MUST be padded to that length with 0. The pad MUST be added at the end of the bit string. The parity operation is then applied across the bit strings. The result is the bit string used to build the FEC packet. We will call this the FEC bit string. The first (most significant) two bits in the FEC bit string are skipped. The next bit in the FEC bit string is written into the P recovery bit of the FEC header in the FEC packet. The next bit in the FEC bit string is written into the E recovery bit of the FEC header. The next four bits of the FEC bit string are written into the CC recovery field of the FEC header. The next bit is written into the M recovery bit of the FEC header. The next 7 bits of the FEC bit string are written into the PT recovery field in the FEC header. The next 16 bits are skipped. The next 32 bits of the FEC bit string are written into the TS recovery field in the FEC header. The next 16 bits are written into the length recovery field in the packet header. The remaining bits are set to be the payload of the FEC packet. 7.2. Protection Operation in the Extended Mode 7.2.1. Extended Mode - Protection Level 0 Protection operation in the extended mode on protection level 0 is very similar to the protection operation in the baseline mode as Adam H. Li [Page 12]
I-Draft An RTP Payload Format for Generic FEC October 2003 described above. The only difference is that the protection is only applied to the payload RTP packet up to the protection length plus 96 bits (The 96-bit corresponds to the length of the RTP header upto the end of the SSRC field). That means the padding of the shorter packages is necessary only up to the point of the payload RTP packet at the protection length plus 96 bits. Likewise for the parity operation and copying of the resulting parity string into the FEC packets. 7.2.2. Extended Mode - Protection Level 1 and Higher The protected data of the corresponding packets are copied into the bit strings. If the packet ends before the Protection Length of the current level is reached, the string is padded to that length. Any value may be used for the padding. The padding MUST be added at the end of the bit string. The parity operation is applied across the protected data of the corresponding packets. The generated FEC bit of that level is then appended to the payload of the FEC packet. 8. Recovery Procedures The FEC packets allow end systems to recover from the loss of media packets. All of the header fields of the missing packets, including CSRC lists, extensions, padding bits, marker and payload type, are recoverable. This section describes the procedure for performing this recovery. Recovery requires two distinct operations. The first determines which packets (media and FEC) must be combined in order to recover a missing packet. Once this is done, the second step is to actually reconstruct the data. The second step MUST be performed as described below. The first step MAY be based on any algorithm chosen by the implementer. Different algorithms result in a tradeoff between complexity and the ability to recover missing packets, if possible. 8.1. Reconstruction in the Baseline Mode Let T be the list of packets (FEC and media) which can be combined to recover some media packet xi. The procedure is as follows: 1. For the media packets in T, compute the bit string as described in the protection operation in the baseline mode of the previous section. 2. For the FEC packet in T, compute the bit string by concatenating the first 64 bits of the FEC header with the FEC payload. Adam H. Li [Page 13]
I-Draft An RTP Payload Format for Generic FEC October 2003 3. If any of the bit strings generated from the media packets are shorter than the bit string generated from the FEC packet, pad them to be the same length as the bit string generated from the FEC. The padding of value 0 MUST be added at the end of the bit string. 4. Perform the exclusive-or (parity) operation across the bit strings, resulting in a recovery bit string. 5. Create a new packet with the standard 12 byte RTP header and no payload. 6. Set the version of the new packet to 2. Skip the first two bits in the recovery bit string. 7. Set the Padding bit in the new packet to the next bit in the recovery bit string. 8. Set the Extension bit in the new packet to the next bit in the recovery bit string. 9. Set the CC field to the next four bits in the recovery bit string. 10. Set the marker bit in the new packet to the next bit in the recovery bit string. 11. Set the payload type in the new packet to the next 7 bits in the recovery bit string. 12. Set the SN field in the new packet to xi. Skip the next 16 bits in the recovery bit string. 13. Set the TS field in the new packet to the next 32 bits in the recovery bit string. 14. Take the next 16 bits of the recovery bit string. Whatever unsigned integer this represents (assuming network-order), take that many bytes from the recovery bit string and append them to the new packet. This represents the CSRC list, extension, payload, and padding. 15. Set the SSRC of the new packet to the SSRC of the media stream it's protecting. This procedure will completely recover both the header and payload of an RTP packet. Adam H. Li [Page 14]
I-Draft An RTP Payload Format for Generic FEC October 2003 8.2. Reconstruction in the Extended Mode 8.2.1. Extended Mode - Reconstruction of Level 0 The reconstruction in the extended mode is the same as the reconstruction in the baseline mode, except that in step 14, instead of copying a number of bytes as recovered from the length recovery field, a string of number of bytes of the protection length 0 is copied to the new packet. This procedure will recover both the header and payload of an RTP packet up to the Protection Length of level 0. 8.2.2. Extended Mode - Reconstruction of Level 1 and Higher Let T be the list of packets (FEC and media) which can be combined to recover some media packet xi at certain protection level. The procedure is as follows: 1. For the media packet in T, get the protection length of that level. Copy the data of the that protection level (data of the length read following the level header) to the bit strings. 2. If any of the bit strings generated from the media packets are shorter than the Protection Length of the current level, pad them to that length. The padding MUST be added at the end of the bit string, and MUST be of the same value as used in the process of generating the FEC packets. 3. Perform the exclusive-or (parity) operation across the bit strings, resulting in a recovery bit string. The data protected at lower protection level is almost always recoverable if the higher level protected data is recoverable. This procedure (together with the procedure for the lower protection levels) will usually recover both the header and payload of an RTP packet up to the Protection Length of the current level. 9. Examples Consider 4 media packets to be sent, A, B, C and D, from SSRC 2. Their sequence numbers are 8, 9, 10 and 11, respectively, and have timestamps of 3, 5, 7 and 9, respectively. Packet A and C uses payload type 11, and packet B and D uses payload type 18. Packet A has 200 bytes of payload, packet B 140, packet C 100 and packet D 340. Packet A and C have their marker bit set. Adam H. Li [Page 15]
I-Draft An RTP Payload Format for Generic FEC October 2003 9.1. A Baseline Mode Example We can protect the four payload packet with one FEC packet in the baseline mode. The scheme is as shown in Figure 5. +-------------------+ : Packet A | | : +-------------+-----+ : Packet B | | : +---------+---+ : Packet C | | : +---------+-----------------------+ Packet D | | +---------------------------------+ : +---------------------------------+ Packet FEC | | +---------------------------------+ : : :<------------- L0 -------------->: Figure 5 FEC scheme in the baseline mode An FEC packet is generated from these four packets. We assume that payload type 127 is used to indicate an FEC packet. The resulting RTP header is shown in Figure 6. The FEC header in the FEC packet is shown in Figure 7. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1 0|0|0|0 0 0 0|0|1 1 1 1 1 1 1|0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Version: 2 Padding: 0 Extension: 0 Marker: 0 PT: 127 SN: 1 TS: 9 SSRC: 2 Figure 6: RTP Header of FEC for Packets A, B, C and D Adam H. Li [Page 16]
I-Draft An RTP Payload Format for Generic FEC October 2003 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|0|0|0|0 0 0 0|0|0 0 0 0 0 0 0|0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 1 0 1 1 1 0 1 0 0|0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ E: 0 [FEC in baseline mode] L: 0 [short 16-bit mask] P rec.: 0 [0 XOR 0 XOR 0 XOR 0] X rec.: 0 [0 XOR 0 XOR 0 XOR 0] CC rec.: 0 [0 XOR 0 XOR 0 XOR 0] M rec.: 0 [0 XOR 0 XOR 0 XOR 0] PT rec.: 0 [11 XOR 18 XOR 11 XOR 18] SN base: 8 [min(8,9,10,11)] TS rec.: 8 [3 XOR 5 XOR 7 XOR 9] len. rec.: 372 [200 XOR 140 XOR 100 XOR 340] mask: 15 [with Packet 8, 9, 10, and 11 marked] Figure 7: FEC Header of ULP Packet (baseline mode) 9.2. An Extended Mode Example With One Protection Level Suppose we want to protect the data of length L0 = 70 bytes of them at the beginning of these packets, as illustrated in Figure 8 below. +------:------------+ Packet A | : | +------:------+-----+ Packet B | : | +------:--+---+ Packet C | : | +------:--+-----------------------+ Packet D | : | +------:--------------------------+ : +------+ Packet FEC | | +------+ : : :<-L0->: Figure 8 ULP FEC scheme with only protection level 0 Adam H. Li [Page 17]
I-Draft An RTP Payload Format for Generic FEC October 2003 The resulting FEC packet will have the same RTP header as shown in Figure 6. The FEC header in the FEC packet is shown in Figure 9. It is the same as the FEC header in Figure 7, except that the E bit is set to 1. The ULP header level 0 present before the parity data of level 0 in the FEC packet is as shown in Figure 10. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1|0|0|0|0 0 0 0|0|0 0 0 0 0 0 0|0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 1 0 1 1 1 0 1 0 0|0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ E: 1 [FEC in extended mode] L: 0 [short 16-bit mask] P rec.: 0 [0 XOR 0 XOR 0 XOR 0] X rec.: 0 [0 XOR 0 XOR 0 XOR 0] CC rec.: 0 [0 XOR 0 XOR 0 XOR 0] M rec.: 0 [0 XOR 0 XOR 0 XOR 0] PT rec.: 0 [11 XOR 18 XOR 11 XOR 18] SN base: 8 [min(8,9,10,11)] TS rec.: 8 [3 XOR 5 XOR 7 XOR 9] len. rec.: 372 [200 XOR 140 XOR 100 XOR 340] mask: 15 [with Packet 8, 9, 10, and 11 marked] Figure 9: FEC Header of ULP Packet (baseline mode) 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ L0: 70 The payload length for level 0 is 70 bytes. Figure 10: ULP Level Header (Level 0) Adam H. Li [Page 18]
I-Draft An RTP Payload Format for Generic FEC October 2003 9.3. An Extended Mode Example With Two Protection Levels A more complex example is to use extended mode FEC at two levels. The level 0 FEC will provide greater protection to the beginning part of the payload packets. The level 1 FEC will apply additional protection to the rest of the packets. This is illustrated in Figure 11. In this example, we take L0 = 70 and L1 = 90. +------:--------:---+ Packet A | : : | +------:------+-:---+ Packet B | : | : +------:--+---+ : : : +------+ : ULP #1 | | : +------+ : : : +------:--+ : Packet C | : | : +------:--+-----:-----------------+ Packet D | : : | +------:--------:-----------------+ : : +------:--------+ ULP #2 | : | +------:--------+ : : : :<-L0->:<--L1-->: Figure 11 ULP FEC scheme with protection level 0 and level 1 This will result in two extended mode FEC packets - #1 and #2. The resulting ULP FEC packet #1 will have the RTP header as shown in Figure 12. The FEC header for ULP FEC packet #1 will be as shown in Figure 13. The level 0 ULP header for #1 will be shown in Figure 14. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1 0|0|0|0 0 0 0|1|1 1 1 1 1 1 1|0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Version: 2 Adam H. Li [Page 19]
I-Draft An RTP Payload Format for Generic FEC October 2003 Padding: 0 Extension: 0 Marker: 1 PT: 127 SN: 1 TS: 5 SSRC: 2 Figure 12: RTP Header of FEC Packet #1 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1|0|0|0|0 0 0 0|0|0 0 1 1 0 0 1|0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0|0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ E: 1 [FEC in extended mode] L: 0 [short 16-bit mask] P rec.: 0 [0 XOR 0 XOR 0 XOR 0] X rec.: 0 [0 XOR 0 XOR 0 XOR 0] CC rec.: 0 [0 XOR 0 XOR 0 XOR 0] M rec.: 0 [0 XOR 0 XOR 0 XOR 0] PT rec.: 25 [11 XOR 18] SN base: 8 [min(8,9)] TS rec.: 6 [3 XOR 5] len. rec.: 68 [200 XOR 140] mask: 3 [Packet 8 and 9 marked] Figure 13: FEC Header of ULP FEC Packet #1 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ L0: 70 The payload length for level 0 is 70 bytes. Figure 14: ULP Level Header (Level 0) for FEC Packet #1 The resulting FEC packet #2 will have the RTP header as shown in Figure 15. The FEC header for FEC packet #2 will be as shown in Adam H. Li [Page 20]
I-Draft An RTP Payload Format for Generic FEC October 2003 Figure 16. The level 0 ULP header for #2 will be shown in Figure 17. The level 1 ULP header for #2 will be shown in Figure 18. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1 0|0|0|0 0 0 0|1|1 1 1 1 1 1 1|0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Version: 2 Padding: 0 Extension: 0 Marker: 1 PT: 127 SN: 2 TS: 9 SSRC: 2 Figure 15: RTP Header of FEC Packet #2 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1|0|0|0|0 0 0 0|0|0 0 1 1 0 0 1|0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 1 0 0 1 1 0 0 0 0|0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ E: 1 [FEC in extended mode] L: 0 [short 16-bit mask] P rec.: 0 [0 XOR 0 XOR 0 XOR 0] X rec.: 0 [0 XOR 0 XOR 0 XOR 0] CC rec.: 0 [0 XOR 0 XOR 0 XOR 0] M rec.: 0 [0 XOR 0 XOR 0 XOR 0] PT rec.: 25 [11 XOR 18] SN base: 8 [min(8,9,10,11)] TS rec.: 14 [7 XOR 9] len. rec.: 304 [100 XOR 340] mask: 12 [Packet 10 and 11 marked] Figure 16: FEC Header of FEC Packet #2 Adam H. Li [Page 21]
I-Draft An RTP Payload Format for Generic FEC October 2003 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 0| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ L0: 70 The payload length for level 0 is 70 bytes. Figure 17: ULP Level Header (Level 0) for FEC Packet #2 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 0 0 0 0 0 0 1 0 1 1 0 1 0|0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ L1: 90 mask: 15 [Packet 8, 9, 10, and 11 marked] The payload length for level 1 is 90 bytes. Figure 18: ULP Level Header (Level 1) for FEC Packet #2 10. Security and Congestion Considerations The use of FEC has implications on the usage and changing of keys for encryption. As the FEC packets do consist of a separate stream, there are a number of combinations on the usage of encryption. These include: o The FEC stream may be encrypted, while the media stream is not. o The media stream may be encrypted, while the FEC stream is not. o The media stream and FEC stream are both encrypted, but using different keys. o The media stream and FEC stream are both encrypted, but using the same key. The first three of these would require all application level signaling protocols used to be aware of the usage of FEC, and to thus exchange keys and negotiate encryption usage on the media and FEC streams separately. In the final case, no such additional mechanisms are needed. The first two cases present a layering violation, as ULP FEC packets should be treated no differently than other RTP packets. Encrypting just one stream may also make certain Adam H. Li [Page 22]
I-Draft An RTP Payload Format for Generic FEC October 2003 known-plaintext attacks possible. For these reasons, applications utilizing encryption SHOULD encrypt both streams. The changing of encryption keys is another crucial issue needs to be addressed. Consider the case where two packets a and b are sent along with the FEC packet that protects them. The keys used to encrypt a and b are different, so which key should be used to decode the FEC packet? In general, old keys need to be cached, so that when the keys change for the media stream, the old key can be used until it is determined that the key has changed for the ULP FEC packets as well. Another issue with the use of FEC is its impact on network congestion. In many situations, the packet loss in the network is induced by congestions. In such scenarios, adding FEC when encountering increasing network losses should be avoided. If it is used on a widespread basis, this can result in increased congestion and eventual congestion collapse. The applications may include stronger protections while at the same time reduce the bandwidth for the payload packets. In any event, implementations MUST NOT substantially increase the total amount of bandwidth in use (including the payload and the FEC) as network losses increase. 11. Indicating FEC Usage in SDP FEC packets contain RTP packets with dynamic payload type values. In addition, the FEC packets can be sent on separate multicast groups or separate ports from the media. The FEC can even be carried in packets containing media using the redundant encoding payload format [5]. These configuration options MUST be indicated out of band. This section describes how this can be accomplished using the Session Description Protocol (SDP), specified in RFC 2327 [6]. 11.1. FEC as a Separate Stream In the first case, the FEC packets are sent as a separate stream. This means that they can be sent on a different port and/or multicast group from the media. When this is done, several pieces of information must be conveyed: o The address and port where the FEC is being sent to o The payload type number for the FEC o Which media stream the FEC is protecting The payload type number for the FEC is conveyed in the m line of the media it is protecting, listed as if it were another valid encoding for the stream. There is no static payload type assignment for FEC, so dynamic payload type numbers MUST be used. The binding to the Adam H. Li [Page 23]
I-Draft An RTP Payload Format for Generic FEC October 2003 number is indicated by an rtpmap attribute. The name used in this binding is "ulpfec". The presence of the payload type number in the m line of the media it is protecting does not mean the FEC is sent to the same address and port as the media. Instead, this information is conveyed through an fmtp attribute line. The presence of the FEC payload type on the m line of the media serves only to indicate which stream the FEC is protecting. The format for the fmtp line for FEC is: a=fmtp:<number> <port> <network type> <addresss type> <connection address> where 'number' is the payload type number present in the m line. Port is the port number where the FEC is sent to. The remaining three items - network type, address type, and connection address - have the same syntax and semantics as the c line from SDP. This allows the fmtp line to be partially parsed by the same parser used on the c lines. Note that since FEC cannot be hierarchically encoded, the <number of addresses> parameter MUST NOT appear in the connection address. The following is an example SDP for FEC: v=0 o=hamming 2890844526 2890842807 IN IP4 128.97.90.168 s=ULP FEC Seminar c=IN IP4 224.2.17.12/127 t=0 0 m=audio 49170 RTP/AVP 0 78 a=rtpmap:78 ulpfec/8000 a=fmtp:78 49172 IN IP4 224.2.17.12/127 m=video 51372 RTP/AVP 31 79 a=rtpmap:79 ulpfec/8000 a=fmtp:79 51372 IN IP4 224.2.17.13/127 The presence of two m lines in this SDP indicates that there are two media streams - one audio and one video. The media format of 0 indicates that the audio uses PCM, and is protected by FEC with payload type number 78. The FEC is sent to the same multicast group and TTL as the audio, but on a port number two higher (49172). The video is protected by FEC with payload type number 79. The FEC appears on the same port as the video (51372), but on a different multicast address. 11.2. Use with Redundant Encoding When the FEC stream is being sent as a secondary codec in the redundant encoding format, this must be signaled through SDP. To do this, the procedures defined in RFC 2198 [5] are used to signal the Adam H. Li [Page 24]
I-Draft An RTP Payload Format for Generic FEC October 2003 use of redundant encoding. The FEC payload type is indicated in the same fashion as any other secondary codec. An rtpmap attribute MUST be used to indicate a dynamic payload type number for the FEC packets. The FEC MUST protect only the main codec. In this case, the fmtp attribute for the FEC MUST NOT be present. For example: m=audio 12345 RTP/AVP 121 0 5 100 a=rtpmap:121 red/8000/1 a=rtpmap:100 ulpfec/8000 a=fmtp:121 0/5/100 This SDP indicates that there is a single audio stream, which can consist of PCM (media format 0) , DVI (media format 5), the redundant encodings (indicated by media format 121, which is bound to read through the rtpmap attribute), or FEC (media format 100, which is bound to ulpfec through the rtpmap attribute). Although the FEC format is specified as a possible coding for this stream, the FEC MUST NOT be sent by itself for this stream. Its presence in the m line is required only because non-primary codecs must be listed here according to RFC 2198. The fmtp attribute indicates that the redundant encodings format can be used, with DVI as a secondary coding and FEC as a tertiary encoding. 11.3. Usage with RTSP RTSP [7] can be used to request FEC packets to be sent as a separate stream. When SDP is used with RTSP, the Session Description does not include a connection address and port number for each stream. Instead, RTSP uses the concept of a "Control URL". Control URLs are used in SDP in two distinct ways. 1. There is a single control URL for all streams. This is referred to as "aggregate control". In this case, the fmtp line for the FEC stream is omitted. 2. There is a Control URL assigned to each stream. This is referred to as "non-aggregate control". In this case, the fmtp line specifies the Control URL for the stream of FEC packets. The URL may be used in a SETUP command by an RTSP client. The format for the fmtp line for FEC with RTSP and non-aggregate control is: a=fmtp:<number> <control URL> where 'number' is the payload type number present in the m line. Control URL is the URL used to control the stream of FEC packets. Note that the Control URL does not need to be an absolute URL. The rules for converting a relative Control URL to an absolute URL are given in RFC 2326, Section C.1.1. Adam H. Li [Page 25]
I-Draft An RTP Payload Format for Generic FEC October 2003 12. MIME Registrations Four new MIME sub-type as described in this section is to be registered. 12.1. Registration of audio/ulpfec To: ietf-types@iana.org Subject: Registration of MIME media type audio/ulpfec MIME media type name: audio MIME subtype name: ulpfec Required parameters: none Note that it is mandated that RTP payload formats without a defined rate must define a rate parameter as part of their MIME registration. The payload format for ULP FEC does not specify a rate parameter. However, the rate for ULP FEC data is equal to the rate of the media data it protects. Optional parameters: none Typical optional parameters [8], such as the number of channels, and the duration of audio per packet, do not apply to ULP FEC data. The number of channels is effectively the same as the media data it protects; the same is true for the duration of audio per packet. Encoding considerations: This format is only defined for transport within the Real Time Transport protocol (RTP) [3]. Its transport within RTP is fully specified with RFC xxxx. Security considerations: the same security considerations apply to these MIME registrations as to the payloads for them, as detailed in RFC xxxx. Interoperability considerations: none Published specification: RFC xxxx. Applications which use this media type: Audio and video streaming tools which seek to improve resiliency to loss by sending additional data with the media stream. Additional information: none Person & email address to contact for further information: Adam H. Li [Page 26]
I-Draft An RTP Payload Format for Generic FEC October 2003 Adam Li Department of Electrical Engineering University of California Los Angeles, CA 90095 adamli@icsl.ucla.edu Intended usage: COMMON Author/Change controller: This registration is part of the IETF registration tree. RTP and SDP Issues: Usage of this format within RTP and the Session Description Protocol (SDP) [6] are fully specified within Section 10 of RFC xxxx. 12.2. Registration of video/ulpfec To: ietf-types@iana.org Subject: Registration of MIME media type video/ulpfec MIME media type name: video MIME subtype name: ulpfec Required parameters: none Note that it is mandated that RTP payload formats without a defined rate must define a rate parameter as part of their MIME registration. The payload format for ULP FEC does not specify a rate parameter. However, the rate for ULP FEC data is equal to the rate of the media data it protects. Optional parameters: none Typical optional parameters [8], such as the number of channels, and the duration of audio per packet, do not apply to ULP FEC data. The number of channels is effectively the same as the media data it protects; the same is true for the duration of video per packet. Encoding considerations: This format is only defined for transport within the Real Time Transport protocol (RTP) [3]. Its transport within RTP is fully specified with RFC xxxx. Security considerations: the same security considerations apply to these MIME registrations as to the payloads for them, as detailed in RFC xxxx. Interoperability considerations: none Published specification: RFC xxxx. Adam H. Li [Page 27]
I-Draft An RTP Payload Format for Generic FEC October 2003 Applications which use this media type: Audio and video streaming tools which seek to improve resiliency to loss by sending additional data with the media stream. Additional information: none Person & email address to contact for further information: Adam Li Department of Electrical Engineering University of California Los Angeles, CA 90095 adamli@icsl.ucla.edu Intended usage: COMMON Author/Change controller: This registration is part of the IETF registration tree. RTP and SDP Issues: Usage of this format within RTP and the Session Description Protocol (SDP) [6] are fully specified within Section 10 of RFC xxxx. 12.3. Registration of text/ulpfec To: ietf-types@iana.org Subject: Registration of MIME media type text/ulpfec MIME media type name: text MIME subtype name: ulpfec Required parameters: none Note that it is mandated that RTP payload formats without a defined rate must define a rate parameter as part of their MIME registration. The payload format for ULP FEC does not specify a rate parameter. However, the rate for ULP FEC data is equal to the rate of the media data it protects. Optional parameters: none Typical optional parameters [8], such as the number of channels, and the duration of audio per packet, do not apply to ULP FEC data. The number of channels is effectively the same as the media data it protects; the same is true for the duration of video per packet. Encoding considerations: This format is only defined for transport within the Real Time Transport protocol (RTP) [3]. Its transport within RTP is fully specified with RFC xxxx. Adam H. Li [Page 28]
I-Draft An RTP Payload Format for Generic FEC October 2003 Security considerations: the same security considerations apply to these MIME registrations as to the payloads for them, as detailed in RFC xxxx. Interoperability considerations: none Published specification: RFC xxxx. Applications which use this media type: Audio, video and text streaming tools which seek to improve resiliency to loss by sending additional data with the media stream. Additional information: none Person & email address to contact for further information: Adam Li Department of Electrical Engineering University of California Los Angeles, CA 90095 adamli@icsl.ucla.edu Intended usage: COMMON Author/Change controller: This registration is part of the IETF registration tree. RTP and SDP Issues: Usage of this format within RTP and the Session Description Protocol (SDP) [6] are fully specified within Section 10 of RFC xxxx. 12.4. Registration of application/ulpfec To: ietf-types@iana.org Subject: Registration of MIME media type application/ulpfec MIME media type name: application MIME subtype name: ulpfec Required parameters: none Note that it is mandated that RTP payload formats without a defined rate must define a rate parameter as part of their MIME registration. The payload format for ULP FEC does not specify a rate parameter. However, the rate for ULP FEC data is equal to the rate of the media data it protects. Optional parameters: none Adam H. Li [Page 29]
I-Draft An RTP Payload Format for Generic FEC October 2003 Typical optional parameters [8], such as the number of channels, and the duration of audio per packet, do not apply to ULP FEC data. The number of channels is effectively the same as the media data it protects; the same is true for the duration of video per packet. Encoding considerations: This format is only defined for transport within the Real Time Transport protocol (RTP) [3]. Its transport within RTP is fully specified with RFC xxxx. Security considerations: the same security considerations apply to these MIME registrations as to the payloads for them, as detailed in RFC xxxx. Interoperability considerations: none Published specification: RFC xxxx. Applications which use this media type: Audio/video streaming tools and other applications which seek to improve resiliency to loss by sending additional data with the media stream. Additional information: none Person & email address to contact for further information: Adam Li Department of Electrical Engineering University of California Los Angeles, CA 90095 adamli@icsl.ucla.edu Intended usage: COMMON Author/Change controller: This registration is part of the IETF registration tree. RTP and SDP Issues: Usage of this format within RTP and the Session Description Protocol (SDP) [6] are fully specified within Section 10 of RFC xxxx. 13. Application Statement The generic FEC algorithm specified in this document is designed to deal with any type of packet loss occurring in transmission. This FEC algorithm is fully interoperable between the hosts that are FEC- capable and those that are not. Since the media payload is not altered and the protection is sent as additional information, the receivers that are unaware of the generic FEC as specified in this document can simply ignore the additional FEC information and process the main media payload. This interoperability is particularly important for backward compatibility with existing Adam H. Li [Page 30]
I-Draft An RTP Payload Format for Generic FEC October 2003 hosts, and also in the scenario where many different hosts need to communicate with each other at the same time, such as during multicast. The generic FEC algorithm specified in this document is also a generic protection algorithm with the following features: (1) it is independent of the nature of the media being protected, whether that media is audio, video, or otherwise, (2) it is flexible enough to support a wide variety of FEC mechanisms and settings, (3) it is designed for adaptivity, so that the FEC parameters can be modified easily without resorting to out of band signaling, and (4) it supports a number of different mechanisms for transporting the FEC packets. The extended mode of FEC (also called ULP) further generalizes and extends the generic FEC algorithm here, and provides user with Unequal Error Protection capabilities. Some other algorithms may also provide the Unequal Error Protection capabilities thought other means. For example, an Unequal Erasure Protection (UXP) scheme has been proposed in the AVT Working Group in "An RTP Payload Format for Erasure-Resilient Transmission of Progressive Multimedia Streams". The UXP scheme applies unequal error protection to the media payloads by interleaving the payload stream to be protected with the additional redundancy information obtained using Reed-Solomon operations. By altering the structure of the protected media payload, the UXP scheme sacrifices the backward compatibility with terminals that do not support UXP. This makes it more difficult to apply UXP when backward compatibility is desired. In the case of ULP, however, the media payload remains un-altered and can always be used by the terminals. The extra protection can simply be ignored if the receiving terminals do not support ULP. At the same time, also because the structure of the media payload is altered in UXP, UXP offers the unique ability to change packet size independent of the original media payload structure and protection applied, and is only subject to the protocol overhead constraint. This property is useful in scenarios when altering the packet size of the media at transport level is desired. Because of the interleaving used in UXP, delays will be introduced at both the encoding and decoding sides. For UXP, all data within a transmission block need to arrive before encoding can begin, and a reasonable number of packets must be received before a transmission block can be decoded. The ULP scheme introduces little delay at the encoding side. On the decoding side, correctly received packets can be delivered immediately. Delay is only introduced in ULP when packet losses occur. Because UXP is an interleaved scheme, the un-recoverable errors occurring in data protected by UXP usually result in a number of Adam H. Li [Page 31]
I-Draft An RTP Payload Format for Generic FEC October 2003 corrupted holes in the payload stream. In ULP, on the other hand, the unrecoverable errors due to packet loss in the bitstream usually appear as contiguous missing pieces at the end of the packets. Depending on the encoding of the media payload stream, many applications may find it easier to parse and extract data from a packet with only a contiguous piece missing at the end than a packet with multiple corrupted holes, especially when the holes are not coincident with the independently decodable fragment boundaries. The exclusive-or (XOR) parity check operation used by ULP is simpler and faster than the more complex operations required by Reed-Solomon codes. This makes ULP more suitable for applications where computational cost is a constraint. As discussed above, both the ULP and the UXP schemes apply unequal error protection to the RTP media stream, but each uses a different technique. Both schemes have their own unique characteristics, and each can be applied to scenarios with different requirements. 14. Acknowledgments The following authors have made significant contributions to this document: Adam H. Li, Fang Liu, John D. Villasenor, Dong-Seek Park, Jeong-Hoon, Yung-Lyul Lee, Jonathan D. Rosenberg, and Henning Schulzrinne. The authors would also like to acknowledge the suggestions from many people, particularly Magnus Westerlund, Stephen Casner, Colin Perkins, Tao Tian, Matthieu Tisserand, Stephen Wenger, Jay Fahlen, and Jeffery Tseng. 15. Bibliography [1] J. Rosenberg and H. Schulzrinne, "An RTP Payload Format for Generic Forward Error Correction," Request for Comments (Proposed Standard) 2733, Internet Engineering Task Force, December 1999. [2] C. Perkins and O. Hodson, "Options for repair of streaming media, "Request for Comments (Informational) 2354, Internet Engineering Task Force, June 1998. [3] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: a transport protocol for real-time applications," Request for Comments (Proposed Standard) 1889, Internet Engineering Task Force, January 1996. [4] S. Bradner, "Key words for use in RFCs to indicate requirement levels," Request for Comments (Best Current Practice) 2119, Internet Engineering Task Force, March 1997. Adam H. Li [Page 32]
I-Draft An RTP Payload Format for Generic FEC October 2003 [5] C. Perkins, I. Kouvelas, O. Hodson, V. Hardman, M. Handley, J.C. Bolot, A. Vega-Garcia, and S. Fosse-Parisis, "RTP Payload for Redundant Audio Data", RFC 2198, September 1997. [6] M. Handley, and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998. [7] H. Schulzrinne, A. Rao, and R. Lanphier, "Real Time Streaming Protocol (RTSP)", RFC 2326, April 1998. [8] S. Casner, and P. Hoschka, "MIME type registration of RTP payload formats", Work in Progress. [9] J. Rosenberg and H. Schulzrine, "Registration of parityfec MIME types", Request for Comments (Proposed Standard) 3009, Internet Engineering Task Force, November 2000. 16. Author's Addresses Adam H. Li Electrical Engineering Department University of California, Los Angeles Los Angeles, CA 90095 USA Phone: +1-310-825-5178 Fax : +1-310-825-7928 EMail: adamli@icsl.ucla.edu Adam H. Li [Page 33]
