Audio Video Transport WG                                          Q. Xie
Internet-Draft                                             J. Schumacher
Updates: RFC 2198                                               Motorola
(if approved)                                            August 21, 2007
Intended status: Standards Track
Expires: February 22, 2008


             Forward-shifted RTP Redundancy Payload Support
                draft-xie-avt-forward-shifted-red-02.txt

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on February 22, 2008.

Copyright Notice

   Copyright (C) The IETF Trust (2007).

Abstract

   This document defines a simple enhancement to RFC 2198 to support RTP
   sessions with forward-shifted redundant encodings, i.e., redundant
   data is sent before the corresponding primary data.  Forward-shifted
   redundancy can be used to conceal losses of a large number of
   consecutive media frames (e.g., consecutive loss of seconds or even
   tens of seconds of media).



Xie & Schumacher        Expires February 22, 2008               [Page 1]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


Table of Contents

   1.  Conventions  . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
     2.1.  Sending Redundant Data Inband vs. Out-of-band  . . . . . .  3
   3.  Allowing Forward-shifted Redundant Data  . . . . . . . . . . .  4
   4.  Registration of Media Type "fwdred"  . . . . . . . . . . . . .  5
   5.  Mapping MIME Parameters into SDP . . . . . . . . . . . . . . .  6
   6.  Usage in Offer/Answer  . . . . . . . . . . . . . . . . . . . .  7
   7.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . .  7
   8.  Security Considerations  . . . . . . . . . . . . . . . . . . .  7
   9.  Normative References . . . . . . . . . . . . . . . . . . . . .  7
   Appendix A.  Anti-shadow Loss Concealment Using
                Forward-shifted Redundancy  . . . . . . . . . . . . .  8
     A.1.  Sender Side Operations . . . . . . . . . . . . . . . . . .  8
     A.2.  Receiver Side Operations . . . . . . . . . . . . . . . . . 10
       A.2.1.  Normal Mode Operation  . . . . . . . . . . . . . . . . 10
       A.2.2.  Anti-shadow Mode Operation . . . . . . . . . . . . . . 11
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12
   Intellectual Property and Copyright Statements . . . . . . . . . . 13































Xie & Schumacher        Expires February 22, 2008               [Page 2]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


1.  Conventions

   The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD,
   SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL, when
   they appear in this document, are to be interpreted as described in
   RFC 2119 [1].


2.  Introduction

   This document defines a simple enhancement to RFC 2198 [2] to support
   RTP sessions with forward-shifted redundant encodings, i.e.,
   redundant data is sent before the corresponding primary data.

   Forward-shifted redundancy can be used to conceal losses of a large
   number of consecutive media frames (e.g., consecutive loss of seconds
   of media).  Such capability is highly desirable, especially in
   wireless mobile communication environments where the radio signal to
   a mobile wireless media receiver can be temporarily blocked by tall
   buildings, mountains, tunnels, etc.  In other words, the receiver
   enters into a shadow of the radio coverage.  No new data will be
   received when the receiver is in a shadow.

   In some extreme cases, the receiver may have to spend seconds or even
   tens of seconds in a shadow.  The traditional backward-shifted
   redundant encoding scheme (i.e., redundant data is sent after the
   primary data), as currently supported by RFC 2198 [2], is known to be
   ineffective in dealing with such consecutive frame losses.

   In contrast, the forward-shifted redundancy, when used in combination
   with the anti-shadow loss management at the receiver (as described in
   Appendix A), can effectively prevent service interruptions when a
   mobile receiver runs into such a shadow.

2.1.  Sending Redundant Data Inband vs. Out-of-band

   Regardless of the direction of time shift (e.g., forward-shifting or
   backward-shifting as in RFC 2198) or the encoding scheme (e.g., FEC,
   or non-FEC), there is always the option of sending the redundant data
   and the primary data either in the same RTP session (i.e., inband) or
   in separate RTP sessions (i.e., out-of-band).  There are pros and
   cons for either approach, as outlined below.

   Inband Approach:







Xie & Schumacher        Expires February 22, 2008               [Page 3]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


   o  Pro: A single RTP session is faster to setup and easier to manage.

   o  Pro: A single RTP session presents a simpler problem for NAT/
      firewall traverse.

   o  Pro: Less overall overhead - one RTP/UDP/IP overhead.

   o  Con: Lack of flexibility - difficult for middle boxes such as
      gateways to add/remove the redundant data.

   o  Con: Need more specification - special payload formats need to be
      defined to carry the redundant data inband.

   Out-of-band Approach:

   o  Pro: Flexibility - redundant data can be more easily added,
      removed, or replaced by a middle box such as a gateway.

   o  Pro: Little or none specification - no new payload format is
      needed.

   o  Con: Multiple RTP sessions may take longer to setup and more
      complexity to manage.

   o  Con: Multiple RTP sessions NAT/firewall traverse are harder to
      address.

   o  Con: Bigger overall overhead - more than one RTP/UDP/IP overhead.

   It is noteworthy that the specification of inband payload formats
   such as this and RFC 2198 does not preclude a deployment from using
   the out-of-band approach.  Rather, it gives the deployment the choose
   to use whichever approach deemed most beneficiary under a given
   circumstance.


3.  Allowing Forward-shifted Redundant Data

   In RFC 2198, the timestamp offset in the additional header
   corresponding to a redundant block is defined as a 14 bits unsigned
   offset of timestamp relative to timestamp given in the RTP header.
   As stated in RFC 2198:

     "The use of an unsigned offset implies that redundant data must be
     sent after the primary data, and is hence a time to be subtracted
     from the current timestamp to determine the timestamp of the data
     for which this block is the redundancy."




Xie & Schumacher        Expires February 22, 2008               [Page 4]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


   This effectively prevents RFC 2198 from being used to support
   forward-shifted redundant blocks.

   In order to support the use of forward-shifted redundant blocks, the
   media type "fwdred" which allows an optional MIME parameters,
   "forwardshift", is introduced for indicating the capability and
   willingness of using forward-shifted redundancy and the base value of
   timestamp forward-shifting.  The base value of "forwardshift" is an
   integer equal or greater than '0'.

   In an RTP session which uses forward-shifted redundant encodings, the
   timestamp of a redundant block in a received RTP packet is determined
   as follows:

     timestamp of redundant block = timestamp in RTP header
                         - timestamp offset in additional header
                         + forward shift base value


4.  Registration of Media Type "fwdred"

   (The definition is based on media type "red" defined in RFC 2198 [2],
   with the addition of the optional "forwardshift" parameter.)

   Type name:  audio

   Subtype names:  fwdred

   Required parameters:  none

   Optional parameters:

      forwardshift:  An unsigned integer can be specified as value.

         If this parameter is present with a value greater than '0', it
         indicates that the sender of this parameter will use forward
         shifting with a base value as specified when sending out
         redundant data.

         If this parameter is absent or present with a value of '0', it
         indicates that the sender of this parameter will not use
         forward shifting when sending its redundant data.  I.e., the
         sender will have the same behaviors as define in RFC 2198.








Xie & Schumacher        Expires February 22, 2008               [Page 5]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


   Encoding considerations:
      This media type is framed binary data (see RFC 4288, Section 4.8)
      and is only defined for transfer of RTP redundant data frames
      specified in RFC 2198.

   Security considerations:  See Section 6 "Security Considerations" of
      RFC 2198.

   Interoperability considerations:  None.

   Published specification:
      RTP redundant data frame format is specified in RFC 2198.

   Applications that use this media type:
      It is expected that real-time audio/video applications that want
      protection against losses of a large number of consecutive frames
      will be interested in using this type.

   Additional information:  none

   Person & email address to contact for further information:
      Qiaobing Xie <Qiaobing.Xie@motorola.com>

   Intended usage:  COMMON

   Restrictions on usage:
      This media type depends on RTP framing, and hence is only defined
      for transfer via RTP (RFC 3550 [3]).  Transfer within other
      framing protocols is not defined at this time.

   Author:
      Qiaobing Xie

   Change controller:
      IETF Audio/Video Transport working group delegated from the IESG.


5.  Mapping MIME Parameters into SDP

   The information carried in the MIME media type specification has a
   specific mapping to fields in the Session Description Protocol (SDP)
   [4], which is commonly used to describe RTP sessions.  When SDP is
   used to specify sessions employing the forward-shifted redundant
   format, the mapping is as follows:

   o  The MIME type ("audio") goes in SDP "m=" as the media name.





Xie & Schumacher        Expires February 22, 2008               [Page 6]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


   o  The MIME subtype ("fwdred") goes in SDP "a=rtpmap" as the encoding
      name.

   o  The optional parameter "forwardshift" goes in the SDP "a=fmtp"
      attribute by copying it directly from the MIME media type string
      as "forwardshift=value".

   Example of usage of indicating forward-shifted (by 5.1 sec)
   redundancy:

     m=audio 12345 RTP/AVP 121 0 5
     a=rtpmap:121 fwdred/8000/1
     a=fmtp:121 0/5 forwardshift=40800

   Example of usage of indicating sending redundancy without forward-
   shifting (equivalent to RFC 2198):

     m=audio 12345 RTP/AVP 121 0 5
     a=rtpmap:121 fwdred/8000/1
     a=fmtp:121 0/5 forwardshift=0


6.  Usage in Offer/Answer

   The optional "forwardshift" SDP parameter specified in this document
   is declarative, and all reasonable values are expected to be
   supported.


7.  IANA Considerations

   The registration of the new MIME subtype "fwdred", as described in
   Section 4, is required.


8.  Security Considerations

   See Section 6 "Security Considerations" of RFC 2198 [2].


9.  Normative References

   [1]  Bradner, S., "Key words for use in RFCs to Indicate Requirement
        Levels", BCP 14, RFC 2119, March 1997.

   [2]  Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., Handley, M.,
        Bolot, J., Vega-Garcia, A., and S. Fosse-Parisis, "RTP Payload
        for Redundant Audio Data", RFC 2198, September 1997.



Xie & Schumacher        Expires February 22, 2008               [Page 7]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


   [3]  Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson,
        "RTP: A Transport Protocol for Real-Time Applications",
        RFC 3550, July 2003.

   [4]  Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
        Description Protocol", RFC 4566, July 2006.


Appendix A.  Anti-shadow Loss Concealment Using Forward-shifted
             Redundancy

   It is not unusual in a wireless mobile communication environment
   where the radio signal to a mobile wireless media receiver can be
   temporarily blocked by tall buildings, mountains, tunnels, etc. for a
   period of time.  In other words, the receiver enters into a shadow of
   the radio coverage.  When the receiver is in such a shadow no new
   data will be received.  In some extreme cases, the receiver may have
   to spend seconds or even tens of seconds in such a shadow.

   Without special design considerations to compensate the loss of data
   due to shadowing, a mobile user may experience an unacceptable level
   of service interruptions.  And traditional redundant encoding schemes
   (including RFC 2198 and most FEC schemes) are known to be ineffective
   in dealing with such losses of consecutive frames.

   However, the employment of forward-shifted redundancy, in combination
   with the anti-shadow loss concealment at the receiver, as described
   here, can effectively prevent service interruptions due to the effect
   of shadowing.

A.1.  Sender Side Operations

   For anti-shadow loss management, the RTP sender simply adds a
   forward-shifted redundant stream (called anti-shadow or AS stream)
   while transmitting the primary media stream.  The amount of forward-
   shifting, which should remain constant for the duration of the
   session, will determine the maximal length of shadows that can be
   completely concealed at the receiver, as explained below.

   Except for the fast that it is forward-shifted relative to the
   primary stream (i.e., the redundant data is sent ahead of the
   corresponding primary data), the design decision and trade-offs on
   the quality, encoding, bandwidth overhead, etc. of the redundant
   stream is not different from the traditional RTP payload redundant
   scheme.

   The following diagram illustrates a segment of the transmission
   sequence of a forward-shifted redundant RTP session, in which the AS



Xie & Schumacher        Expires February 22, 2008               [Page 8]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


   stream is forward-shifted by 155 frames.  If, for simplicity here, we
   assume the value of timestamp is incremented by 1 between two
   consecutive frames, this forward-shifted redundancy can then be
   indicated with:

       forwardshift=155

   and the setting of timestamp offset to 0 in all the additional
   headers.  This can mean a 3.1 second of forward shifting if each
   frame represents 20 ms of original media,

                      Primary stream    AS stream

          ...               |                |
                            v                v
          Pkt k+8        [ 111 ]          [ 266 ]
                            |                |
                            v                v
          Pkt k+7        [ 110 ]          [ 265 ]
                            |                |
                            v                v
      ^   Pkt k+6        [ 109 ]          [ 264 ]
      |                     |                |
      |                     v                v
          Pkt k+5        [ 108 ]          [ 263 ]
      T                     |                |
      I                     v                v
      M   Pkt k+4        [ 107 ]          [ 262 ]
      E                     |                |
                            v                v
          Pkt k+3        [ 106 ]          [ 261 ]
                            |                |
                            v                v
          Pkt k+2        [ 105 ]          [ 260 ]
                            |                |
                            v                v
          Pkt k+1        [ 104 ]          [ 259 ]
                            |                |
                            v                v
          Pkt k          [ 103 ]          [ 258 ]
                            |                |
                            v                v

                              Transmit first

      Figure 1. An example of forward-shifted redundant RTP packet
                              transmission.




Xie & Schumacher        Expires February 22, 2008               [Page 9]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


A.2.  Receiver Side Operations

   The anti-shadow receiver is illustrated in the following diagram.

                                                  +---------+
                                normal mode   sw1 | media   |     media
     Primary stream ======================o___o==>| decoder |===> output
     AS stream     ----                           +---------+     device
                      |             AS mode o
                      |       +---------+   |
                      |       | anti-   |   |
                      ------->| shadow  |----
                              | buffer  |
                              +---------+
                                   |
                                   V
                              expired frames
                              discarded

                 Figure 2. Anti-shadow RTP receiver.

   The anti-shadow receiver operates between two modes - "normal mode"
   and "AS mode".  When the receiver is not in a shadow (it can easily
   tell that if it is still receiving new data), the receiver operates
   in the normal mode.  Otherwise, it operates in the AS mode.

A.2.1.  Normal Mode Operation

   In the normal mode, after receiving a new RTP packet that contains
   the primary data and forward-shifted AS data, the receiver passes the
   primary data directly to the appropriate media decoder for play-out
   (a de-jittering buffer may be used before the play-out, but for
   simplicity we assume none is used here), while the received AS data
   is stored in an anti-shadow buffer.

   Moreover, data stored in the anti-shadow buffer will be continuously
   checked to determine whether it has expired.  If a redundant data in
   the anti-shadow buffer is found to have a timestamp older (i.e.,
   smaller) than that of the last primary frame passed to the media
   decoder, it will be considered expired and be purged from the anti-
   shadowing buffer.

   The following example illustrates the operation of the anti-shadow
   buffer in normal mode.  We use the same assumption as in Figure 1,
   and assume that the initial timestamp value is 103 when the session
   starts.





Xie & Schumacher        Expires February 22, 2008              [Page 10]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


             Timestamp     Timestamp
     Time      being      of media in
    (in ms)  played out    AS buffer         Note
   ------------------------------------------------------------------
     t < 0                 --             (buffer empty)
      ...
     t=0       103         258            (hold 1 AS frame)
     t=20      104         258-259        (hold 2 AS frames)
     t=40      105         258-260        (hold 3 AS frames)

      ...
     t=3080    257         258-412        (full, hold 154 AS frames)
     t=3100    258         259-413        (full, frame 258 purged)
     t=3120    259         260-414        (full, frame 259 purged)
      ...

     t=6240    415         416-570        (always holds 3.08 sec
                                           worth of redundant data)
      ...


    Figure 3. Example of anti-shadow buffer operation in normal mode.


   At the beginning of the session (t=0), the anti-shadow buffer will be
   empty.  When the first primary frame is received, the play-out will
   start immediately, and the first received AS frame is stored in the
   anti-shadow buffer.  And with the arriving of more forward-shifted
   redundant frames, the anti-shadow buffer will gradually be filled up.

   For the example shown in Figure 1, after 3.08 seconds (the amount of
   the forward-shifting minus one frame) from the start of the session,
   the anti-shadow buffer will be full, holding exactly 3.08 seconds
   worth of redundant data, with the oldest frame corresponding to t=3.1
   sec and youngest frame corresponding to t=6.18 sec.

   And it is not difficult to see that in normal mode because of the
   continuous purge of expired frames and the addition of new frames,
   the anti-shadowing buffer will always be full holding the next
   forward-shift amount of redundant frames.

A.2.2.  Anti-shadow Mode Operation

   When the receiver enters a shadow (or any other conditions that
   prevent the receiver from getting new media data), the receiver
   switches to the anti-shadow mode, in which it simply continues the
   play-out from the forward-shifted redundant data stored in the anti-
   shadow buffer.



Xie & Schumacher        Expires February 22, 2008              [Page 11]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


   For the example in Figure 3, if the receiver enters a shadow at
   t=3120, it can continue the play-out by using the forward-shifted
   redundant frames (ts=260-414) from the anti-shadow buffer.  As far as
   the receiver can move out of the shadow by t=6240, there will be no
   service interruption.

   When the shadow condition ends (meaning new data starts to arrive
   again), the receiver immediately switches back to normal mode of
   operation, playing out from newly arrived primary frames.  And at the
   same time, the arrival of new AS frames will start to re-fill the
   anti-shadow buffer.

   However, if the duration of the shadow is longer than the amount of
   forward-shifting, the receiver will run out of media frames from its
   anti-shadow buffer.  At that point, service interruption will occur.

   Anti-shadow loss concealment described above can be readily applied
   to the streaming of pre-recorded media.  Because of the need of
   generating the forward-shifted anti-shadow redundant stream, to apply
   anti-shadow loss concealment to the streaming of live media will
   require the insertion of a delay equal to or greater than the amount
   of forward-shifting at the source of media.


Authors' Addresses

   Qiaobing Xie
   Motorola, Inc.
   1501 W. Shure Drive, 2-F9
   Arlington Heights, IL  60004
   US

   Phone: +1-847-632-3028
   Email: Qiaobing.Xie@Motorola.com


   Joe Schumacher
   Motorola, Inc.
   1501 W. Shure Drive, 2-B11
   Arlington Heights, IL  60004
   US

   Phone: +1-847-632-5978
   Email: j.schumacher@motorola.com







Xie & Schumacher        Expires February 22, 2008              [Page 12]


Internet-Draft       Forward-shifted RTP Redundancy          August 2007


Full Copyright Statement

   Copyright (C) The IETF Trust (2007).

   This document is subject to the rights, licenses and restrictions
   contained in BCP 78, and except as set forth therein, the authors
   retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
   THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
   OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   ietf-ipr@ietf.org.


Acknowledgment

   Funding for the RFC Editor function is provided by the IETF
   Administrative Support Activity (IASA).





Xie & Schumacher        Expires February 22, 2008              [Page 13]