AT&T's Error Resilient Video Transmission Technique
draft-civanlar-hplp-00

Internet Engineering Task Force                         M. Reha Civanlar
  INTERNET-DRAFT                                             Glenn L. Cash
  File: draft-civanlar-hplp-00.txt                        Barry G. Haskell

                                                        AT&T Labs-Research

                                                                July, 1998

           AT&T's Error Resilient Video Transmission Technique

                           Status of this Memo

  This document is an Internet-Draft.  Internet-Drafts are working
  documents of the Internet Engineering Task Force (IETF), its areas,
  and its working groups.  Note that other groups may also distribute
  working documents as Internet-Drafts.

  Internet-Drafts are draft documents valid for a maximum of six months
  and may be updated, replaced, or obsoleted by other documents at any
  time.  It is inappropriate to use Internet- Drafts as reference
  material or to cite them other than as ``work in progress.''

  To view the entire list of current Internet-Drafts, please check the
  "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow
  Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern Europe),
  ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific Rim),
  ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast).

  Distribution of this memo is unlimited.

                                 Abstract

  This document describes a set of techniques for packet loss resilient
  transmission of compressed video bitstreams based on reliable delivery
  of their vital information-carrying segments. The described techniques
  can be used over packet networks without packet prioritization. These
  techniques are related to AT&T/Lucent patents [1, 2].

draft-civanlar-hplp-00.txt                                      [Page 1]
INTERNET-DRAFT AT&T's Resilient Video Transmission Technique   July 1998

  1. Introduction

  It is well known that every bit in a compressed video bitstream is not
  equal. Some bits belong to segments defining vital information such as
  picture types, quantization values, parameter ranges, average
  intensity values for image blocks, etc. When transporting compressed
  video bitstreams over packet networks, packet losses from such
  segments cause a much longer lasting and severe degradation on the
  output of a decoder than that caused by packet losses from other
  segments. We will call the vital information-carrying segments "High
  Priority (HP)" segments. The rest of the bitstream consists of "Low
  Priority (LP)" segments. Clearly, the video outputs resulting from
  transport techniques that protect the HP segments against packet
  losses are more resilient to packet losses in general.

  Protection of the HP segments can be accomplished in many ways. These
  include:
    - redundant transmission of the HP segments as described
    in [3] for MPEG RTP payloads
    - using forward error correction (FEC) techniques
    - transmitting HP segments over reserved channels or using
    differentiated services.
 Both redundant transmission and FEC techniques increase the bandwidth
 needed to transmit the compressed video bitstream. FEC techniques
 increase the effectiveness of this additional bandwidth for packet loss
 protection at the expense of increased processing at the receiver and
 the transmitter ends and increased overall delay. Using channel
 reservations or differentiated services based approaches may be the
 best solutions for protecting the HP segments but, they require network
 infrastructure changes.

 This document outlines another set of HP segment protection techniques
 based on AT&T/Lucent patents [1, 2] that can be used for reliable video
 transmission over packet networks without a built-in prioritization
 mechanism. These techniques use reliable transport protocols and "out-
 of-band" delivery approaches. In this context, the term "out-of-band"
 is used to imply information transmission means other than those used
 for transmitting the main video stream.  The details of these
 techniques are discussed in the following sections. An implementation
 of these, as applied to MPEG-2 video transmission over IP networks, is
 described in [4].

 2. Identification of the HP segments

 The classification of a part of a video bitstream as an HP segment
 depends on two factors.  The first one is the encoding algorithm used
 in compressing the video data. It is impossible to segment a compressed
 video bitstream without knowing the syntax and the semantics of the

draft-civanlar-hplp-00.txt                                      [Page 2]
INTERNET-DRAFT AT&T's Resilient Video Transmission Technique   July 1998

 encoding algorithm. The second factor is the determination of a
 compromise between the HP segment size and the corresponding loss
 resilience. As the segment size increases, so does the loss resilience.
 On the other hand, it may not be feasible to deliver large HP segments
 reliably.

 As an example, the "data partitioning" method of the MPEG-2 standard
 [5] defines the syntax and semantics for one particular way of
 partitioning an MPEG-2 encoded video bitstream into HP and LP segments.
 In data partitioning, the smallest useful HP segment can be selected to
 contain only the header information, which is usually less than two
 percent of the video data. HP segments defined this way contain vital
 information including picture type, quantization factor, motion vector
 ranges, etc. without which the rest of the bitstream is not decodable.
 As an alternative, the DC coefficients (the average values) for each
 picture macroblock may be included in the HP segment increasing its
 size to about 40% of the bitstream.  This way HP segments can be made
 to carry somewhat usable video information also; however, their
 reliable transmission may become a demanding task.

 Since it is not possible to formulate a general technique that can be
 used for identifying the HP segments in any encoded video bitstream, we
 will assume that such segments are identified some way prior to the
 transmission. For example, some encoders can generate HP and LP
 segments separately, a stored bitstream can be in the partitioned
 format, etc. Also, consistent with most of the popular coding
 techniques, we assume that the HP segments (HP1, HP2, ...) are
 dispersed on the entire bitstream over time as shown in Fig. 1.

 +---+----------------+---+----------------------+---+-----
 |HP1|     LP1        |HP2|        LP2           |HP3| ...
 +---+----------------+---+----------------------+---+-----

 Figure 1 - HP segments dispersed on an encoded video bitstream over time

 3. Transmission of HP data using a reliable transport protocol [1]

 In this approach, one or more of the HP segments are transmitted using
 a reliable transport protocol prior to starting the transmission of the
 LP segments. For point-to-point applications, TCP, for multipoint
 applications, an appropriate reliable multicast protocol [6] may be
 used for transporting the HP segments. The number of HP segments to be
 sent before starting the transmission of the LP segments depends on the
 application's tolerance to the start-up delay.  Depending on the HP
 segment size and the path-MTU [7], one or more HP segments can be put
 in each packet carrying the HP data.

 HP segments can be packetized using RTP with the following definitions

draft-civanlar-hplp-00.txt                                      [Page 3]
INTERNET-DRAFT AT&T's Resilient Video Transmission Technique   July 1998

 for the header fields:

   Payload Type: A distinct payload type number, which may be dynamic,
   should be assigned to HP segments of each video payload.

   M Bit: Set for packets containing HP data for key pictures.

   timestamp: Uses the same format as that of the video payload. Shows
   the sampling time for the video data following the first HP segment
   in the packet.

The SSRC field may be defined following the rules developed for the
transmission of layered media streams in [8]. That is:
  - A single SSRC space is used for the HP segment packets and the main
  video stream. Only the latter is used for SSRC allocation and conflict
  resolution. When a source discovers that it has collided, it transmits
  an RTCP BYE message on only the main video stream.

  - A participant sends sender identification (SDES) on only the main
  video stream.

Most HP segments are self-identifying and can be packed without any
additional headers. For others, techniques used for packetizing generic
payload types may be used or special payload types may be defined.

It is possible to send the HP data along with the LP data (i.e., the
original, unpartitioned bitstream) in addition to sending the HP
segments separately. This way, the separately transmitted HP segments
are needed only when packet losses occur.

4. Out-of-band transmission of the HP information [2]

In cases where a certain sequence of HP segments is used periodically
for the entire duration of the video bitstream, this sequence may be
transmitted once before the start of video transmission using a reliable
transport protocol. The receiver can save this information and use it to
recover lost HP segments during the main video transmission.

In this approach, the timestamps are not meaningful for the HP data and
they may not be included in the transmitted HP segment sequence. In most
cases, the synchronization between the stored HP segments and the LP
data stream can be accomplished using the key-frames because the HP data
sequence usually cover the video segment between two key-frames (e.g. a
group-of-pictures (GOP) in MPEG). If the sequence of HP segments covers
a video sequence with more than one key-frame, some indicator, e.g. if
available the M-bit may be used to indicate a packet which carries the
beginning of LP data that follows the first stored HP segment.

draft-civanlar-hplp-00.txt                                      [Page 4]
INTERNET-DRAFT AT&T's Resilient Video Transmission Technique   July 1998

5. Security considerations

RTP packets transmitted according to the techniques outlined in this
document are subject to the security considerations discussed in the RTP
specification [9]. This implies that confidentiality of the media
streams is achieved by encryption. Because the data compression used is
applied end-to-end, encryption may be performed after compression so
there is no conflict between the two operations. For certain coding
techniques and applications, encrypting only the HP segments may provide
sufficent confidentiality.

The described techniques do not introduce any significant additional
non-uniformity in the receiver side computational complexity for packet
processing to cause a potential denial-of-service threat.

References:

[1] Glenn L. Cash, Mehmet R. Civanlar, "Method Of And Apparatus For
    The Transmission Of High And Low Priority Segments Of A Video
    Bitstream Over Packet Networks," United States Patent Number:
    5,481,312, Jan. 2, 1996.

[2] Glenn L. Cash, Mehmet R. Civanlar, "Video Bitstream Regeneration
    Using Previously Agreed To High Priority Segments," United States
    Patent Number: 5,510,844, April 23, 1996.

[3] D.Hoffman, G. Fernando, V. Goyal, M. R. Civanlar, "RTP Payload Format
    for MPEG1/MPEG2 Video," RFC 2250, April 1997.

[4] M. R. Civanlar, G. L. Cash, "A practical system for MPEG-2 based
    video-on-demand over ATM packet networks and the WWW," Signal
    Processing: Image Communication, no. 8, pp. 221-227, Elsevier, 1996.

[5] ISO/IEC International Standard 13818; "Generic coding of moving
    pictures and associated audio information," November 1994.

[6] Overview of Reliable Multicast Protocols Web Page,
    URL http://gaia.cs.umass.edu/sigcomm_mcast/talk1.html.

[7] J. Mogul, S. Deering, "Path MTU Discovery," RFC 1191, November 1990.

[8] M. F. Speer, S. McCanne, "RTP Usage with Layered Multimedia
    Streams," Internet Draft, draft-speer-avt-layered-video-02.txt,
    December 1996.

[9] H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson,
    "RTP: A Transport Protocol for Real-Time Applications,"
    RFC 1889, January 1996.

draft-civanlar-hplp-00.txt                                      [Page 5]
INTERNET-DRAFT AT&T's Resilient Video Transmission Technique   July 1998

Author's  Address:

   M. Reha Civanlar
   Glenn L. Cash
   Barry G. Haskell

   AT&T Labs-Research
   100 Schultz Drive
   Red Bank, NJ 07701
   USA

   e-mail: civanlar|glenn|bgh@research.att.com

draft-civanlar-hplp-00.txt                                      [Page 6]