Skip to main content

Associating Time-Codes with RTP Streams

The information below is for an old version of the document that is already published as an RFC.
Document Type
This is an older version of an Internet-Draft that was ultimately published as RFC 5484.
Author David Singer
Last updated 2015-10-14 (Latest revision 2009-01-15)
Replaces draft-singer-smpte-rtp
RFC stream Internet Engineering Task Force (IETF)
Additional resources Mailing list discussion
Stream WG state (None)
Document shepherd (None)
IESG IESG state RFC 5484 (Proposed Standard)
Consensus boilerplate Unknown
Telechat date (None)
Responsible AD Cullen Fluffy Jennings
Send notices to (None)
AVT                                                            D. Singer
Internet-Draft                                       Apple Computer Inc.
Intended status: Standards Track                        January 15, 2009
Expires: July 19, 2009

                Associating Time-codes with RTP streams

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at

   The list of Internet-Draft Shadow Directories can be accessed at

   This Internet-Draft will expire on July 19, 2009.

Copyright Notice

   Copyright (c) 2009 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   ( in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.

Singer                    Expires July 19, 2009                 [Page 1]
Internet-Draft            RTP SMPTE Time-codes              January 2009


   This document describes a mechanism for associating time-codes, as
   defined by the Society of Motion Picture and Television Engineers
   (SMPTE), with media streams, in a way that is independent of the RTP
   payload format of the media stream itself.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Requirements Notation  . . . . . . . . . . . . . . . . . . . .  4
   3.  Design Goals . . . . . . . . . . . . . . . . . . . . . . . . .  5
   4.  Requirements and Constraints . . . . . . . . . . . . . . . . .  6
   5.  Signaling Information  . . . . . . . . . . . . . . . . . . . .  7
   6.  In-stream Information  . . . . . . . . . . . . . . . . . . . .  9
     6.1.  Compact Format of the Time-code  . . . . . . . . . . . . .  9
     6.2.  Full Format of the Time-code . . . . . . . . . . . . . . .  9
     6.3.  Associations in RTCP . . . . . . . . . . . . . . . . . . . 10
     6.4.  Associations in RTP  . . . . . . . . . . . . . . . . . . . 12
   7.  Implementation Note (Informative)  . . . . . . . . . . . . . . 13
   8.  Discussion (Informative) . . . . . . . . . . . . . . . . . . . 14
   9.  Security Considerations  . . . . . . . . . . . . . . . . . . . 15
   10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 16
   11. RFC Editor Considerations  . . . . . . . . . . . . . . . . . . 17
   12. Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 18
   13. References . . . . . . . . . . . . . . . . . . . . . . . . . . 19
     13.1. Normative References . . . . . . . . . . . . . . . . . . . 19
     13.2. informative References . . . . . . . . . . . . . . . . . . 19
   Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 20

Singer                    Expires July 19, 2009                 [Page 2]
Internet-Draft            RTP SMPTE Time-codes              January 2009

1.  Introduction

   First a brief background on Time-codes [SMPTE-12M].

   The time-code system in common use is defined by the Society of
   Motion Picture and Television Engineers (SMPTE), and in it, time-
   codes count frames.  A common form of the display looks like a normal
   clock value (hh:mm:ss.frame).  When the frame rate is truly integer,
   then this can be a normal clock value, in that seconds tick by at the
   same rate as the seconds we know and love.

   However, NTSC video infamously runs slightly slower than 30 frames/
   second.  Some people call it 29.97 (which isn't quite right), but
   correctly a frame takes 1001 ticks of a 30000 tick/second clock.  Be
   that as it may, SMPTE time codes count 30 of these frames and deem
   that to make a second.

   This causes a SMPTE time-code display to 'run slow' compared to real-
   time.  To ameliorate this, sometimes a format called drop-frame is
   used.  Some of the frame numbers are skipped, so that the counter
   periodically 'catches up' (so some time-code seconds actually only
   have 28 frames in them).

   It is worth noting that in neither case is the SMPTE time-code an
   accurate clock; in the first case, it runs slow, and in the second,
   the adjustments are abrupt and periodic - and still not quite
   accurate.  Hence the rest of this document tries to be clear when
   referring to a second in a time-code as a 'time-code second'.

   However, SMPTE time-codes do run in real-time when used with systems
   with integral frames/second (e.g. film content at 24 frames/second,
   or PAL video).

   This specification defines how to carry time-codes in RTP and RTCP,
   associate them with a media stream, and synchronize the time-codes
   with the RTP time-stamps.  It uses the general RTP header extension
   mechanism [RFC5285].

Singer                    Expires July 19, 2009                 [Page 3]
Internet-Draft            RTP SMPTE Time-codes              January 2009

2.  Requirements Notation

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   document are to be interpreted as described in [RFC2119].

Singer                    Expires July 19, 2009                 [Page 4]
Internet-Draft            RTP SMPTE Time-codes              January 2009

3.  Design Goals

   What we desire is a system that allows us to associate a SMPTE time-
   code with some media in an RTP [RFC3550] stream.  Since in RTP all
   media has a clock already, we can often leverage that fact.  If we
   treat the media as having 'segments' of time in which the time-code
   is simply counting up, then the time-code anywhere within a segment
   can be calculated if you know:

   o  the RTP timestamp of the start of the segment;

   o  the time-code of the start of the segment;

   o  the counting rate and other parameters of the time-code;

   o  the RTP timestamp where you want to know the time-code.

   There are two cases to consider:

   1.  the time-codes are piece-wise continuous with only occasional

   2.  the continuity of the time-codes is not certain (or not known).

   The first can be handled by providing details of the time-code axis
   and an initial mapping from RTP time to time-code time, and periodic
   mappings in RTCP packets.  This is defined in section 6.3.

   The second requires in-band signaling within the RTP packets
   themselves.  This is defined in section 6.4.

   There are applications where the transport of all 8 bytes of the
   SMPTE 12M timecode are important (e.g. when the date of the time-code
   must be known, or when the RTP transport is used as a transparent
   pipe).  On the other hand, there are cases (e.g. when timecodes are
   used with compressed audio) when bandwidth is also important.  To
   support both use cases, provision is made for both compact and full
   forms of the time-code.

Singer                    Expires July 19, 2009                 [Page 5]
Internet-Draft            RTP SMPTE Time-codes              January 2009

4.  Requirements and Constraints

   Receivers MUST support timecodes in both RTCP and RTP, and both forms
   (compact and full) of the time-code.  Senders, of course, are free to

   Note that the compact form allows frame numbers greater than the full
   form (a field of 6 bits vs. a full BCD digit and a 2-bit BCD digit,
   which gives a maximum transmitted value of 29).  In some cases, the
   color frame flag (bit 11) is used to 'extend' the tens of frames
   field from 2 to 3 bits; however, such practices are outside the scope
   of this specification.

   In the case that a presentation contains more than one stream senders
   MUST continue to send the standard RTP synchronization information in
   RTCP, even if the streams carry SMPTE time-codes that could be used
   for synchronization.  In fact, when time-code is carried by more than
   one stream, this draft does not constrain the time-codes: at a given
   point in time, they may be the same, or they may differ (e.g. if they
   carry the original time-codes of different source material that was
   edited together).

Singer                    Expires July 19, 2009                 [Page 6]
Internet-Draft            RTP SMPTE Time-codes              January 2009

5.  Signaling Information

   If the recipient must ever calculate time-codes based on the RTP
   times, then some setup information is needed.  This MUST be sent out-
   of-band, for example in a SIP offer/answer exchange [RFC3264].  Since
   this is a general header extension [RFC5285], when SDP is used the
   'extmap' attribute defined by the extension mechanism is used.

   The setup information should include:

   1.  the duration, in the RTP timescale, of a single frame-count in
       the 'frames' portion of the time-code (frame_duration)

   2.  the number of those frames that make a time-code second
       (frames_per_tc_second); framecounter values may be between 0 and
       (frames_per_tc_second - 1)

   3.  is-NTSC-drop-frame: should the usual 'left out numbers' of drop-
       frame be applied or not?

   Note that other information we need to do the calculation (e.g. the
   clock rate of the RTP timestamp) is supplied already and assumed to
   be available.

   For example, if associated with a video stream with the common time-
   scale of 90000 ticks per second, then frame-duration of 3003 and
   frames-per-tc-second of 30 would yield a 'normal' SMPTE time-code for
   NTSC video.  Similarly values of 3750 and 24 yield a time-code for 24
   fps film content, and so on.

   Note also that we supply explicitly the frame duration and frames/
   second, even though they are obviously closely related.  This removes
   any ambiguity of what the counter values should be in the case of
   drop-frame counting.  These three values MUST correspond with each

   When SDP is used, these three parameters are transmitted as
   extensionattributes, as defined in the header extension specification
   [RFC5285], with the following syntax.  The form of the extension
   attributes is 'owned' by the extension name.  These parameters to the
   extension do not need registration action beyond their documentation
   here.  Note that the parameters are supplied as extension attributes,
   suitable for in-line use in RTP, even if in a given stream only the
   RTCP mapping is used.

Singer                    Expires July 19, 2009                 [Page 7]
Internet-Draft            RTP SMPTE Time-codes              January 2009

    digit = "0"/"1"/"2"/"3"/"4"/"5"/"6"/"7"/"8"/"9"

    integer = 1*digit

    frame-duration-length = integer

    timestamp-rate = integer

    frame-duration = frame-duration-length "@" timestamp-rate

    frames-per-tc-second = integer

    drop = "/drop"

    extensionattributes = frame-duration "/" frames-per-tc-second [drop]

   The frame duration is specified as a count of ticks of a clock that
   has timestamp-rate ticks per second.  It is recommended that the
   timestamp-rate be the same as the clock rate of the RTP stream in
   which the extension is embedded, to avoid the loss of accuracy in
   conversion of timestamps.  If the payload type changes during a
   stream, especially between payloads with different clock rates, it is
   strongly recommended that the header extension be included on the
   first packet(s) of the new payload, to set the mapping for the new
   clock rate explicitly.

   If '/drop' is specified, then the first two frame numbers are omitted
   from the count of each minute, except for minutes 00, 10, 20, 30, 40,
   50, as documented in SMPTE specification [SMPTE-12M] section 4.2.2.
   (Note that this usually only applies to NTSC video.)

   The URI used for the signaling is
   "urn:ietf:params:rtp-hdrext:smpte-tc".  This URI signals the possible
   presence of associations in RTCP or RTP, as defined below.

   An example in SDP, for film material, on a stream with a timescale of
   600, might be:

     a=extmap:4 urn:ietf:params:rtp-hdrext:smpte-tc 25@600/24

   Another example, for drop-frame NTSC, on a stream with a timescale of
   600, might be:

     a=extmap:4 urn:ietf:params:rtp-hdrext:smpte-tc 20@600/30/drop

Singer                    Expires July 19, 2009                 [Page 8]
Internet-Draft            RTP SMPTE Time-codes              January 2009

6.  In-stream Information

6.1.  Compact Format of the Time-code

   A compact binary SMPTE time-code in this design occupies 24 bits.  It
   is NOT formatted in the BCD system, but uses binary fixed-width
   fields.  It has the following structure:

   sign(1)  -- 1 for negative, 0 for positive

   hours (5 bits)  -- 0 to 23; the values 24-31 are reserved

   minutes (6 bits)  -- 0 to 59; 60-63 are reserved

   seconds (6 bits)  -- 0 to 59; 60-63 are reserved

   frames(6 bits)  -- 0 to (frames-per-tc-second - 1)

   Note that these fields are larger than the provision in SMPTE 12M
   where binary-coded decimal is used (and notably, where only two bits
   are provided for the tens digit of the frame count, so frame numbers
   above 39 cannot be represented).

6.2.  Full Format of the Time-code

   A full SMPTE time-code occupies 64 bits.  It is formatted exactly as
   defined in section 7 and 8 of SMPTE 12M [SMPTE-12M], without the 16-
   bit syncword.  The value of the "drop frame flag" MUST agree with the
   use of the "drop" indicator in the signalling.

   Here are the bit assignments from SMPTE 12M, for information:

   0--3  Units of frames

   4--7  First binary group

   8--9  Tens of frames

   10 Drop frame flag

   11 Color frame flag

   12--15  Second binary group

   16--19  Units of seconds

Singer                    Expires July 19, 2009                 [Page 9]
Internet-Draft            RTP SMPTE Time-codes              January 2009

   20--23  Third binary group

   24--26  Tens of seconds

   27 Polarity correction

   28--31  Fourth binary group

   32--35  Units of minutes

   36--39  Fifth binary group

   40--42  Tens of minutes

   43 Binary group flag BGF0

   44--47  Sixth binary group

   48--51  Units of hours

   52--55  Seventh binary group

   56--57  Tens of hours

   58 Binary group flag BGF1

   59   Binary group flag BGF2

   60--63  Eighth binary group

6.3.  Associations in RTCP

   When the time-codes are piece-wise continuous, we then supply in RTCP
   packets an RTP timestamp and an SMPTE time-code, for the start of
   each run of calculable time-codes.  This establishes the time-code
   for all RTP times greater than or equal to the one given, until a
   subsequent RTCP packet reestablishes the mapping.

   Note that the RTP time-stamp in the RTCP mapping may not match the
   time-stamp of any frame in the media stream.  For video, it normally
   would; but a time-stamp transition may happen part-way through a
   decoded audio frame.  Since they share the same clock, the timing of
   that transition and the timing of the audio stream itself have the
   same accuracy.

   The RTCP packets need not use the same RTP timestamp as the sender
   report (or transmission time) in the same RTCP packet.  They can be
   sent 'ahead of need' if possible (e.g. for stored content, when the

Singer                    Expires July 19, 2009                [Page 10]
Internet-Draft            RTP SMPTE Time-codes              January 2009

   server can look-ahead) or just-in-time - sent as early feedback
   packets, for example following the rules in [RFC4585], after a
   discontinuity in the time-code is detected, and allow media-buffering
   in the client the chance to 'catch' the RTCP before the matching RTP
   packet is processed and displayed.

   The association is a new RTCP Control Packet Type, using the value
   194 (see section 10).  This control packet has one of the two
   following forms, differentiated by its length:

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      |V=2|P|    SC   |PT=SMPTETC=194 |             length=3          |
      |                     SSRC of packet sender                     |
      |                         RTP timestamp                         |
      |S|  hours  |  minutes  |  seconds  |  frames   |  reserved=0   |

                     Figure 1: RTCP Short form packet

   The fields S (sign), hours, minutes, seconds, and frames, are defined
   in section 6.1.

   For this short form, the length takes the fixed value 3, indicating a
   control packet of 4 32-bit words.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      |V=2|P|    SC   |PT=SMPTETC=194 |             length=4          |
      |                     SSRC of packet sender                     |
      |                         RTP timestamp                         |
      |                          Full 8-byte                          |
      |                      SMPTE 12M timecode                       |

                      Figure 2: RTCP full form packet

Singer                    Expires July 19, 2009                [Page 11]
Internet-Draft            RTP SMPTE Time-codes              January 2009

   For this full time-code, long form, the length takes the fixed value
   4, indicating a control packet of 5 32-bit words.

6.4.  Associations in RTP

   When the time-codes are not known to be piece-wise continuous, or
   absolute surety of mapping is desired, then the mapping can be placed
   into some or all of the RTP packets.  This is a less desirable route;
   it uses the RTP header extension [RFC5285], which some terminals may
   find problematic.  And clearly placing mapping information in every
   packet uses more bandwidth.

   In as many RTP packets as needed (possibly all), a RTP header
   extension is used [RFC5285] to associate an RTP time to a SMPTE time-

   There are two forms of this header extension, again differentiated by
   their length.  The short form associates a compact time-code with the
   RTP timestamp of the packet.  The long form allows associates a full
   time-code with a timestamp offset from the RTP timestamp of the

   The short form has a length of 3 bytes (24 bits).  The long form has
   a length of 12 bytes (96 bits), and consists of a full SMPTE 12M
   time-code, followed by a signed 32-bit offset D from the RTP
   timestamp.  If the packet has timestamp T, this establishes an RTP to
   time-code association for the RTP time T+D.

Singer                    Expires July 19, 2009                [Page 12]
Internet-Draft            RTP SMPTE Time-codes              January 2009

7.  Implementation Note (Informative)

   This section contains a suggestion on how to calculate a time-code
   for a time T2, given an initial code at time T1, and the frame

   It might seem that when drop-frame is used, there is a 'fence post'
   problem: how many minutes in which frame-numbers are dropped have
   past since the initial time-code?  However, this can be avoided if
   all calculations are 'zero-based'; then the number of 'fence posts'
   is known.

     framesSinceTCzero := TimeCodeToFrameCount( initialTimeCode );
     framesSinceMapping := floor( (T2-T1)/frameDuration );
     totalFrames := framesSinceTCzero + framesSinceMapping;
     timeCode := FrameCountToTimeCode( totalFrames );

   The SMPTE engineering guideline [smpte-eg40] contains all the
   appropriate equations, constants etc. for performing these and other

Singer                    Expires July 19, 2009                [Page 13]
Internet-Draft            RTP SMPTE Time-codes              January 2009

8.  Discussion (Informative)

   This design has the advantage of not requiring the introduction of
   new IP packets into the sessions or new data into the main data
   channel, using low-bandwidth (vanishingly low in the case of streams
   with no discontinuities), and is independent of the design of the RTP
   packets themselves: the RTP profile (including possibly encryption)
   and the RTP payload format.  SMPTE time-codes can be associated with
   any RTP stream, including those with existing payload formats.

   It might be argued that we could set the initial mapping also in the
   SDP, since RTCP packets might get lost.  But this means that the SDP
   now has to have knowledge of the RTP random offset, which is nasty;
   and if one puts this RTCP packet into all sender reports, it's
   probably good enough.  Then if you don't have time-codes, you don't
   have audio-video-sync either.

   This associates the time-code with a particular media stream.  An
   alternative would be to make it an RTP stream in its own right; but
   the data rate is so low, this seems egregious.  And by packing it
   inline, we can do this backwards-compatible for gateways etc. that
   already handle dual-stream.

   There is no way in this draft to detect that an RTCP packet has been
   lost, and that a mapping may be being used outside its intended

   The design assumes that clients will hold mappings until they are
   superseded, and that a client may need to buffer some number of
   upcoming mappings.

Singer                    Expires July 19, 2009                [Page 14]
Internet-Draft            RTP SMPTE Time-codes              January 2009

9.  Security Considerations

   SMPTE time-codes are only informative and there are no known security
   considerations from associating them with media streams.

Singer                    Expires July 19, 2009                [Page 15]
Internet-Draft            RTP SMPTE Time-codes              January 2009

10.  IANA Considerations

   The RTCP packet type used for SMPTE time-code needs to be registered,
   in accordance with section 15 of [RFC3550].  IANA is instructed to
   add a new value to the RTCP Control Packet types subregistry of the
   Real-Time Transport Protocol (RTP) Parameters registry, according to
   the following data:

   abbrev.    name                     value   Reference
   _________  _______________________  ______  _________
   SMPTETC    SMPTE time-code mapping  yyy     [RFC-avt-smpte-rtp-15]

   Note: it is suggested that IANA allocates the value 194 for yyy

   Additionally, IANA is instructed to register a new extension URI to
   the RTP Compact Header Extensions subregistry of the Real-Time
   Transport Protocol (RTP) Parameters registry, according to the
   following data (split in two lines for formating purposes):

   Extension URI                          Reference
   -------------------------------------  ---------
   urn:ietf:params:rtp-hdrext:smpte-tc    [RFC-avt-smpte-rtp-15]

   Contact            Description
   -----------------  -----------------------   SMPTE time-code mapping

Singer                    Expires July 19, 2009                [Page 16]
Internet-Draft            RTP SMPTE Time-codes              January 2009

11.  RFC Editor Considerations

   The reference to an Internet Draft needs to be updated to the RFC
   when it is published (which should be before this draft).

   RFCxxxx in the IANA considerations needs to be updated with the
   number of this RFC.

Singer                    Expires July 19, 2009                [Page 17]
Internet-Draft            RTP SMPTE Time-codes              January 2009

12.  Acknowledgments

   Both Brian Link and John Lazzaro provided helpful comments on an
   initial draft.  Colin Perkins was helpful in reviewing and dealing
   with the details.  Ladan Gharai provided a thoughtful final review.

Singer                    Expires July 19, 2009                [Page 18]
Internet-Draft            RTP SMPTE Time-codes              January 2009

13.  References

13.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC3264]  Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
              with the Session Description Protocol (SDP)", RFC 3264,
              June 2002.

   [RFC3550]  Schulzrinne, H., Casner, S., Frederick, R., and V.
              Jacobson, "RTP: A Transport Protocol for Real-Time
              Applications", RFC 3550, STD 0064, July 2003.

   [RFC4585]  Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
              "Extended RTP Profile for Real-time Transport Control
              Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
              August 2006.

   [RFC5285]  Singer, D. and H. Desineni, "A general mechanism for RTP
              Header Extensions", RFC 5285, July 2008.

13.2.  informative References

              Society of Motion Picture and Television Engineers, "SMPTE
              Standard for Television - Time and Control Code",
              SMPTE 12M-1-2008.

              SMPTE, "Conversion of Time Values Between SMPTE 12M Time
              Code, MPEG-2  PCR Time Base and Absolute Time",
              SMPTE EG40-2002, August 2002.

Singer                    Expires July 19, 2009                [Page 19]
Internet-Draft            RTP SMPTE Time-codes              January 2009

Author's Address

   David Singer
   Apple Computer Inc.
   1 Infinite Loop
   Cupertino, CA  95014

   Phone: +1 408 996 1010

Singer                    Expires July 19, 2009                [Page 20]