Matroska Media Container Control Track Specifications
draft-ietf-cellar-control-00

Document Type Active Internet-Draft (cellar WG)
Authors Steve Lhomme , Moritz Bunkus , Dave Rice 
Last updated 2021-10-09
Stream Internet Engineering Task Force (IETF)
Intended RFC status (None)
Formats plain text html xml pdf htmlized bibtex
Stream WG state WG Document
Document shepherd No shepherd assigned
IESG IESG state I-D Exists
Consensus Boilerplate Unknown
Telechat date
Responsible AD (None)
Send notices to (None)
cellar                                                         S. Lhomme
Internet-Draft                                                          
Intended status: Standards Track                               M. Bunkus
Expires: 12 April 2022                                                  
                                                                 D. Rice
                                                          9 October 2021

         Matroska Media Container Control Track Specifications
                      draft-ietf-cellar-control-00

Abstract

   This document defines the Control Track usage found in the Matroska
   container.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 12 April 2022.

Copyright Notice

   Copyright (c) 2021 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Simplified BSD License text
   as described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Simplified BSD License.

Lhomme, et al.            Expires 12 April 2022                 [Page 1]
Internet-Draft           Matroska Control Track             October 2021

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Status of this document . . . . . . . . . . . . . . . . . . .   2
   3.  Security Considerations . . . . . . . . . . . . . . . . . . .   2
   4.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   2
   5.  Notation and Conventions  . . . . . . . . . . . . . . . . . .   3
   6.  Edition Flags . . . . . . . . . . . . . . . . . . . . . . . .   3
     6.1.  EditionFlagHidden . . . . . . . . . . . . . . . . . . . .   3
     6.2.  EditionFlagDefault  . . . . . . . . . . . . . . . . . . .   4
     6.3.  Default Edition . . . . . . . . . . . . . . . . . . . . .   4
   7.  Chapter Flags . . . . . . . . . . . . . . . . . . . . . . . .   6
     7.1.  ChapterFlagEnabled  . . . . . . . . . . . . . . . . . . .   6
   8.  Matroska Schema . . . . . . . . . . . . . . . . . . . . . . .   6
     8.1.  Segment . . . . . . . . . . . . . . . . . . . . . . . . .   6
       8.1.1.  Chapters  . . . . . . . . . . . . . . . . . . . . . .   7
         8.1.1.1.  EditionEntry  . . . . . . . . . . . . . . . . . .   7
   9.  Menu Specifications . . . . . . . . . . . . . . . . . . . . .   8
     9.1.  Requirements  . . . . . . . . . . . . . . . . . . . . . .   9
       9.1.1.  Highlights/Hotspots . . . . . . . . . . . . . . . . .   9
       9.1.2.  Playback features . . . . . . . . . . . . . . . . . .  10
       9.1.3.  Player requirements . . . . . . . . . . . . . . . . .  10
     9.2.  Working Graph . . . . . . . . . . . . . . . . . . . . . .  10
   10. Normative References  . . . . . . . . . . . . . . . . . . . .  10
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  11

1.  Introduction

2.  Status of this document

   This document is a work-in-progress specification defining the
   Matroska file format as part of the IETF Cellar working group
   (https://datatracker.ietf.org/wg/cellar/charter/).  It uses basic
   elements and concept already defined in the Matroska specifications
   defined by this workgroup.

3.  Security Considerations

   This document inherits security considerations from the EBML and
   Matroska documents.

4.  IANA Considerations

   To be determined.

Lhomme, et al.            Expires 12 April 2022                 [Page 2]
Internet-Draft           Matroska Control Track             October 2021

5.  Notation and Conventions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

6.  Edition Flags

6.1.  EditionFlagHidden

   When the EditionFlagHidden flag is set to false it means the Edition
   is visible and selectable in a Matroska Player.  All ChapterAtoms
   Elements MUST be interpreted with their own ChapterFlagHidden flags.

              +===================+=======+======+=========+
              | ChapterFlagHidden | False | True | visible |
              +===================+=======+======+=========+
              | Chapter 1         | X     |      | yes     |
              +-------------------+-------+------+---------+
              | Chapter 2         |       | X    | no      |
              +-------------------+-------+------+---------+

               Table 1: ChapterAtom visibility to the user

   When the EditionFlagHidden flag is set to true the Edition is hidden
   and SHOULD NOT be selectable in a Matroska Player.  If all Editions
   EditionFlagHidden flags are set to true, there is no visible Edition.
   In this case all ChapterAtoms Elements MUST also be interpreted as if
   their ChapterFlagHidden flag is also set to true, regardless with
   their own ChapterFlagHidden flags.

              +===================+=======+======+=========+
              | ChapterFlagHidden | False | True | visible |
              +===================+=======+======+=========+
              | Chapter 1         | X     |      | no      |
              +-------------------+-------+------+---------+
              | Chapter 2         |       | X    | no      |
              +-------------------+-------+------+---------+

                Table 2: ChapterAtom visibility in hidden
                                 editions

Lhomme, et al.            Expires 12 April 2022                 [Page 3]
Internet-Draft           Matroska Control Track             October 2021

6.2.  EditionFlagDefault

   It is RECOMMENDED that no more than one Edition have an
   EditionFlagDefault Flag set to true.  The first Edition with both the
   EditionFlagDefault Flag set to true and the EditionFlagHidden Flag
   set to false is the Default Edition.  When all EditionFlagDefault
   Flags are set to false, then the first Edition with the
   EditionFlagHidden Flag set to false is the Default Edition.  The
   Default Edition is the edition that should be used for playback by
   default.

6.3.  Default Edition

   The Default Edition is the Edition that a Matroska Player SHOULD use
   for playback by default.

   The first Edition with both the EditionFlagDefault flag set to true
   and the EditionFlagHidden flag set to false is the Default Edition.
   When all EditionFlagDefault flags are set to false and all
   EditionFlagHidden flag set to true, then the first Edition is the
   Default Edition.  When all EditionFlagHidden flags are set to true,
   then the first Edition with the EditionFlagDefault flag set to true
   is the Default Edition.  When all EditionFlagDefault flags are set to
   false, then the first Edition with the EditionFlagHidden flag set to
   false is the Default Edition.  When there is no Edition with a
   EditionFlagDefault flag are set to true and a EditionFlagHidden flags
   are set to false, then the first Edition with the EditionFlagHidden
   flag set to false is the Default Edition.

   In other words, in case the Default Edition is not obvious, the first
   Edition with a EditionFlagHidden flag set to false SHOULD be
   preferred.

        +===========+============+=============+=================+
        | Edition   | FlagHidden | FlagDefault | Default Edition |
        +===========+============+=============+=================+
        | Edition 1 | true       | true        |                 |
        +-----------+------------+-------------+-----------------+
        | Edition 2 | true       | true        |                 |
        +-----------+------------+-------------+-----------------+
        | Edition 3 | false      | true        | X               |
        +-----------+------------+-------------+-----------------+

           Table 3: Default edition, some visible, all default

Lhomme, et al.            Expires 12 April 2022                 [Page 4]
Internet-Draft           Matroska Control Track             October 2021

        +===========+============+=============+=================+
        | Edition   | FlagHidden | FlagDefault | Default Edition |
        +===========+============+=============+=================+
        | Edition 1 | true       | false       | X               |
        +-----------+------------+-------------+-----------------+
        | Edition 2 | true       | false       |                 |
        +-----------+------------+-------------+-----------------+
        | Edition 3 | true       | false       |                 |
        +-----------+------------+-------------+-----------------+

             Table 4: Default edition, all hidden, no default

        +===========+============+=============+=================+
        | Edition   | FlagHidden | FlagDefault | Default Edition |
        +===========+============+=============+=================+
        | Edition 1 | true       | false       |                 |
        +-----------+------------+-------------+-----------------+
        | Edition 2 | true       | true        | X               |
        +-----------+------------+-------------+-----------------+
        | Edition 3 | true       | false       |                 |
        +-----------+------------+-------------+-----------------+

            Table 5: Default edition, all hidden, with default

        +===========+============+=============+=================+
        | Edition   | FlagHidden | FlagDefault | Default Edition |
        +===========+============+=============+=================+
        | Edition 1 | true       | false       |                 |
        +-----------+------------+-------------+-----------------+
        | Edition 2 | false      | false       | X               |
        +-----------+------------+-------------+-----------------+
        | Edition 3 | false      | false       |                 |
        +-----------+------------+-------------+-----------------+

            Table 6: Default edition, some visible, no default

        +===========+============+=============+=================+
        | Edition   | FlagHidden | FlagDefault | Default Edition |
        +===========+============+=============+=================+
        | Edition 1 | true       | false       |                 |
        +-----------+------------+-------------+-----------------+
        | Edition 2 | true       | true        |                 |
        +-----------+------------+-------------+-----------------+
        | Edition 3 | false      | false       | X               |
        +-----------+------------+-------------+-----------------+

           Table 7: Default edition, some visible, some default

Lhomme, et al.            Expires 12 April 2022                 [Page 5]
Internet-Draft           Matroska Control Track             October 2021

7.  Chapter Flags

   If a Control Track toggles the parent's ChapterFlagHidden flag to
   false, then only the parent ChapterAtom and its second child
   ChapterAtom MUST be interpreted as if ChapterFlagHidden is set to
   false.  The first child ChapterAtom, which has the ChapterFlagHidden
   flag set to true, retains its value until its value is toggled to
   false by a Control Track.

   The ChapterFlagEnabled value can be toggled by control tracks.

7.1.  ChapterFlagEnabled

   If the ChapterFlagEnabled flag is set to false a Matroska Player MUST
   NOT use this Chapter and all his Nested Chapters.  For Simple
   Chapters, a Matroska Player MAY display this enabled Chapter with a
   marker in the timeline.  For Ordered Chapters a Matroska Player MUST
   use the duration of this enabled Chapter.

         +==========================+====================+======+
         | Chapter + Nested Chapter | ChapterFlagEnabled | used |
         +==========================+====================+======+
         | Chapter 1                | true               | yes  |
         +--------------------------+--------------------+------+
         | +Nested Chapter 1.1      | true               | yes  |
         +--------------------------+--------------------+------+
         | +Nested Chapter 1.2      | false              | no   |
         +--------------------------+--------------------+------+
         | ++Nested Chapter 1.2.1   | true               | no   |
         +--------------------------+--------------------+------+
         | ++Nested Chapter 1.2.2   | false              | no   |
         +--------------------------+--------------------+------+
         | Chapter 2                | false              | no   |
         +--------------------------+--------------------+------+
         | +Nested Chapter 2.1      | true               | no   |
         +--------------------------+--------------------+------+
         | +Nested Chapter 2.2      | true               | no   |
         +--------------------------+--------------------+------+

                                 Table 8

8.  Matroska Schema

   Extra elements used to handle Control Tracks and advanced selection
   features:

8.1.  Segment

Lhomme, et al.            Expires 12 April 2022                 [Page 6]
Internet-Draft           Matroska Control Track             October 2021

8.1.1.  Chapters

8.1.1.1.  EditionEntry

8.1.1.1.1.  EditionFlagHidden Element

   name:  EditionFlagHidden

   path:  \Segment\Chapters\EditionEntry\EditionFlagHidden

   id:  0x45BD

   minOccurs:  1

   maxOccurs:  1

   range:  0-1

   default:  0

   type:  uinteger

   definition:  Set to 1 if an edition is hidden.  Hidden editions
      SHOULD NOT be available to the user interface (but still to
      Control Tracks; see Section 7 on Chapter flags).

8.1.1.1.1.1.  ChapterFlagEnabled Element

   name:  ChapterFlagEnabled

   path:  \Segment\Chapters\EditionEntry\+ChapterAtom\ChapterFlagEnabled

   id:  0x4598

   minOccurs:  1

   maxOccurs:  1

   range:  0-1

   default:  1

   type:  uinteger

   definition:  Set to 1 if the chapter is enabled.  It can be enabled/
      disabled by a Control Track.  When disabled, the movie SHOULD skip
      all the content between the TimeStart and TimeEnd of this chapter;
      see Section 7 on Chapter flags.

Lhomme, et al.            Expires 12 April 2022                 [Page 7]
Internet-Draft           Matroska Control Track             October 2021

8.1.1.1.1.2.  ChapterTrack Element

   name:  ChapterTrack

   path:  \Segment\Chapters\EditionEntry\+ChapterAtom\ChapterTrack

   id:  0x8F

   maxOccurs:  1

   type:  master

   definition:  List of tracks on which the chapter applies.  If this
      Element is not present, all tracks apply

8.1.1.1.1.3.  ChapterTrackUID Element

   name:  ChapterTrackUID

   path:  \Segment\Chapters\EditionEntry\+ChapterAtom\ChapterTrack\Chapt
      erTrackUID

   id:  0x89

   minOccurs:  1

   range:  not 0

   type:  uinteger

   definition:  UID of the Track to apply this chapter to.  In the
      absence of a control track, choosing this chapter will select the
      listed Tracks and deselect unlisted tracks.  Absence of this
      Element indicates that the Chapter SHOULD be applied to any
      currently used Tracks.

9.  Menu Specifications

   This document is a _draft of the Menu system_ that will be the
   default one in Matroska.  As it will just be composed of a Control
   Track, it will be seen as a "codec" and could be replaced later by
   something else if needed.

   A menu is like what you see on DVDs, when you have some screens to
   select the audio format, subtitles or scene selection.

Lhomme, et al.            Expires 12 April 2022                 [Page 8]
Internet-Draft           Matroska Control Track             October 2021

9.1.  Requirements

   What we'll try to have is a system that can do almost everything done
   on a DVD, or more, or better, or drop the unused features if
   necessary.

   As the name suggests, a Control Track is a track that can control the
   playback of the file and/or all the playback features.  To make it as
   simple as possible for Matroska Players, the Control Track will just
   give orders to the Matroska Player and get the actions associated
   with the highlights/hotspots.

9.1.1.  Highlights/Hotspots

   A highlight is basically a rectangle/key associated with an action
   UID.  When that rectangle/key is activated, the Matroska Player send
   the UID of the action to the Control Track handler (codec).  The fact
   that it can also be a key means that even for audio only files, a
   keyboard shortcut or button panel could be used for menus.  But in
   that case, the hotspot will have to be associated with a name to
   display.

   This highlight is sent from the Control Track to the Matroska Player.
   Then the Matroska Player has to handle that highlight until it's
   deactivated; see Section 9.1.2.

   The highlight contains a UID of the action, a displayable name (UTF-
   8), an associated key (list of keys to be defined, probably
   up/down/left/right/select), a screen position/range and an image to
   display.  The image will be displayed either when the user place the
   mouse over the rectangle (or any other shape), or when an option of
   the screen is selected (not activated).  There could be a second
   image used when the option is activated.  And there could be a third
   image that can serve as background.  This way you could have a still
   image (like in some DVDs) for the menu and behind that image blank
   video (small bitrate).

   When a highlight is activated by the user, the Matroska Player has to
   send the UID of the action to the Control Track.  Then the Control
   Track codec will handle the action and possibly give new orders to
   the Matroska Player.

   The format used for storing images SHOULD be extensible.  For the
   moment we'll use PNG and BMP, both with alpha channel.

Lhomme, et al.            Expires 12 April 2022                 [Page 9]
Internet-Draft           Matroska Control Track             October 2021

9.1.2.  Playback features

   All the following features will be sent from the Control Track to the
   Matroska Player :

   *  Jump to chapter (UID, prev, next, number)

   *  Disable all tracks of a kind (audio, video, subtitle)

   *  Enable track UID (the kind doesn't matter)

   *  Define/Disable a highlight

   *  Enable/Disable jumping

   *  Enable/Disable track selection of a kind

   *  Select Edition ID (see chapters)

   *  Pause playback

   *  Stop playback

   *  Enable/Disable a Chapter UID

   *  Hide/Unhide a Chapter UID

   All the actions will be written in a normal Matroska track, with a
   timestamp.  A "Menu Frame" SHOULD be able to contain more that one
   action/highlight for a given timestamp. (to be determined, EBML
   format structure)

9.1.3.  Player requirements

   Some Matroska Players might not support the control track.  That mean
   they will play the active/looped parts as part of the data.  So I
   suggest putting the active/looped parts of a movie at the end of a
   movie.  When a Menu-aware Matroska Player encounter the default
   Control Track of a Matroska file, the first order SHOULD be to jump
   at the start of the active/looped part of the movie.

9.2.  Working Graph

   Matroska Source file -> Control Track <-> Player.
                        -> other tracks   -> rendered

10.  Normative References

Lhomme, et al.            Expires 12 April 2022                [Page 10]
Internet-Draft           Matroska Control Track             October 2021

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

Authors' Addresses

   Steve Lhomme

   Email: slhomme@matroska.org

   Moritz Bunkus

   Email: moritz@bunkus.org

   Dave Rice

   Email: dave@dericed.com

Lhomme, et al.            Expires 12 April 2022                [Page 11]