avtcore S. Lugan
Internet-Draft A. Descampe
Intended status: Standards Track C. Damman
Expires: October 10, 2020 intoPIX
T. Richter
IIS
A. Willeme
UCL/ICTEAM
April 8, 2020
RTP Payload Format for ISO/IEC 21122 (JPEG XS)
draft-ietf-payload-rtp-jpegxs-03
Abstract
This document specifies a Real-Time Transport Protocol (RTP) payload
format to be used for transporting JPEG XS (ISO/IEC 21122) encoded
video. JPEG XS is a low-latency, lightweight image coding system.
Compared to an uncompressed video use case, it allows higher
resolutions and frame rates, while offering visually lossless
quality, reduced power consumption, and end-to-end latency confined
to a fraction of a frame.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on October 10, 2020.
Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
Lugan, et al. Expires October 10, 2020 [Page 1]
Internet-Draft RTP Payload Format for JPEG XS April 2020
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Conventions, Definitions, and Abbreviations . . . . . . . . . 3
3. Media Format Description . . . . . . . . . . . . . . . . . . 4
3.1. Image Data Structures . . . . . . . . . . . . . . . . . . 4
3.2. Codestream . . . . . . . . . . . . . . . . . . . . . . . 5
3.3. Video support box and colour specification box . . . . . 5
3.4. JPEG XS Frame . . . . . . . . . . . . . . . . . . . . . . 5
4. Payload Format . . . . . . . . . . . . . . . . . . . . . . . 6
4.1. RTP packetization . . . . . . . . . . . . . . . . . . . . 6
4.2. Payload Header . . . . . . . . . . . . . . . . . . . . . 8
4.3. Payload Data . . . . . . . . . . . . . . . . . . . . . . 11
4.4. Traffic Shaping and Delivery Timing . . . . . . . . . . . 14
5. Congestion Control Considerations . . . . . . . . . . . . . . 14
6. Payload Format Parameters . . . . . . . . . . . . . . . . . . 14
6.1. Media Type Definition . . . . . . . . . . . . . . . . . . 14
6.2. Mapping to SDP . . . . . . . . . . . . . . . . . . . . . 17
6.2.1. General . . . . . . . . . . . . . . . . . . . . . . . 17
6.2.2. Media type and subtype . . . . . . . . . . . . . . . 18
6.2.3. Traffic shaping . . . . . . . . . . . . . . . . . . . 18
6.2.4. Offer/Answer Considerations . . . . . . . . . . . . . 18
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19
8. Security Considerations . . . . . . . . . . . . . . . . . . . 19
9. RFC Editor Considerations . . . . . . . . . . . . . . . . . . 20
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.1. Normative References . . . . . . . . . . . . . . . . . . 20
10.2. Informative References . . . . . . . . . . . . . . . . . 22
10.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22
1. Introduction
This document specifies a payload format for packetization of JPEG XS
encoded video signals into the Real-time Transport Protocol (RTP)
[RFC3550].
The JPEG XS coding system offers compression and recompression of
image sequences with very moderate computational resources while
remaining robust under multiple compression and decompression cycles
Lugan, et al. Expires October 10, 2020 [Page 2]
Internet-Draft RTP Payload Format for JPEG XS April 2020
and mixing of content sources, e.g. embedding of subtitles, overlays
or logos. Typical target compression ratios ensuring visually
lossless quality are in the range of 2:1 to 10:1, depending on the
nature of the source material. The end-to-end latency can be
confined to a fraction of a frame, typically between a small number
of lines down to below a single line.
2. Conventions, Definitions, and Abbreviations
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Application Data Unit (ADU)
The unit of source data provided as payload to the transport
layer, and corresponding, in this RTP payload definition, to a
single JPEG XS frame.
Colour specification box (CS box)
A ISO colour specification box defined in ISO/IEC 21122-3
[ISO21122-3] that includes colour-related metadata required to
correctly display JPEG XS frames, such as colour primaries,
transfer characteristics and matrix coefficients.
EOC marker
A marker that consists of the two bytes 0xff11 indicating the end
of a JPEG XS codestream.
JPEG XS codestream
A sequence of bytes representing a compressed image formatted
according to JPEG XS Part-1 [ISO21122-1].
JPEG XS codestream header
A sequence of bytes, starting with a SOC marker, at the beginning
of each JPEG XS codestream encoded in multiple markers and marker
segments that does not carry entropy coded data, but metadata such
as the frame dimension and component precision.
JPEG XS frame
The concatenation of a video support box, as defined in ISO/IEC
21122-3 [ISO21122-3], a colour specification box, as defined in
ISO/IEC 21122-3 [ISO21122-3] as well, and either one JPEG XS
codestream in the case of a progressive frame or two JPEG XS
codestreams in the case of an interlaced frame.
JPEG XS header segment
The concatenation of a video support box, as defined in ISO/IEC
21122-3 [ISO21122-3], a colour specification box, as defined in
Lugan, et al. Expires October 10, 2020 [Page 3]
Internet-Draft RTP Payload Format for JPEG XS April 2020
ISO/IEC 21122-3 as well [ISO21122-3] and a JPEG XS codestream
header.
JPEG XS stream
A sequence of JPEG XS frames.
Marker
A two-byte functional sequence that is part of a JPEG XS
codestream starting with a 0xff byte and a subsequent byte
defining its function.
Marker segment
A marker along with a 16-bit marker size and payload data
following the size.
Packetization unit
A portion of a Application Data Unit whose boundaries shall
coincide with boundaries of RTP packet payloads, i.e. the first
(resp. last) byte of a packetization unit shall be the first
(resp. last) byte of a RTP packet payload.
Slice
The smallest independently decodable unit of a JPEG XS codestream,
bearing in mind that it decodes to wavelet coefficients which
still require inverse wavelet filtering to give an image.
SOC marker
A marker that consists of the two bytes 0xff10 indicating the
start of a JPEG XS codestream.
Video support box (VS box)
A ISO video support box defined in ISO/IEC 21122-3 [ISO21122-3]
that includes metadata required to play back a JPEG XS stream,
such as its maximum bitrate, its subsampling structure, its buffer
model and its frame rate.
3. Media Format Description
3.1. Image Data Structures
JPEG XS is a low-latency lightweight image coding system for coding
continuous-tone grayscale or continuous-tone colour digital images.
This coding system provides an efficient representation of image
signals through the mathematical tool of wavelet analysis. The
wavelet filter process separates each component into multiple bands,
where each band consists of multiple coefficients describing the
image signal of a given component within a frequency domain specific
Lugan, et al. Expires October 10, 2020 [Page 4]
Internet-Draft RTP Payload Format for JPEG XS April 2020
to the wavelet filter type, i.e. the particular filter corresponding
to the band.
Wavelet coefficients are grouped into precincts, where each precinct
includes all coefficients over all bands that contribute to a spatial
region of the image.
One or multiple precincts are furthermore combined into slices
consisting of an integer number of precincts. Precincts do not cross
slice boundaries, and wavelet coefficients in precincts that are part
of different slices can be decoded independently from each other.
Note, however, that the wavelet transformation runs across slice
boundaries. A slice always extends over the full width of the image,
but may only cover parts of its height.
3.2. Codestream
A JPEG XS codestream header, followed by several slices, and
terminated by an EOC marker form a JPEG XS codestream.
The overall codestream format, including the definition of all
markers, is further defined in ISO/IEC 21122-1 [ISO21122-1]. It
represents sample values of a single image, bare any interpretation
relative to a colour space.
3.3. Video support box and colour specification box
While the information defined in the codestream is sufficient to
reconstruct the sample values of one image, the interpretation of the
samples remains undefined by the codestream itself. This
interpretation is given by the video support box and the colour
specification box which contain significant information to correctly
play the JPEG XS stream. The layout and syntax of these boxes,
together with their content, are defined in ISO/IEC 21122-3
[ISO21122-3]. The video support box provides information on the
maximum bitrate, the frame rate, the subsampling image format, the
timecode of the current JPEG XS frame, the profile, level and
sublevel used (as defined in ISO/IEC 21122-2 [ISO21122-2]), and
optionally on the buffer model and the mastering display metadata.
The colour specification box indicates the colour primaries, transfer
characteristics, matrix coefficients and video full range flag needed
to specify the colour space of the video stream.
3.4. JPEG XS Frame
The concatenation of a video support box, a colour specification box
and one or two JPEG XS codestreams forms a JPEG XS frame. In the
case of a video stream made of progressive frames, only one
Lugan, et al. Expires October 10, 2020 [Page 5]
Internet-Draft RTP Payload Format for JPEG XS April 2020
codestream follows the boxes. In the case of a video stream made of
interlaced frames, two codestreams follow the boxes, each
corresponding to a field of the interlaced frame. The video
information box included in the video support box contains a frat
field indicating if the frame is progressive or interlaced (see ISO/
IEC 21122-3 [ISO21122-3]). This information can also be found in
each RTP packet header (see Section 4.2).
4. Payload Format
This section specifies the payload format for JPEG XS streams over
the Real-time Transport Protocol (RTP) [RFC3550].
In order to be transported over RTP, each JPEG XS stream is
transported in a distinct RTP stream, identified by a distinct SSRC.
A JPEG XS stream is divided into Application Data Units (ADUs), each
ADU corresponding to a single JPEG XS frame.
4.1. RTP packetization
An ADU is made of several packetization units. If a packetization
unit is bigger than the maximum size of a RTP packet payload, the
unit is split into multiple RTP packet payloads, as illustrated in
Figure 1. As seen there, each packet shall contain (part of) one and
only one packetization unit. A packetization unit may extend over
multiple packets. The payload of every packet shall have the same
size (based e.g. on the Maximum Transfer Unit of the network), except
(possibly) the last packet of a packetization unit. The boundaries
of a packetization unit shall coincide with the boundaries of the
payload of a packet, i.e. the first (resp. last) byte of the
packetization unit shall be the first (resp. last) byte of the
payload.
Lugan, et al. Expires October 10, 2020 [Page 6]
Internet-Draft RTP Payload Format for JPEG XS April 2020
RTP +-----+------------------------+
Packet #1 | Hdr | Packetization unit #1 |
+-----+------------------------+
RTP +-----+--------------------------------------+
Packet #2 | Hdr | Packetization unit #2 |
+-----+--------------------------------------+
RTP +-----+--------------------------------------------------+
Packet #3 | Hdr | Packetization unit #3 (part 1/3) |
+-----+--------------------------------------------------+
RTP +-----+--------------------------------------------------+
Packet #4 | Hdr | Packetization unit #3 (part 2/3) |
+-----+--------------------------------------------------+
RTP +-----+----------------------------------------------+
Packet #5 | Hdr | Packetization unit #3 (part 3/3) |
+-----+----------------------------------------------+
...
RTP +-----+-----------------------------------------+
Packet #P | Hdr | Packetization unit #N (part q/q) |
+-----+-----------------------------------------+
Figure 1: Example of ADU packetization
There are two different packetization modes defined for this RTP
payload format.
1. Codestream packetization mode: in this mode, the packetization
unit shall be the entire codestream, preceeded by boxes, if any.
This means that a progressive frame will have a single
packetization unit, while an interlaced frame will have two. The
progressive case is illustrated in Figure 2.
2. Slice packetization mode: in this mode, the packetization unit
shall be the slice, i.e. there shall be data from no more than
one slice per RTP packet. The first packetization unit shall be
made of the JPEG XS header segment (i.e. the concatenation of the
VS box, the CS box and the JPEG XS codestream header). This
first unit is then followed by successive units, each containing
one and only one slice. The packetization unit containing the
last slice of a JPEG XS codestream shall also contain the EOC
marker immediately following this last slice. This is
illustrated in Figure 3. In the case of interlaced frame, the
JPEG XS codestream header of the second field shall be in its own
packetization unit.
Lugan, et al. Expires October 10, 2020 [Page 7]
Internet-Draft RTP Payload Format for JPEG XS April 2020
RTP +-----+--------------------------------------------------+
Packet #1 | Hdr | VS box + CS box + JPEG XS codestream (part 1/q) |
+-----+--------------------------------------------------+
RTP +-----+--------------------------------------------------+
Packet #2 | Hdr | JPEG XS codestream (part 2/q) |
+-----+--------------------------------------------------+
...
RTP +-----+--------------------------------------+
Packet #P | Hdr | JPEG XS codestream (part q/q) |
+-----+--------------------------------------+
Figure 2: Example of codestream packetization mode
RTP +-----+----------------------------+
Packet #1 | Hdr | JPEG XS header segment |
+-----+----------------------------+
RTP +-----+--------------------------------------------------+
Packet #2 | Hdr | Slice #1 (part 1/2) |
+-----+--------------------------------------------------+
RTP +-----+-------------------------------------------+
Packet #3 | Hdr | Slice #1 (part 2/2) |
+-----+-------------------------------------------+
RTP +-----+--------------------------------------------------+
Packet #4 | Hdr | Slice #2 (part 1/3) |
+-----+--------------------------------------------------+
...
RTP +-----+---------------------------------------+
Packet #P | Hdr | Slice #N (part q/q) + EOC marker |
+-----+---------------------------------------+
Figure 3: Example of slice packetization mode
Thanks to the constant bit-rate of JPEG XS, the codestream
packetization mode guarantees that a JPEG XS RTP stream will produce
a constant number of bytes per frame, and a constant number of RTP
packets per frame. To reach the same guarantee with the slice
packetization mode, an additional constraint needs to be set at the
rate allocation stage in the JPEG XS encoder. For instance, one
option would be to impose a constant bit-rate at the slice level.
4.2. Payload Header
Figure 4 illustrates the RTP payload header used in order to
transport a JPEG XS stream.
Lugan, et al. Expires October 10, 2020 [Page 8]
Internet-Draft RTP Payload Format for JPEG XS April 2020
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| V |P|X| CC |M| PT | sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| contributing source (CSRC) identifiers |
| .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|T|K|L| I |F counter| SEP counter | P counter |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4: RTP and payload headers
The version (V), padding (P), extension (X), CSRC count (CC),
sequence number, synchronization source (SSRC) and contributing
source (CSRC) fields follow their respective definitions in RFC 3550
[RFC3550].
The timestamp SHOULD be based on a 90 kHz clock reference.
As per specified in RFC 3550 [RFC3550] and RFC 4175 [RFC4175], the
RTP timestamp designates the sampling instant of the first octet of
the frame to which the RTP packet belongs. Packets shall not include
data from multiple frames, and all packets belonging to the same
frame shall have the same timestamp. Several successive RTP packets
will consequently have equal timestamps if they belong to the same
frame (that is until the marker bit is set to 1, marking the last
packet of the frame), and the timestamp is only increased when a new
frame begins.
If the sampling instant does not correspond to an integer value of
the clock, the value shall be truncated to the next lowest integer,
with no ambiguity.
The remaining fields are defined as follows:
Marker (M) [1 bit]:
The M bit is used to indicate the last packet of a frame. This
enables a decoder to finish decoding the frame.
Payload Type (PT) [7 bits]:
A dynamically allocated payload type field that designates the
payload as JPEG XS video.
Lugan, et al. Expires October 10, 2020 [Page 9]
Internet-Draft RTP Payload Format for JPEG XS April 2020
Transmission mode (T) [1 bit]:
The T bit is set to indicate that packets are sent sequentially by
the transmitter. A receiver could use this information to
dimension its input buffer(s) accordingly. If T=0, nothing can be
assumed about the transmission order and packets may be sent out-
of-order by the transmitter. If T=1, packets must be sent
sequentially by the transmitter.
pacKetization mode (K) [1 bit]:
The K bit is set to indicate which packetization mode is used.
K=0 indicates codestream packetization mode, while K=1 indicates
slice packetization mode. If Transmission mode (T) is set to 0,
slice packetization mode must be used and K must be set to 1.
Last (L) [1 bit]:
The L bit is set to indicate the last packet of a packetization
unit. As the end of the frame also ends the packet containing the
last unit of the frame, the L bit is set whenever the M bit is
set. In the case of a progressive frame using the codestream
packetization mode, the L bit and M bit are equivalent.
Interlaced mode (I) [2 bit]:
These 2 bits are used to indicate how the JPEG XS frame is scanned
(progressive or interlaced).
00: The payload is progressively scanned.
01: Reserved for future use.
10: The payload is part of the first field of an interlaced video
frame. The height specified in the JPEG XS picture header is
half of the height of the entire displayed image.
11: The payload is part of the second field of an interlaced
video frame. The height specified in the JPEG XS picture header
is half of the height of the entire displayed image.
F counter [5 bits]:
The frame (F) counter identifies the frame number modulo 32 to
which a packet belongs. Frame numbers are incremented by 1 for
each frame transmitted. The frame number, in addition to the time
stamp, may help the decoder manage its input buffer and bring
packets back into their natural order.
SEP counter [11 bits]:
The Slice and Extended Packet (SEP) counter is used differently
depending on the packetization mode.
Lugan, et al. Expires October 10, 2020 [Page 10]
Internet-Draft RTP Payload Format for JPEG XS April 2020
* In the case of codestream packetization mode (K=0), this
counter resets whenever the Packet counter resets (see
hereunder), and increments by 1 whenever the Packet counter
overruns.
* In the case of slice packetization mode (K=1), this counter
identifies the slice modulo 2047 to which the packet
contributes. If the data belongs to the JPEG XS header
segment, this field shall have its maximal value, namely 2047 =
0x07ff. Otherwise, it is the slice index modulo 2047. Slice
indices are counted from 0 (corresponding to the top of the
frame).
P counter [11 bits]:
The packet (P) counter identifies the packet number modulo 2048
within the current packetization unit. It is set to 0 at the
start of the packetization unit and incremented by 1 for every
subsequent packet (if any) belonging to the same unit.
Practically, if codestream packetization mode is enabled, this
field counts the packets within a codestream and is extended by
the SEP counter when it overruns. If slice packetization mode is
enabled, this field counts the packets within a slice or within
the JPEG XS header segment.
4.3. Payload Data
The payload data of a JPEG XS RTP stream consists of a concatenation
of multiple JPEG XS frames.
Each JPEG XS frame is the concatenation of one or more packetization
unit(s), as explained in Section 4.1. Figure 5 depicts this layout
for an interlaced frame in the codestream packetization mode and
Figure 6 depicts this layout for a progressive frame in the slice
packetization mode. The Frame counter value is not indicated because
the value is constant for all packetization units of a given frame.
+=====[ Packetization unit (PU) #1 ]====+
| Video support box | SEP counter = 0
| +---------------------------------+ | P counter = 0
| : Sub boxes of the VS box : |
| +---------------------------------+ |
+- - - - - - - - - - - - - - - - - - - -+
| Colour specification box |
| +---------------------------------+ |
| : Fields of the CS box : |
| +---------------------------------+ |
+- - - - - - - - - - - - - - - - - - - -+
| JPEG XS codestream (field 0) |
Lugan, et al. Expires October 10, 2020 [Page 11]
Internet-Draft RTP Payload Format for JPEG XS April 2020
: (part 1/q) : M=0, K=0, L=0, I=10
+---------------------------------------+
| JPEG XS codestream (field 0) | SEP counter = 0
| (part 2/q) | P counter = 1
: : M=0, K=0, L=0, I=10
+---------------------------------------+
| JPEG XS codestream (field 0) | SEP counter = 0
| (part 3/q) | P counter = 2
: : M=0, K=0, L=0, I=10
+---------------------------------------+
: :
+---------------------------------------+
| JPEG XS codestream (field 0) | SEP counter = 1
| (part 2049/q) | P counter = 0
: : M=0, K=0, L=0, I=10
+---------------------------------------+
: :
+---------------------------------------+
| JPEG XS codestream (field 0) | SEP counter = (q-1) div 2048
| (part q/q) | P counter = (q-1) mod 2048
: : M=0, K=0, L=1, I=10
+===============[ PU #2 ]===============+
| JPEG XS codestream (field 1) | SEP counter = 0
| (part 1/q) | P counter = 0
: : M=0, K=0, L=0, I=11
+---------------------------------------+
| JPEG XS codestream (field 1) | SEP counter = 0
| (part 2/q) | P counter = 1
: : M=0, K=0, L=0, I=11
+---------------------------------------+
: :
+---------------------------------------+
| JPEG XS codestream (field 1) | SEP counter = (q-1) div 2048
| (part q/q) | P counter = (q-1) mod 2048
: : M=1, K=0, L=1, I=11
+=======================================+
Figure 5: Example of JPEG XS Payload Data (codestream packetization
mode, interlaced frame)
+====[ PU#1: JPEG XS Header segment ]===+
| Video support box | SEP counter = 0x07FF
| +---------------------------------+ | P counter = 0
| : Sub boxes of the VS box : |
| +---------------------------------+ |
+- - - - - - - - - - - - - - - - - - - -+
| Colour specification box |
Lugan, et al. Expires October 10, 2020 [Page 12]
Internet-Draft RTP Payload Format for JPEG XS April 2020
| +---------------------------------+ |
| : Fields of the CS box : |
| +---------------------------------+ |
+- - - - - - - - - - - - - - - - - - - -+
| JPEG XS codestream header |
| +---------------------------------+ |
| : Markers and marker segments : |
| +---------------------------------+ | M=0, T=0, K=1, L=1, I=00
+==========[ PU#2: Slice #1 ]===========+
| +---------------------------------+ | SEP counter = 0
| | SLH Marker | | P counter = 0
| +---------------------------------+ |
| : Entropy Coded Data : |
| +---------------------------------+ | M=0, T=0, K=1, L=1, I=00
+==========[ PU#3: Slice #2 ]===========+
| Slice #2 | SEP counter = 1
| (part 1/q) | P counter = 0
: : M=0, T=0, K=1, L=0, I=00
+---------------------------------------+
| Slice #2 | SEP counter = 1
| (part 2/q) | P counter = 1
: : M=0, T=0, K=1, L=0, I=00
+---------------------------------------+
: :
+---------------------------------------+
| Slice #2 | SEP counter = 1
| (part q/q) | P counter = q-1
: : M=0, T=0, K=1, L=1, I=00
+=======================================+
: :
+========[ PU#N: Slice #(N-1) ]=========+
| Slice #(N-1) | SEP counter = N-2
| (part 1/r) | P counter = 0
: : M=0, T=0, K=1, L=0, I=00
+---------------------------------------+
: :
+---------------------------------------+
| Slice #(N-1) | SEP counter = N-2
| (part r/r) | P counter = r-1
: + EOC marker : M=1, T=0, K=1, L=1, I=00
+=======================================+
Figure 6: Example of JPEG XS Payload Data (slice packetization mode,
progressive frame)
Lugan, et al. Expires October 10, 2020 [Page 13]
Internet-Draft RTP Payload Format for JPEG XS April 2020
4.4. Traffic Shaping and Delivery Timing
The traffic shaping and delivery timing shall be in accordance with
the Network Compatibility Model compliance definitions specified in
SMPTE ST 2110-21 [SMPTE-ST2110-21] for either Narrow Linear Senders
(Type NL) or Wide Senders (Type W). The session description shall
include a format-specific parameter of either TP=2110TPNL or
TP=2110TPW to indicate compliance with Type NL or Type W
respectively.
NOTE: The Virtual Receiver Buffer Model compliance definitions of ST
2110-21 do not apply.
5. Congestion Control Considerations
Congestion control for RTP SHALL be used in accordance with RFC 3550
[RFC3550], and with any applicable RTP profile: e.g., RFC 3551
[RFC3551]. An additional requirement if best-effort service is being
used is users of this payload format MUST monitor packet loss to
ensure that the packet loss rate is within acceptable parameters.
Circuit Breakers [RFC8083] is an update to RTP [RFC3550] that defines
criteria for when one is required to stop sending RTP Packet Streams
and applications implementing this standard MUST comply with it. RFC
8085 [RFC8085] provides additional information on the best practices
for applying congestion control to UDP streams.
6. Payload Format Parameters
6.1. Media Type Definition
Type name: video
Subtype name: jxsv
Required parameters:
rate: The RTP timestamp clock rate. Applications using this
payload format SHOULD use a value of 90000.
transmission mode: Indicates if packets are sent sequentially by
the transmitter. A receiver could use this information to
dimension its input buffer(s) accordingly. If set to 0, nothing
can be assumed about the transmission order and packets may be sent
out-of-order. If value is 1, packets must be sent sequentially by
the transmitter.
Optional parameters:
Lugan, et al. Expires October 10, 2020 [Page 14]
Internet-Draft RTP Payload Format for JPEG XS April 2020
profile: The JPEG XS profile in use, as defined in ISO/IEC 21122-2
(JPEG XS Part 2) [ISO21122-2].
level: The JPEG XS level in use, as defined in ISO/IEC 21122-2
(JPEG XS Part 2) [ISO21122-2].
sublevel: The JPEG XS sublevel in use, as defined in ISO/IEC
21122-2 (JPEG XS Part 2) [ISO21122-2].
sampling: Signals the colour difference signal sub-sampling
structure.
Signals utilizing the non-constant luminance Y'C'B C'R signal
format of Recommendation ITU-R BT.601-7, Recommendation ITU-R
BT.709-6, Recommendation ITU-R BT.2020-2, or Recommendation ITU-R
BT.2100 shall use the appropriate one of the following values for
the Media Type Parameter "sampling":
YCbCr-4:4:4 (4:4:4 sampling)
YCbCr-4:2:2 (4:2:2 sampling)
YCbCr-4:2:0 (4:2:0 sampling)
Signals utilizing the Constant Luminance Y'C C'BC C'RC signal
format of Recommendation ITU-R BT.2020-2 shall use the appropriate
one of the following values for the Media Type Parameter
"sampling":
CLYCbCr-4:4:4 (4:4:4 sampling)
CLYCbCr-4:2:2 (4:2:2 sampling)
CLYCbCr-4:2:0 (4:2:0 sampling)
Signals utilizing the constant intensity I CT CP signal format of
Recommendation ITU-R BT.2100 shall use the appropriate one of the
following values for the Media Type Parameter "sampling":
ICtCp-4:4:4 (4:4:4 sampling)
ICtCp-4:2:2 (4:2:2 sampling)
ICtCp-4:2:0 (4:2:0 sampling)
Signals utilizing the 4:4:4 R' G' B' or RGB signal format (such as
that of Recommendation ITU-R BT.601, Recommendation ITU-R BT.709,
Recommendation ITU-R BT.2020, Recommendation ITU-R BT.2100, SMPTE
ST 2065-1 or ST 2065-3) shall use the following value for the Media
Type Parameter sampling.
RGB RGB or R' G' B' samples
Lugan, et al. Expires October 10, 2020 [Page 15]
Internet-Draft RTP Payload Format for JPEG XS April 2020
Signals utilizing the 4:4:4 X' Y' Z' signal format (such as defined
in SMPTE ST 428-1) shall use the following value for the Media Type
Parameter sampling.
XYZ X' Y' Z' samples
Key signals as defined in SMPTE RP 157 shall use the value key for
the Media Type Parameter sampling. The Key signal is represented
as a single component.
KEY samples of the key signal
depth: Determines the number of bits per sample. This is an
integer with typical values including 8, 10, 12, and 16.
width: Determines the number of pixels per line. This is an
integer between 1 and 32767.
height: Determines the number of lines per frame. This is an
integer between 1 and 32767.
exactframerate: Signals the frame rate in frames per second.
Integer frame rates shall be signaled as a single decimal number
(e.g. "25") whilst non-integer frame rates shall be signaled as a
ratio of two integer decimal numbers separated by a "forward-slash"
character (e.g. "30000/1001"), utilizing the numerically smallest
numerator value possible.
colorimetry: Specifies the system colorimetry used by the image
samples. Valid values and their specification are:
BT601-5 ITU Recommendation BT.601-5
BT709-2 ITU Recommendation BT.709-2
SMPTE240M SMPTE standard 240M
BT601 as specified in Recommendation ITU-R BT.601-7
BT709 as specified in Recommendation ITU-R BT.709-6
BT2020 as specified in Recommendation ITU-R BT.2020-2
BT2100 as specified in Recommendation ITU-R BT.2100
Table 2 titled "System colorimetry"
ST2065-1 as specified in SMPTE ST 2065-1 Academy Color
Encoding Specification (ACES)
ST2065-3 as specified for Academy Density Exchange
Encoding (ADX) in SMPTE ST 2065-3
XYZ as specified in ISO 11664-1 section titled
"1931 Observer"
Lugan, et al. Expires October 10, 2020 [Page 16]
Internet-Draft RTP Payload Format for JPEG XS April 2020
Signals utilizing the Recommendation ITU-R BT.2100 colorimetry
should also signal the representational range using the optional
parameter RANGE defined below.
interlace: If this OPTIONAL parameter name is present, it indicates
that the video is interlaced. If this parameter name is not
present, the progressive video format shall be assumed.
TCS: Transfer Characteristic System. This parameter specifies the
transfer characteristic system of the image samples. Valid values
and their specification are:
SDR (Standard Dynamic Range) Video streams of standard
dynamic range, that utilize the OETF of Recommendation
ITU-R BT.709 or Recommendation ITU-R BT.2020. Such
streams shall be assumed to target the EOTF specified
in ITU-R BT.1886.
PQ Video streams of high dynamic range video that utilize
the Perceptual Quantization system of Recommendation
ITU-R BT.2100
HLG Video streams of high dynamic range video that utilize
the Hybrid Log-Gamma system of Recommendation ITU-R
BT.2100
RANGE: This parameter should be used to signal the encoding range
of the sample values within the stream. When paired with ITU Rec
BT.2100 colorimetry, this parameter has two allowed values NARROW
and FULL, corresponding to the ranges specified in table 9 of ITU
Rec BT.2100. In any other context, this parameter has three
allowed values: NARROW, FULLPROTECT, and FULL, which correspond to
the ranges specified in SMPTE RP 2077. In the absence of this
parameter, NARROW shall be the assumed value in either case.
Encoding considerations:
This media type is framed and binary; see Section 4.8 in RFC 6838
[RFC6838].
Security considerations:
Please see the Security Considerations section in RFC XXXX
6.2. Mapping to SDP
6.2.1. General
A Session Description Protocol (SDP) object shall be created for each
RTP stream and it shall be in accordance with the provisions of SMPTE
ST 2110-10 [SMPTE-ST2110-10].
Lugan, et al. Expires October 10, 2020 [Page 17]
Internet-Draft RTP Payload Format for JPEG XS April 2020
The information carried in the media type specification has a
specific mapping to fields in the Session Description Protocol (SDP),
which is commonly used to describe RTP sessions.
6.2.2. Media type and subtype
The media type ("video") goes in SDP "m=" as the media name.
The media subtype ("jxsv") goes in SDP "a=rtpmap" as the encoding
name, followed by a slash ("/") and the required parameter "rate"
corresponding to the RTP timestamp clock rate (which for the payload
format defined in this document SHOULD be 90000), followed by a slash
("/") and the required parameter "transmission mode" set to 1 if
packets are sent sequentially by the transmitter, or 0 if
transmission order is not constrained. The optional parameters go in
the SDP "a=fmtp" attribute by copying them directly from the MIME
media type string as a semicolon-separated list of parameter=value
pairs.
A sample SDP mapping for JPEG XS video is as follows:
m=video 30000 RTP/AVP 112
a=rtpmap:112 jxsv/90000/1
a=fmtp:112 sampling=YCbCr-4:2:2; width=1920; height=1080;
depth=10; colorimetry=BT709; TCS=SDR;
RANGE=FULL; TP=2110TPNL
In this example, a JPEG XS RTP stream is being sent to UDP
destination port 30000, with an RTP dynamic payload type of 112 and a
media clock rate of 90000 Hz. Note that the "a=fmtp:" line has been
wrapped to fit this page, and will be a single long line in the SDP
file.
6.2.3. Traffic shaping
The SDP object shall include the TP parameter (either 2110TPNL or
2110TPW as specified in Section 4.4) and may include the CMAX
parameter as specified in SMPTE ST 2110-21 [SMPTE-ST2110-21].
6.2.4. Offer/Answer Considerations
The following considerations apply when using SDP offer/answer
procedures [RFC3264] to negotiate the use of the JPEG XS payload in
RTP:
o The "encode" parameter can be used for sendrecv, sendonly, and
recvonly streams. Each encode type MUST use a separate payload
type number.
Lugan, et al. Expires October 10, 2020 [Page 18]
Internet-Draft RTP Payload Format for JPEG XS April 2020
o Any unknown parameter in an offer MUST be ignored by the receiver
and MUST NOT be included in the answer.
7. IANA Considerations
This memo requests that IANA registers video/jxsv as specified in
Section 6.1. The media type is also requested to be added to the
IANA registry for "RTP Payload Format MIME types" [1].
8. Security Considerations
RTP packets using the payload format defined in this specification
are subject to the security considerations discussed in the RTP
specification [RFC3550] and in any applicable RTP profile such as
RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711], or RTP/
SAVPF [RFC5124]. This implies that confidentiality of the media
streams is achieved by encryption.
However, as "Securing the RTP Framework: Why RTP Does Not Mandate a
Single Media Security Solution" [RFC7202] discusses, it is not an RTP
payload format's responsibility to discuss or mandate what solutions
are used to meet the basic security goals like confidentiality,
integrity, and source authenticity for RTP in general. This
responsibility lies on anyone using RTP in an application. They can
find guidance on available security mechanisms and important
considerations in "Options for Securing RTP Sessions" [RFC7201].
Applications SHOULD use one or more appropriate strong security
mechanisms.
This payload format and the JPEG XS encoding do not exhibit any
substantial non-uniformity, either in output or in complexity to
perform the decoding operation and thus are unlikely to pose a
denial-of-service threat due to the receipt of pathological
datagrams.
It is important to note that HD or UHDTV JPEG XS-encoded video can
have significant bandwidth requirements (typically more than 1 Gbps
for ultra high-definition video, especially if using high framerate).
This is sufficient to cause potential for denial-of-service if
transmitted onto most currently available Internet paths.
Accordingly, if best-effort service is being used, users of this
payload format MUST monitor packet loss to ensure that the packet
loss rate is within acceptable parameters. Packet loss is considered
acceptable if a TCP flow across the same network path, and
experiencing the same network conditions, would achieve an average
throughput, measured on a reasonable timescale, that is not less than
the RTP flow is achieving. This condition can be satisfied by
Lugan, et al. Expires October 10, 2020 [Page 19]
Internet-Draft RTP Payload Format for JPEG XS April 2020
implementing congestion control mechanisms to adapt the transmission
rate (or the number of layers subscribed for a layered multicast
session), or by arranging for a receiver to leave the session if the
loss rate is unacceptably high.
This payload format may also be used in networks that provide
quality-of-service guarantees. If enhanced service is being used,
receivers SHOULD monitor packet loss to ensure that the service that
was requested is actually being delivered. If it is not, then they
SHOULD assume that they are receiving best-effort service and behave
accordingly.
9. RFC Editor Considerations
Note to RFC Editor: This section may be removed after carrying out
all the instructions of this section.
RFC XXXX is to be replaced by the RFC number this specification
receives when published.
10. References
10.1. Normative References
[ISO21122-1]
International Organization for Standardization (ISO) -
International Electrotechnical Commission (IEC),
"Information technology - JPEG XS low-latency lightweight
image coding system - Part 1: Core coding system", ISO/
IEC PRF 21122-1, under development,
<https://www.iso.org/standard/74535.html>.
[ISO21122-2]
International Organization for Standardization (ISO) -
International Electrotechnical Commission (IEC),
"Information technology - JPEG XS low-latency lightweight
image coding system - Part 2: Profiles and buffer models",
ISO/IEC PRF 21122-2, under development,
<https://www.iso.org/standard/74536.html>.
[ISO21122-3]
International Organization for Standardization (ISO) -
International Electrotechnical Commission (IEC),
"Information technology - JPEG XS low-latency lightweight
image coding system - Part 3: Transport and container
formats", ISO/IEC FDIS 21122-3, under development,
<https://www.iso.org/standard/74537.html>.
Lugan, et al. Expires October 10, 2020 [Page 20]
Internet-Draft RTP Payload Format for JPEG XS April 2020
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
with Session Description Protocol (SDP)", RFC 3264,
DOI 10.17487/RFC3264, June 2002,
<https://www.rfc-editor.org/info/rfc3264>.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
July 2003, <https://www.rfc-editor.org/info/rfc3550>.
[RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and
Video Conferences with Minimal Control", STD 65, RFC 3551,
DOI 10.17487/RFC3551, July 2003,
<https://www.rfc-editor.org/info/rfc3551>.
[RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K.
Norrman, "The Secure Real-time Transport Protocol (SRTP)",
RFC 3711, DOI 10.17487/RFC3711, March 2004,
<https://www.rfc-editor.org/info/rfc3711>.
[RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type
Specifications and Registration Procedures", BCP 13,
RFC 6838, DOI 10.17487/RFC6838, January 2013,
<https://www.rfc-editor.org/info/rfc6838>.
[RFC8083] Perkins, C. and V. Singh, "Multimedia Congestion Control:
Circuit Breakers for Unicast RTP Sessions", RFC 8083,
DOI 10.17487/RFC8083, March 2017,
<https://www.rfc-editor.org/info/rfc8083>.
[RFC8085] Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage
Guidelines", BCP 145, RFC 8085, DOI 10.17487/RFC8085,
March 2017, <https://www.rfc-editor.org/info/rfc8085>.
[SMPTE-ST2110-10]
Society of Motion Picture and Television Engineers, "SMPTE
Standard - Professional Media Over Managed IP Networks:
System Timing and Definitions", SMPTE ST 2110-10:2017,
2017, <https://doi.org/10.5594/SMPTE.ST2110-10.2017>.
Lugan, et al. Expires October 10, 2020 [Page 21]
Internet-Draft RTP Payload Format for JPEG XS April 2020
[SMPTE-ST2110-21]
Society of Motion Picture and Television Engineers, "SMPTE
Standard - Professional Media Over Managed IP Networks:
Traffic Shaping and Delivery Timing for Video", SMPTE ST
2110-21:2017, 2017,
<https://doi.org/10.5594/SMPTE.ST2110-21.2017>.
10.2. Informative References
[RFC4175] Gharai, L. and C. Perkins, "RTP Payload Format for
Uncompressed Video", RFC 4175, DOI 10.17487/RFC4175,
September 2005, <https://www.rfc-editor.org/info/rfc4175>.
[RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
"Extended RTP Profile for Real-time Transport Control
Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
DOI 10.17487/RFC4585, July 2006,
<https://www.rfc-editor.org/info/rfc4585>.
[RFC5124] Ott, J. and E. Carrara, "Extended Secure RTP Profile for
Real-time Transport Control Protocol (RTCP)-Based Feedback
(RTP/SAVPF)", RFC 5124, DOI 10.17487/RFC5124, February
2008, <https://www.rfc-editor.org/info/rfc5124>.
[RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP
Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014,
<https://www.rfc-editor.org/info/rfc7201>.
[RFC7202] Perkins, C. and M. Westerlund, "Securing the RTP
Framework: Why RTP Does Not Mandate a Single Media
Security Solution", RFC 7202, DOI 10.17487/RFC7202, April
2014, <https://www.rfc-editor.org/info/rfc7202>.
10.3. URIs
[1] http://www.iana.org/assignments/rtp-parameters
Authors' Addresses
Sebastien Lugan
intoPIX S.A.
Rue Emile Francqui, 9
1435 Mont-Saint-Guibert
Belgium
Phone: +32 10 23 84 70
Email: rtp@intopix.com
URI: http://www.intopix.com
Lugan, et al. Expires October 10, 2020 [Page 22]
Internet-Draft RTP Payload Format for JPEG XS April 2020
Antonin Descampe
intoPIX S.A.
Rue Emile Francqui, 9
1435 Mont-Saint-Guibert
Belgium
Phone: +32 10 23 84 70
Email: a.descampe@intopix.com
URI: http://www.intopix.com
Corentin Damman
intoPIX S.A.
Rue Emile Francqui, 9
1435 Mont-Saint-Guibert
Belgium
Phone: +32 10 23 84 70
Email: c.damman@intopix.com
URI: http://www.intopix.com
Thomas Richter
Fraunhofer IIS
Am Wolfsmantel 33
91048 Erlangen
Germany
Phone: +49 9131 776 5126
Email: thomas.richter@iis.fraunhofer.de
URI: https://www.iis.fraunhofer.de/
Alexandre Willeme
Universite catholique de Louvain
Place du Levant, 2 - bte L5.04.04
1348 Louvain-la-Neuve
Belgium
Phone: +32 10 47 80 82
Email: alexandre.willeme@uclouvain.be
URI: https://uclouvain.be/en/icteam
Lugan, et al. Expires October 10, 2020 [Page 23]
