Skip to main content

Scalable Quality Extension for the Opus Codec (Opus HD)
draft-ietf-mlcodec-opus-scalable-quality-extension-02

Document Type Active Internet-Draft (mlcodec WG)
Author Jean-Marc Valin
Last updated 2026-08-27 (Latest revision 2026-06-12)
Replaces draft-valin-opus-scalable-quality-extension
RFC stream Internet Engineering Task Force (IETF)
Intended RFC status Proposed Standard
Formats
Additional resources Mailing list discussion
Stream WG state In WG Last Call
Document shepherd (None)
IESG IESG state I-D Exists
Consensus boilerplate Yes
Telechat date (None)
Responsible AD (None)
Send notices to (None)
draft-ietf-mlcodec-opus-scalable-quality-extension-02
mlcodec                                                        JM. Valin
Internet-Draft                                                    Google
Updates: 6716 (if approved)                                 12 June 2026
Intended status: Standards Track                                        
Expires: 14 December 2026

        Scalable Quality Extension for the Opus Codec (Opus HD)
         draft-ietf-mlcodec-opus-scalable-quality-extension-02

Abstract

   This document updates [RFC6716] to add support for a scalable quality
   layer.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 14 December 2026.

Copyright Notice

   Copyright (c) 2026 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Valin                   Expires 14 December 2026                [Page 1]
Internet-Draft         Scalable Quality Extension              June 2026

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   2
   2.  Scalable Quality Extension  . . . . . . . . . . . . . . . . .   2
     2.1.  Extended resolution . . . . . . . . . . . . . . . . . . .   3
       2.1.1.  Fine energy quantizer . . . . . . . . . . . . . . . .   3
       2.1.2.  PVQ . . . . . . . . . . . . . . . . . . . . . . . . .   4
       2.1.3.  Angle quantizer . . . . . . . . . . . . . . . . . . .   4
       2.1.4.  Cubic quantizer . . . . . . . . . . . . . . . . . . .   5
     2.2.  Extended frequency range  . . . . . . . . . . . . . . . .   5
     2.3.  Bit allocation  . . . . . . . . . . . . . . . . . . . . .   7
     2.4.  Time-domain processing at 96 kHz  . . . . . . . . . . . .   7
   3.  Format  . . . . . . . . . . . . . . . . . . . . . . . . . . .   8
   4.  Conformance . . . . . . . . . . . . . . . . . . . . . . . . .   8
     4.1.  Decoder . . . . . . . . . . . . . . . . . . . . . . . . .   9
     4.2.  Encoder . . . . . . . . . . . . . . . . . . . . . . . . .   9
   5.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  10
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .  10
   7.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  10
     7.1.  Normative References  . . . . . . . . . . . . . . . . . .  10
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  11

1.  Introduction

   This document updates [RFC6716] to add support for a scalable quality
   extension layer.  Implementations conforming to this document will be
   referred to as Opus HD.

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

2.  Scalable Quality Extension

   The Opus codec was designed to operate at sampling frequencies up to
   48 kHz, with an audio bandwidth up to 20 kHz.  The CELT mode that is
   used for high bitrate coding uses vector quantization with a mostly
   implicit bit allocation system that is dictated by the bitstream
   definition.  Opus can allocate up to 8 bits per MDCT bin in some of
   the bands.

Valin                   Expires 14 December 2026                [Page 2]
Internet-Draft         Scalable Quality Extension              June 2026

   While Opus capabilities listed above are sufficient to achieve
   perceptually transparent audio coding, there is a use for codecs that
   scale beyond those specs.  That includes the current market for
   24-bit/96 kHz codecs, but also any application where the intended
   recipient is not (only) a human being, e.g. ultra-sonic applications.

   This document proposes a scalable quality extension layer that both
   increases the resolution of existing Opus quantizers below 20 kHz,
   and defines a way of coding audio above 20 kHz, with a sampling rate
   of 96 kHz.  The extension is designed to be forward and backward
   compatible with [RFC6716].  All extra bits use the Opus extension
   mechanism defined in [opus-extension] and a 96 kHz decoder is
   designed to decode a regular 48 kHz [RFC6716] stream and vice versa.

   The code implementing this draft is available on the main branch of
   the Opus repository at https://gitlab.xiph.org/xiph/opus/ and
   requires building with --enable-qext (or defining ENABLE_QEXT).

2.1.  Extended resolution

   To reduce the coding error, we need to increase the resolution for 3
   different quantizers: the fine energy quantizer (scalar), the band
   pyramid vector quantizer (PVQ), and the band splitting angle
   quantizer.  We also introduce a new cubic quantizer that scales to
   higher bit depths than PVQ.  To preserve compatibility, all of the
   bits extending the Opus resolution are stored in the extension
   payload.

   As bands are split into sub-vectors (during stereo coupling or band
   partitioning), the extension bit budget is split equally between the
   two partitions.  Because the base Opus layer already accounts for the
   relative magnitudes of the sub-vectors in its own allocation offset,
   a simple equal split is sufficient to maintain a uniform refinement
   depth across the two halves.

   For each sub-vector of dimension N, the refinement bit depth is
   derived by dividing its budget by the degrees of freedom (N-1 for
   PVQ, or 2*N-1 for the angle quantizer).  In both cases, the depth is
   clamped between 2 and 14 bits (or 0 if the budget is insufficient).
   For PVQ sub-vectors, the depth is also reduced if the remaining
   payload budget is too small to fit the worst-case refinement.

2.1.1.  Fine energy quantizer

   For each band we can increase the resolution of the fine energy
   quantizer by adding extra bits.  The extra bits are added in the same
   way as the regular fine energy quantizer adds resolution on top of
   the coarse energy quantizer.

Valin                   Expires 14 December 2026                [Page 3]
Internet-Draft         Scalable Quality Extension              June 2026

2.1.2.  PVQ

   From a size-K PVQ codebook in N dimensions we can create an extended
   codebook of size u*K, where u is always odd and selected as 2^b-1,
   where b is the extra depth.  Let y_i be the (integer) value for
   dimension i of the size-K codebook and z_i be the corresponding value
   for the size-u*K codebook.  We define a refinement r_i = z_i - u*y_i
   where |r_i| < u.  Only the refinement r_i needs to be coded since the
   regular Opus bitstream already includes y_i.

   In the N=2 special case, only the single refinement r_0 is coded as a
   uniform integer in [-(u-1)/2, (u-1)/2].  If the base layer has y_1 !=
   0, the refined components are reconstructed as: z_0 = y_0 * u + r_0 *
   sgn(y_1) and z_1 = y_1 * u - r_0 * sgn(y_0).  If the base layer has
   y_1 = 0, the sign of the refined component z_1 is implicit in r_0:
   z_0 = y_0 * u - |r_0| * sgn(y_0) and z_1 = -r_0 * sgn(y_0).  No
   additional sign bits are coded for N=2.

   For N>2, even though all |r_i| < u are allowed, smaller values of r_i
   are more likely, so we benefit from entropy coding r_i.  We assume
   that the likelihood of |r_i| < (u+1)/2 is 7/8 and use that
   probability for decoding a "large" flag.  If large=0, we decode b
   bits and subtract (u-1)/2 to get r_i.  The maximum value 2^b-1 maps
   to r_i = (u+1)/2 and aliases with a large=1 case.  If large=1, we
   decode a sign bit, followed by an integer with b-1 bits to which we
   add (u+1)/2 and apply the sign.  To prevent exceeding the bit budget,
   the encoder and decoder calculate the worst-case size of the
   refinement, which is (N-1)*(b+3)+1 bits.  If the remaining bit budget
   in the extension payload is less than or equal to this worst-case
   bound, the probability for the 'large' flag falls back to 1/2
   (uncompressed uniform coding) to guarantee that the budget is
   respected.  The last residual value r_{N-1} does not need to be coded
   since it's value can be inferred from the other values and the
   knowledge that the sum of the absolute values is u*K.  The only
   exception for N > 2 is when y_{N-1}=0, in which case a single sign
   bit is coded, but the magnitude is still inferred.

2.1.3.  Angle quantizer

   When using mid-side stereo or when splitting a band, we code an angle
   representing the atan of two sub-vectors' magnitude ratio.  The
   standard Opus encoder can code angles with up to 8 bits.  In a
   similar way to how we define the PVQ refinement, we pick u = 2^b-1
   where u is the number of (equidistant) extra quantization levels to
   be added between each of the original levels.  We code a uint symbol
   between 0 and u-1, where 0 is almost mid-point to the previous
   (lower) quantization level, u-1 is almost mid-point to the next
   (higher) level, and (u-1)/2 perfectly lines up with with the

Valin                   Expires 14 December 2026                [Page 4]
Internet-Draft         Scalable Quality Extension              June 2026

   originally selected quantization of the standard Opus layer.

2.1.4.  Cubic quantizer

   The existing Opus PVQ only scales up to 32-bit codebooks.  For cases
   where there is no PVQ in the base Opus layer, we define a new cubic
   quantizer.  Whereas the PVQ codebook is defined as a reflected
   simplex warped onto the unit sphere, the cubic quantizer warps an
   N-dimensional cubic shell to the same unit sphere.  Cubic codewords
   specify which face of the cube the vector lies on by coding the
   dimension of the largest component as a uniform range-coded integer
   from 0 to N-1, followed by a 1-bit sign of that component.  The face
   of an N-dimensional hyper-cube shell is a full N-1-dimensional cube
   and can be coded with N-1 scalar values from 0 to Q-1.  To minimize
   complexity, each scalar is encoded directly using b bits, consuming
   exactly (N-1)*b bits in the bitstream.  We use even Q (Q=2^b) for
   non-transient bands (B==1) and odd Q (Q=2^b-1) for transient bands
   (B>1).

   For transient bands, the maximum possible bitwise value (2^b-1) is
   simply left unused by the encoding.  If a decoder encounters this
   unused value, it MUST process it normally without applying any
   clamping.  In the cubic synthesis process, this evaluates to a
   coordinate slightly outside the bounds of the hyper-cube face.
   Because the entire reconstructed vector is subsequently
   L_2-normalized to the target band energy, this evaluates to a safe,
   bounded position on the unit sphere.  Processing the value blindly
   prevents the need for branching logic in the decoder's critical path.

2.2.  Extended frequency range

   To extend the audio bandwidth, we define more frequency bands.
   Because psychoacoustics is no longer involved past 20 kHz, all new
   bands are defined to have a width of 2 kHz.  Therefore, when encoding
   48-kHz content we add 2 extra bands and when encoding 96-kHz content,
   we add 14 extra bands.  A flag is encoded to specify whether 2 or 14
   bands are added.  The decoder uses that flag to know how many bands
   to decode, regardless of whether decoding at 48 or 96 kHz.

   The exact band boundaries in terms of MDCT bin indices depend on the
   short MDCT block size (N_{short}):

   *  For 2.5 ms short blocks, (corresponding to N_{short} = 240 at 96
      kHz, or N_{short} = 120 at 48 kHz), the boundaries of the 14 extra
      bands are: 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200,
      210, 220, 230, 240.

Valin                   Expires 14 December 2026                [Page 5]
Internet-Draft         Scalable Quality Extension              June 2026

   *  For 1.875 ms short blocks (corresponding to a N_{short} = 180 at
      96 kHz, or N_{short} = 90 at 48 kHz), the boundaries of the 14
      extra bands are: 74, 82, 90, 98, 106, 114, 122, 130, 138, 146,
      154, 162, 168, 174, 180.

   For a 48 kHz stream, only the first 2 bands (up to index 120 or 90)
   are decoded.

   For these extra bands, coarse energy, fine energy, and spectral
   coefficients are coded in a similar sequence to standard CELT bands
   in [RFC6716]:

   1.  An intra-frame energy flag is decoded first, using a probability
       of 1/8 of being 1.  If set, coarse energy is intra-frame coded;
       otherwise, it is predicted from the previous frame.  Coarse
       energy decoding follows standard CELT coarse energy procedures
       (Section 4.3.2 of [RFC6716]).

   2.  Fine energy is decoded for each band using the allocated
       extra_quant bits (derived from the bit allocation depth described
       in Section 2.3).

   3.  The normalized spectral coefficients are decoded from scratch
       using either plain PVQ or the cubic quantizer.  The cubic
       quantizer is used instead of plain PVQ if the allocation allows
       for a cubic quantizer with a depth of at least 3 bits per
       dimension.  This corresponds to a threshold of 3*N + log2(N) + 1
       bits, where N is the size of the band.  If fewer bits than the
       threshold are available, then regular PVQ is used for the band.

   When the cubic quantizer is selected for a band, if the block size is
   larger than 2.5 ms (where the block size index LM > 0) and the budget
   b (in bits) is greater than 2*N, the band is recursively split into
   two halves.  An angle theta is coded to represent the energy
   distribution between the two halves.  The resolution of theta is
   derived from the budget and the size of the band.  The remaining
   budget (after coding theta) is split between the two halves: b_1 =
   min(b, max(0, floor((b - delta)/2))) and b_2 = b - b_1, where delta
   is a linear approximation of the standard CELT budget offset
   (described in Section 4.3.5 of [RFC6716]): delta = (N-1) * 2.875 *
   (theta / (pi/2) - 0.5).  The two halves are then recursively
   quantized, with their gains scaled by cos(theta) and sin(theta)
   respectively.

   All entropy decoding for these extra bands uses the extension decoder
   context.

Valin                   Expires 14 December 2026                [Page 6]
Internet-Draft         Scalable Quality Extension              June 2026

2.3.  Bit allocation

   The allocation of the extra bit depth b is explicitly signaled for
   each band at a time, using a resolution of 1/4 bit depth between 0
   and a cap c=14.  For band b_i, we use entropy coding to give a higher
   probability to three different cases: b_i=0, b_i=4*c, and
   b_i=b_{i-1}.  In the case where b_{i-1} is either 0 or 4*c, we merge
   two of the probabilities.  The ICDF for the general case is {120,
   112, 70, 0}, where the first symbol means b_i=0, the second means
   b_i=4*c, the third means b_i=b_{i-1}, and the last symbol means that
   b_i is equal to 1 plus a uint value coded from 0 to 4*c-1.  For
   b_{i-1} = 0, we use the ICDF {64, 50, 0} and for b_{i-1}=4*c, we use
   {110, 60, 0}, where the last symbol always means that a uint is
   coded.  We start with b_{-1} = 0.

   Given b_i, the number of extra energy bits is given by (b_i+3)/4.
   The number of 1/8 bits (BITRES) allocated for PVQ refinement and/or
   cubic codebook bits is given by ((W-1)*channels * b_i * 8 + 2)/4,
   where W is the number of bins in the band.

   To guarantee decoder synchronization, both the encoder and decoder
   check the remaining bit budget in the extension payload before
   decoding the bit depth b_i of each band.  If the remaining budget is
   less than 10 bits (80 in 1/8th bit units), the decoding of bit depths
   is terminated, and b_i is set to 0 for all remaining bands.

   The dynamic bit sharing (re-balancing) mechanism for the extension
   payload follows the same algorithm as standard CELT (described in
   Section 4.3.3 of [RFC6716]), replacing the base stream variables with
   their extension payload equivalents.

2.4.  Time-domain processing at 96 kHz

   CELT includes two time-domain filter pairs that require updating for
   96 kHz: the preemphasis/deempahsis filters, as well as the pitch
   prefilter/postfilter.  The CELT deemphasis filter is currently
   defined as D(z)=1/(1 - a1*z^-1) for a 48 kHz signal, where
   a1=27853/32768.  To obtain approximately the same response in the
   0-20 kHz range using a sampling rate of 96 kHz, we instead use
   D(z)=g*(1 - b1*z^-1)/(1 - a1*z^-1), where g=5415/8192, b1=7209/32768,
   a1=30245/32768.

   For the pitch pre-filter/post-filter, we use zero-insertion
   upsampling of the 48 kHz filters, which results in the same frequency
   response below 24 kHz and a "folded" image above 24 kHz.  The
   standard CELT postfilter in the Z-domain is defined as: P(z) = 1/(1 -
   G*(g_0*z^{-T} + g_1*(z^{-T+1} + z^{-T-1}) + g_2*(z^{-T+2} +
   z^{-T-2}))) where T is the pitch period, G is the global postfilter

Valin                   Expires 14 December 2026                [Page 7]
Internet-Draft         Scalable Quality Extension              June 2026

   gain, and g_0, g_1, g_2 are the tapset gains.  For the same pitch at
   96 kHz, the filter becomes: P(z) = 1/(1 - G*(g_0*z^{-2T} +
   g_1*(z^{-2T+2} + z^{-2T-2}) + g_2*(z^{-2T+4} + z^{-2T-4})))
   representing symmetric taps at delays (2T-2), (2T), and (2T+2) within
   the sub-streams.

3.  Format

   The extension payload is entropy-coded in the following order

     +============================+==================================+
     | Symbol(s)                  | PDF/Description                  |
     +============================+==================================+
     | 96 kHz flag                | {1, 1}/2                         |
     +----------------------------+----------------------------------+
     | Intensity stereo           | uint                             |
     +----------------------------+----------------------------------+
     | Dual stereo                | {1, 1}/2, only if Intensity != 0 |
     +----------------------------+----------------------------------+
     | Intra coarse energy        | {7, 1}/8                         |
     +----------------------------+----------------------------------+
     | Coarse energy (high bands) | Section 2.2                      |
     +----------------------------+----------------------------------+
     | Bit allocation             | Section 2.3                      |
     +----------------------------+----------------------------------+
     | Fine energy (low bands)    | Section 2.1.1, Section 2.3       |
     +----------------------------+----------------------------------+
     | PVQ refinement             | Section 2.1.2, Section 2.1.3     |
     +----------------------------+----------------------------------+
     | Fine energy (high bands)   | Section 2.1.1, Section 2.3       |
     +----------------------------+----------------------------------+
     | PVQ and cubic codebook     | Section 2.1.2, Section 2.1.4     |
     | (high bands)               |                                  |
     +----------------------------+----------------------------------+

                                  Table 1

4.  Conformance

   This section defines some tests for evaluating Opus HD conformance.
   The evaluations are based on test vectors, along with a custom-made
   comparison tool named qext_compare and derived from the original
   opus_compare tool from [RFC6716].

Valin                   Expires 14 December 2026                [Page 8]
Internet-Draft         Scalable Quality Extension              June 2026

4.1.  Decoder

   For a decoder to conform to this specification, its output MUST be
   within the specified bounds for all testvectors when compared using
   qext_compare.  Two sets of testvectors are provided.  The first,
   qext_vector01.bit through qext_vector06.bit are high-quality 1024 kb/
   s bitstreams for which the decoder target files are qext_vector01.f32
   through qext_vector06.f32.

   The testvectors can be downloaded at
   https://media.xiph.org/opus/ietf/opushd_testvectors.tar.gz.

   Using the reference decoder, a testvector can be decoded as:

   % opus_demo -d 96000 2 -f32 qext_vector01.f32 qext_test01.f32

   Then the output can be compared to the reference with specific
   thresholds:

   % qext_compare -s -f32 -thresholds 0.05 0.1 0.1 \
                          qext_vector01dec.f32 qext_test01.f32

   which will output "Comparison PASSED" if the tested decoder is close
   enough to the target output.

   The second set of testvectors are meant to test corner cases.  The
   bitstream files are qext_vector01fuzz.bit through
   qext_vector06fuzz.bit, the corresponding target files
   qext_vector01fuzz.f32 through qext_vector06fuzz.f32.  Those are
   decoded in the same way as the first set of testvectors, but the
   comparison thresholds are looser:

   % qext_compare -s -f32 -thresholds 0.1 0.5 1.0  \
                          qext_vector01decfuzz.f32 qext_test01fuzz.f32

   For the tested decoder to be deemed compliant with this
   specification, all testvectors from both sets MUST pass.

4.2.  Encoder

   It is RECOMMENDED, but not mandatory for an encoder to comply with
   the following criteria.  Encoder testing involves encoding uncoded
   testvectors, decoding them with the reference decoder, and comparing
   to the original uncoded files.  The test is meant to evaluate
   encoding at bitrates around 1 Mb/s.  For example, encoding at 1024
   kb/s can be done with (may be different for the encoder being
   tested):

Valin                   Expires 14 December 2026                [Page 9]
Internet-Draft         Scalable Quality Extension              June 2026

   % opus_demo -e audio 96000 2 1024000 -f32 -cbr -qext \
                              qext_vector01.f32 qext_test01.bit

   The resulting bitstream then needs to be decoded with the reference
   implementation with:

   % opus_demo -d 96000 2 -f32 qext_test01.bit qext_test01enc.f32

   The decoded output (from the tested encoder) can then be compared
   against the reference original uncoded PCM:

   % qext_compare -s -f32 -skip <N> \
                      -thresholds 0.1 0.5 <RMS threshold> \
                      qext_vector01.f32 qext_test01enc.f32

   where <RMS threshold> for testvectors 1 through 6 are 5, 320, 20, 20,
   40, and 5, respectively.  The <N> value for the skip compensates for
   the encoder delay.  For the "audio" encoding mode, N=624 samples.
   For "restricted-lowdelay", N=240 samples.

   Because of the inherent limitations of objective quality evaluation
   metrics -- including the qext_compare tool -- it is also RECOMMENDED
   to perform a subjective evaluation of an encoder.

5.  IANA Considerations

   [Note: Until the IANA performs the actions described below,
   implementers should use 124 instead of 33 as the extension number.]

   This document assigns ID 33 to the "Opus Extension IDs" registry
   created in [opus-extension] to implement the proposed scalable
   quality extension.

6.  Security Considerations

   This document does not add security considerations beyond those
   already documented in [RFC6716].

7.  References

7.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

Valin                   Expires 14 December 2026               [Page 10]
Internet-Draft         Scalable Quality Extension              June 2026

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC6716]  Valin, JM., Vos, K., and T. Terriberry, "Definition of the
              Opus Audio Codec", RFC 6716, DOI 10.17487/RFC6716,
              September 2012, <https://www.rfc-editor.org/info/rfc6716>.

   [opus-extension]
              Terriberry, T.B. and J.-M. Valin, "Extension Formatting
              for the Opus Codec (draft-ietf-mlcodec-opus-extension)",
              October 2023.

Author's Address

   Jean-Marc Valin
   Google
   Canada
   Email: jeanmarcv@google.com

Valin                   Expires 14 December 2026               [Page 11]