Integration of Speech Codec Enhancement Algorithms into the Opus Codec
draft-ietf-mlcodec-opus-speech-coding-enhancement-05
| Document | Type | Active Internet-Draft (mlcodec WG) | |
|---|---|---|---|
| Authors | Jan Buethe , Jean-Marc Valin | ||
| Last updated | 2026-10-25 | ||
| Replaces | draft-buethe-opus-speech-coding-enhancement | ||
| RFC stream | Internet Engineering Task Force (IETF) | ||
| Intended RFC status | (None) | ||
| Formats | |||
| Additional resources | Mailing list discussion | ||
| Stream | WG state | WG Document | |
| Document shepherd | (None) | ||
| IESG | IESG state | I-D Exists | |
| Consensus boilerplate | Unknown | ||
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-ietf-mlcodec-opus-speech-coding-enhancement-05
Machine Learning for Audio Coding J. Buethe, Ed.
Internet-Draft Meta Platforms Inc.
Updates: 6716, 7587 (if approved) J.-M. Valin
Intended status: Standards Track Google
Expires: 26 April 2027 23 October 2026
Integration of Speech Codec Enhancement Algorithms into the Opus Codec
draft-ietf-mlcodec-opus-speech-coding-enhancement-05
Abstract
This document proposes a set of requirements for integrating a speech
codec enhancement algorithm into the Opus codec [RFC6716]. It also
updates [RFC7587] to define two optional media type parameters for
signaling decoder-side speech enhancement capabilities.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 26 April 2027.
Copyright Notice
Copyright (c) 2026 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Buethe & Valin Expires 26 April 2027 [Page 1]
Internet-Draft Opus Speech Coding Enhancement October 2026
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3
2. An Illustrative Example . . . . . . . . . . . . . . . . . . . 3
3. Definition of a SILK Enhancement Algorithm . . . . . . . . . 5
4. Qualification of a SILK Enhancement Algorithm . . . . . . . . 6
4.1. General Requirements . . . . . . . . . . . . . . . . . . 6
4.1.1. Subjective Evaluation . . . . . . . . . . . . . . . . 6
4.1.2. Delay and Phase Considerations . . . . . . . . . . . 6
4.1.3. Encoder Requirements . . . . . . . . . . . . . . . . 7
4.1.4. Training Data . . . . . . . . . . . . . . . . . . . . 7
4.2. Requirements Specific to Non-Extending SILK Enhancement
Algorithms for Wideband Speech . . . . . . . . . . . . . 7
4.2.1. Lowband Test (Non-Extending) . . . . . . . . . . . . 7
4.3. Requirements Specific to Extending SILK Enhancement
Algorithms for Wideband Speech . . . . . . . . . . . . . 10
4.3.1. Lowband Test (Extending) . . . . . . . . . . . . . . 10
4.3.2. Highband Test . . . . . . . . . . . . . . . . . . . . 11
5. SDP Considerations . . . . . . . . . . . . . . . . . . . . . 12
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13
7. Security Considerations . . . . . . . . . . . . . . . . . . . 13
8. References . . . . . . . . . . . . . . . . . . . . . . . . . 14
8.1. Normative References . . . . . . . . . . . . . . . . . . 14
8.2. Informative References . . . . . . . . . . . . . . . . . 15
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15
1. Introduction
Since the specification of the original Opus codec [RFC6716], new
data-driven speech codec enhancement algorithms have emerged that
outperform classical enhancement algorithms by a large margin. Using
such enhancement algorithms to improve the quality of the Opus speech
codec SILK requires an update of [RFC6716] since SILK is an embedded
coding mode and changing the output of the SILK decoder will lead to
a violation of the Opus conformance criteria. The purpose of this
document is hence to update [RFC6716] to enable the use of a speech
codec enhancement algorithm. Specifically, this document defines the
notion of a SILK enhancement algorithm and sets forth a list of
requirements, some mandatory, some optional, that aim to ensure
(1) consistent performance of the enhancement algorithm itself,
(2) preservation of decoder performance (e.g. seamless mode
switching), and
(3) preservation of basic interoperability when tuning the Opus
encoder for use with an enhanced decoder.
Buethe & Valin Expires 26 April 2027 [Page 2]
Internet-Draft Opus Speech Coding Enhancement October 2026
While the first two objectives target the Opus decoder alone, the
third objective introduces new restrictions on the Opus encoder.
However, these are not expected to interfere with any existing
implementation of an Opus encoder since they target potential
interoperability issues arising from new incentives connected to the
possibility to enhance the Opus decoder.
The approach of specifying requirements instead of specifying the
enhancement algorithm itself has the advantage of allowing the Opus
decoder to benefit from future improvements in a field that currently
sees rapid development. Still, a description of the linear-adaptive
coding enhancer (LACE) and its integration into the Opus decoder is
included as an illustrative example for a SILK enhancement algorithm.
This document also updates the optional parameters and SDP mapping of
[RFC7587] to allow a receiver to signal its decoder-side speech
enhancement capabilities, as described in Section 5 and Section 6.
1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
2. An Illustrative Example
We use the linear-adaptive coding enhancer (LACE) [lace-paper] as an
illustrative example to highlight the specific challenges of
integrating a speech codec enhancement algorithm into the Opus
decoder. LACE is trained to enhance the output signal of the SILK
decoder, the speech coding mode of Opus, and Figure 1 depicts a high-
level overview of the Opus decoder with LACE added as an enhancement
algorithm.
The first requirement for a speech coding enhancement algorithm
concerns the performance of the algorithm itself. In this example it
relates to the question of how the SILK decoder output compares to
the LACE output. In [lace-paper] this has been evaluated on clean
speech samples using a P.808 listening test [p.808] as well as the
objective method PESQ, which showed consistent improvement for all
tested bitrates. For a general enhancement algorithm it will be
necessary to specify testing material and performance criteria to
prevent unintended quality degradation of the Opus codec.
Buethe & Valin Expires 26 April 2027 [Page 3]
Internet-Draft Opus Speech Coding Enhancement October 2026
The second requirement concerns performance of the Opus decoder as a
whole. Depending on the bitstream the decoder may have to perform
mode switching, e.g. between SILK and CELT, or it may combine the
SILK and CELT outputs when the codec operates in hybrid mode.
Changes to the SILK output signal by an enhancement algorithm, such
as added delay, phase shifts, or level alterations can therefore
negatively impact the performance of the Opus decoder even if the
first requirement is met. LACE solves this problem by adding no
delay and by being approximately phase and level preserving.
However, since many enhancement algorithms are non-causal and non-
phase-preserving, these requirements may be too strict for a general
enhancement algorithm.
The third requirement concerns interoperability. The Opus
specification provides significant freedom for tuning the encoder and
the presence of an enhancement algorithm in the decoder may change
the optimal encoding choices significantly. In the present example
encoding e.g. wideband content at 6 kb/s still leads to fair-to-good
quality when using the LACE-enhanced decoder while the quality of a
legacy decoder is significantly worse. To make full use of these new
enhancement algorithms, such encoder tunings should be allowed but
basic interoperability with legacy decoders or other enhanced
decoders needs to be ensured.
Buethe & Valin Expires 26 April 2027 [Page 4]
Internet-Draft Opus Speech Coding Enhancement October 2026
┌──────────────────────────────┐
│ Bitstream │
└─────┬──────────────────┬─────┘
│ │
▼ ▼
┌───────────┐ ┌───────────┐
│ CELT │ │ SILK │
│ decoder │ │ decoder │
└─────┬─────┘ └─────┬─────┘
│ │
│ ▼
│ ┌───────────┐
│ │ LACE │
│ └─────┬─────┘
│ │
│ ▼
│ ┌───────────┐
│ │ Resampler │
│ └─────┬─────┘
│ │
▼ ▼
┌──────────────────────────────┐
│ Mode Handling │
└──────────────┬───────────────┘
│
▼
decoded signal
Figure 1: A simplified Opus decoder diagram including LACE as
enhancement module
LACE has meanwhile been superseded by the non-linear adaptive coding
enhancer (NoLACE) [nolace-paper] which shares all basic properties of
LACE outlined above but provides higher quality. This stresses the
advantage of specifying requirements for an enhancement algorithm
over specifying the algorithm itself.
3. Definition of a SILK Enhancement Algorithm
A SILK enhancement algorithm denotes any algorithm that replaces the
native SILK decoded output with an enhanced output signal, an example
of which is depicted in Figure 1. Here, the native SILK decoded
output refers to the signal that an unmodified SILK decoder would
produce. A SILK enhancement algorithm is not restricted to operating
on the native SILK decoded output alone; it may equally draw on
internal quantities of the SILK decoder (such as gains, LSFs, pitch
or excitation), modify such internal values, or replace parts of the
SILK decoder altogether. The requirements in this document therefore
Buethe & Valin Expires 26 April 2027 [Page 5]
Internet-Draft Opus Speech Coding Enhancement October 2026
constrain only the relation between the native SILK decoded output
and the output of the enhancement algorithm. If the encoded content
is wideband and the decoder sampling rate allows for a higher
bandwidth than wideband, a SILK enhancement algorithm may also
produce an output signal of higher bandwidth than the native SILK
decoded output, replacing the resampler in Figure 1, or it may modify
the combination of a SILK-decoded wideband signal with a CELT decoded
highband signal in hybrid mode. Such bandwidth extension is only
permitted for wideband speech; narrowband and mediumband content MUST
NOT be extended. In any case, a SILK enhancement algorithm MUST NOT
modify the output of pure CELT frames. A SILK enhancement algorithm
whose output has a higher bandwidth than the native SILK decoded
output will be referred to as extending, whereas a SILK enhancement
algorithm whose output preserves the bandwidth of the native SILK
decoded output will be referred to as non-extending. Furthermore, an
Opus decoder including a SILK enhancement algorithm will be referred
to as enhanced decoder. Note however, that simply resampling the
signal to a higher sampling rate is neither considered enhancing nor
extending.
4. Qualification of a SILK Enhancement Algorithm
4.1. General Requirements
4.1.1. Subjective Evaluation
Objective metrics for quality evaluation have often proved unreliable
especially for evaluating completely new algorithms for processing
speech or audio signals. Therefore, any SILK enhancement algorithm
SHOULD undergo subjective evaluation before integration into the Opus
decoder. For genuinely new algorithms, it is RECOMMENDED to perform
either an absolute category rating (ACR) or degradation category
rating (DCR) listening test according to [p.800] or [p.808], where
the test conditions SHOULD cover a relevant range of bitrates. For
modifications of previously tested algorithms, e.g. changing the size
of a LACE model or adding small tunings for quality improvement or
complexity reduction, at least an informal subjective evaluation
SHOULD be carried out. Any enhancement algorithm SHOULD
significantly improve quality for at least one encoder operating
point while showing no significant degradation for other operating
points.
4.1.2. Delay and Phase Considerations
SILK is approximately phase preserving and to avoid additional delay
and maintain usability for applications relying on phase information,
any SILK enhancement algorithm SHOULD also be approximately phase
preserving.
Buethe & Valin Expires 26 April 2027 [Page 6]
Internet-Draft Opus Speech Coding Enhancement October 2026
4.1.3. Encoder Requirements
The Opus specification [RFC6716] provides much freedom for encoding
an audio signal and the presence of a powerful enhancement algorithm
can provide an incentive to use that freedom to produce bitstreams
that, when decoded with a legacy Opus decoder, do not result in a
reproduction of the input signal anymore. To prevent this, the
following requirement is added for an Opus encoder that is designed
to be used with an enhanced Opus decoder: if an Opus encoder produces
a bitstream that can be decoded into a human-recognizable
reproduction of the encoded signal with an enhanced Opus decoder,
then that bitstream MUST also result in a human-recognizable
reproduction of the encoded signal when decoded with a legacy Opus
decoder.
4.1.4. Training Data
To keep the objective tests meaningful, an enhancement algorithm MUST
NOT be trained on any of the clips used in the tests of this
document, and SHOULD NOT be trained on the datasets from which those
clips are drawn (the Common Voice dataset for the wideband speech
clips and additionally the VCTK corpus for the fullband speech
clips).
4.2. Requirements Specific to Non-Extending SILK Enhancement Algorithms
for Wideband Speech
4.2.1. Lowband Test (Non-Extending)
Every non-extending SILK enhancement algorithm for SILK decoded
wideband speech signals MUST pass all objective tests put forth in
this section. This collection of tests is designed to uncover major
failure points of the tested algorithm that could be due to improper
design or training data, or due to improper integration into the Opus
decoder. It is not designed to (and cannot) assess the quality of a
particular enhancement algorithm.
The normative definition of the degradation and distortion metrics
used in this document is the C comparison tool src/osce_compare.c
distributed in the pinned Opus source archive [OSCE-Compare]. The
rest of the archive provides an informative reference implementation.
The SHA-256 hash of src/osce_compare.c is:
883c2c0728b1aae266b9123276a1a680882f3f429751e0eaf03494449ace2dff
Buethe & Valin Expires 26 April 2027 [Page 7]
Internet-Draft Opus Speech Coding Enhancement October 2026
The same source archive [OSCE-Compare] includes an informative Python
script, scripts/osce/run_osce_tests.py, for running the objective
qualification tests. The script invokes the normative C comparison
tool and automates decoding, score aggregation, and reporting. The
metric definitions and pass criteria specified in this document
remain normative.
The tests are based on comparing a degradation score for audio
samples decoded from a list of bitstreams contained in
https://media.xiph.org/opus/ietf/osce_testvectors_v1.zip (FIXME: find
final location) to a reference degradation score computed from audio
decoded with a reference decoder. Any conforming Opus decoder
[RFC6716] is admissible as a reference decoder. This latitude is
deliberate and necessary: decoding a bitstream may involve behavior
that is not fully specified by [RFC6716], such as packet loss
concealment, so different conforming decoders may produce slightly
different reference scores. Each test corresponds to an encoder
operating point and the test names follow the scheme
osce_test_BITRATE_BITRATEMODE_FRAMESIZEms_BANDWIDTH_cCOMPLEXITY_MODE
where
(1) BITRATE is either a number specifying the encoder bitrate in
bits per second or the string "SWITCHING" indicating the bitrate
has been switched during encoding,
(2) BITRATEMODE is either vbr or cbr indicating variable bitrate or
constant bitrate encoding,
(3) FRAMESIZE is either 10 or 20,
(4) BANDWIDTH specifies the maximal bandwidth and is always WB for
this test (note however that the actual bandwidth can be lower),
(5) COMPLEXITY is a number from 0 to 10 and specifies the encoder
complexity,
(6) MODE refers to the coding mode and is either "native" or
"celtswitching". In "native" mode, the encoder decision whether
to use SILK or CELT is based on signal classification whereas in
"celtswitching" mode the encoder has been forced to switch
between SILK and CELT at a fixed rate.
The testvectors are further divided into groups, where each group
contains either speech samples from the same language or dialect, or
music content. Each group GROUP is tested separately and the test is
passed if it is passed for every group. The bitstreams in TESTNAME/
Buethe & Valin Expires 26 April 2027 [Page 8]
Internet-Draft Opus Speech Coding Enhancement October 2026
GROUP follow the naming pattern CLIPNAME_TESTNAME which associates
each bitstream uniquely with a reference signal reference_clips/
CLIPNAME.s16. For every CLIPNAME in GROUP let REFMOC(CLIPNAME)
denote the reference degradation score, obtained by decoding the
bitstream with a reference decoder and scoring the result against
reference_clips/CLIPNAME.s16 in the same way as the test signal
below. Furthermore, let CLIPNAME_test.s16 denote the signal decoded
with the enhanced decoder under test at a sampling frequency of 16
kHz after delay compensation. The degradation for the test signal
CLIPNAME_test.s16 is calculated using the osce_compare tool
[OSCE-Compare] with the reference signal path as first argument and
the test signal path as second argument. The resulting degradation
score will be referred to as TESTMOC(CLIPNAME).
All degradation and distortion scores in this document MUST be
computed on time-aligned signals: before scoring, the decoder delay
MUST be compensated so that the decoded signal is sample-aligned with
the reference signal reference_clips/CLIPNAME.s16. The osce_compare
tool performs this compensation by discarding a fixed number of
leading samples from the decoded (second-argument) signal, given
through its -delay option. The decoder delay depends on the Opus
implementation and its configuration, and, for an enhanced decoder,
on the SILK enhancement algorithm; it may therefore differ between
the reference decoder and the decoder under test as well as between
sampling rates. The applicable delay MUST be determined for each
decoder and sampling rate used, since residual misalignment degrades
the scores and can invalidate the test result.
From the reference degradation score REFMOC(CLIPNAME) and the test
degradation score TESTMOC(CLIPNAME) a difference score is calculated
according to
REFMOC(CLIPNAME) - TESTMOC(CLIPNAME)
D(CLIPNAME) = ------------------------------------
0.5
0.1 + REFMOC(CLIPNAME)
To pass the test for group GROUP, the following two criteria MUST be
met:
(1) D(CLIPNAME) is larger than A for every CLIPNAME in GROUP,
(2) The average of D(CLIPNAME) over GROUP is larger than B.
The thresholds are A = -0.5 and B = -0.052. A test is passed if it
is passed for all groups in that test.
Buethe & Valin Expires 26 April 2027 [Page 9]
Internet-Draft Opus Speech Coding Enhancement October 2026
4.3. Requirements Specific to Extending SILK Enhancement Algorithms for
Wideband Speech
An extending SILK enhancement algorithm extends the bandwidth of the
decoded speech beyond the 8 kHz upper limit of wideband. The
objective tests in this section therefore evaluate the decoder output
at a sampling frequency of 48 kHz. The blind bandwidth extension
network (BBWENet) [bbwenet-paper] is an example of an extending SILK
enhancement algorithm.
An extending algorithm is subject to two objective tests:
* a lowband test, which ensures that the quality of the 0-8 kHz
lowband is not degraded relative to an unenhanced decoder; and
* a highband test, which checks that the extended signal has a
closer per-band spectral distortion to the fullband reference than
the unenhanced wideband-decoded signal has to that reference.
The lowband test is the objective test of Section 4.2.1 applied to
the 48 kHz output; a non-extending algorithm is subject only to that
test. An extending algorithm MUST pass the lowband test.
Bandwidth extension is blind: the highband content above 8 kHz is not
present in the native SILK decoded output, and many plausible
highband extensions exist, some of which may violate the highband
criterion without being perceptually wrong. The highband test is
therefore relaxed in two ways: the criterion is only required to hold
for most of the test clips rather than every clip, and an extending
algorithm SHOULD, rather than MUST, pass the highband test.
Subjective evaluation as described in Section 4.1.1 remains the
primary means of assessing the highband of an extending algorithm.
4.3.1. Lowband Test (Extending)
The lowband test reuses the testvectors, naming scheme, grouping,
difference score D(CLIPNAME) and group pass criteria of
Section 4.2.1, with the following modifications:
(1) the reference decode (yielding REFMOC(CLIPNAME)) is produced by
decoding the bitstream with the reference decoder at a sampling
frequency of 48 kHz instead of at 16 kHz;
(2) the test signal (yielding TESTMOC(CLIPNAME)) is produced by
decoding the bitstream with the enhanced decoder under test,
i.e. the decoder applying the extending enhancement algorithm,
at a sampling frequency of 48 kHz;
Buethe & Valin Expires 26 April 2027 [Page 10]
Internet-Draft Opus Speech Coding Enhancement October 2026
(3) both degradation scores are computed on the 0-8 kHz lowband
only.
Both degradation scores are obtained with the osce_compare tool
[OSCE-Compare], invoked with the 16 kHz reference signal
reference_clips/CLIPNAME.s16 as first argument and the respective 48
kHz decoded signal as second argument, with the reference and test
sampling rates set accordingly (-fs_ref 16000 -fs_test 48000) and
with the applicable delay compensation (-delay). Given these rates,
osce_compare reduces the 48 kHz decoded signal to the 0-8 kHz lowband
before scoring it against the reference signal. Decoding both the
reference decode and the test signal at 48 kHz ensures that the 16
kHz to 48 kHz resampling effects on the lowband cancel between the
two in the difference score D(CLIPNAME).
In all other respects the lowband test is identical to Section 4.2.1.
An extending algorithm MUST pass the lowband test, i.e. it MUST NOT
degrade the wideband content.
4.3.2. Highband Test
The highband test verifies that the algorithm adds meaningful signal
energy above 8 kHz and does not degrade the highband. It is based on
a separate set of fullband (48 kHz) speech reference clips contained
in https://media.xiph.org/opus/ietf/osce_testvectors_v1.zip (FIXME:
find final location) under the highband subdirectory. Each reference
clip is encoded as wideband and the resulting bitstream is decoded
with the extending decoder under test at a sampling frequency of 48
kHz. Each test corresponds to an encoder operating point and the
test names follow the scheme
osce_hbtest_BITRATE_BITRATEMODE_FRAMESIZEms_BANDWIDTH_cCOMPLEXITY_MODE
with the fields defined as in Section 4.2.1. The bitstreams in
TESTNAME follow the naming pattern CLIPNAME_TESTNAME which associates
each bitstream uniquely with a fullband reference signal
highband/reference_clips/CLIPNAME.s16.
For a decoded test signal and its fullband reference, a per-band
spectral distortion DIST(reference, test) is computed for each of the
four highbands spanning 8.0-9.6, 9.6-12.0, 12.0-15.6 and 15.6-20.0
kHz, together with the distortion DIST(reference, anchor) of a
lowpass anchor. The lowpass anchor is the "no extension" signal and
is derived from the fullband reference by zeroing out all STFT bins
at and above 8 kHz. The exact computation is defined by the
osce_compare tool [OSCE-Compare], invoked with its -highband option.
A clip passes when, in every one of the four highbands BAND, the
extended signal is at least as close to the reference as the anchor:
Buethe & Valin Expires 26 April 2027 [Page 11]
Internet-Draft Opus Speech Coding Enhancement October 2026
DIST(reference, test)(BAND) <= DIST(reference, anchor)(BAND)
As with the lowband tests (see Section 4.2.1), the decoded test
signal MUST be delay-compensated so that it is sample-aligned with
the fullband reference highband/reference_clips/CLIPNAME.s16 before
the distortions are computed. The osce_compare tool applies this
compensation through its -delay option at the 48 kHz test sampling
rate; the applicable decoder delay depends on the Opus implementation
and the enhancement algorithm and may differ between decoders.
For each test (encoder operating point) the pass rate is the fraction
of clips that pass, and the test is passed if more than 90% of the
clips pass. The highband test is only meaningful when the lowband
quality is sufficiently high, which requires a sufficiently high
bitrate for the given frame size. Therefore, in keeping with the
blind nature of the problem described above, an extending algorithm
SHOULD pass all tests for bitrates of 9 kb/s and above when the frame
size is 20 ms, and it SHOULD pass all tests for bitrates of 12 kb/s
and above when the frame size is 10 ms. Lower bitrates are included
in the testvectors, but the test result on those operating points is
informative only.
5. SDP Considerations
This document adds the following two optional parameters to the
audio/opus media subtype defined in [RFC7587]. Both are informative
parameters by which a receiver signals its decoder's intent to
enhance (and, for extbw, to extend) the received signal, so that the
far-end encoder MAY select an operating point (for example, a lower
bitrate or a wideband-only mode) that the enhancement can compensate
for. The decoder is not bound by this signaling and may at runtime
reduce or disable enhancement, for example when CPU load is high.
The parameters do not change which bitstreams a decoder accepts, and
by the requirement of Section 4.1.3 any stream that relies on them
MUST remain intelligible when decoded by a legacy Opus decoder.
senh: an integer from 0 to 40 indicating the effectiveness of the
receiver's speech enhancement (see Section 3). It is calibrated
at 6 kb/s: if the enhanced decoder at 6 kb/s achieves the quality
of an unenhanced decoder at X kb/s, the value is round(10 * (X -
6) / 6). A value of 0 (the default when the parameter is absent)
indicates no speech enhancement; 10 indicates that enhancement at
6 kb/s reaches the quality of an unenhanced decoder at 12 kb/s,
and 40 (the maximum) corresponds to the quality of an unenhanced
decoder at 30 kb/s or better.
extbw: an integer giving the upper limit, in kHz, of the audio
Buethe & Valin Expires 26 April 2027 [Page 12]
Internet-Draft Opus Speech Coding Enhancement October 2026
bandwidth the receiver's decoder produces after blind bandwidth
extension. It applies only to content coded in the speech coding
mode; content coded in the general (music) coding mode is not
affected by this parameter and is reproduced at the bandwidth
selected by the encoder. Typical values follow the Opus bandwidth
tiers: 8 (wideband, i.e. no extension and the default when the
parameter is absent), 12 (super-wideband) and 20 (fullband). For
example, an extending algorithm such as BBWENet [bbwenet-paper]
that extends wideband speech to fullband would signal extbw=20.
The senh and extbw media type parameters map to the SDP a=fmtp
attribute as semicolon-separated parameter=value pairs, following the
mapping in [RFC7587]. Each parameter describes the capabilities of
the receiver that advertises it; the values in an offer and an answer
therefore apply independently to the two receiving directions.
An offerer or answerer MAY include either parameter; a value that is
absent, zero (senh) or 8 (extbw) is equivalent to the corresponding
baseline capability. An implementation that does not recognize these
parameters ignores them, as required by [RFC7587].
6. IANA Considerations
IANA is requested to update the audio/opus registration in the Media
Types registry, originally defined in [RFC7587], by adding senh and
extbw to the Optional parameters field, with the definitions in
Section 5. The Published specification field is to reference this
document in addition to [RFC7587]. All other registration fields
remain unchanged.
7. Security Considerations
A SILK enhancement algorithm runs after the SILK decoder and post-
processes (or replaces) its output. It introduces no new bitstream
elements and does not change which packets a decoder accepts or
rejects, so it does not expand the packet-parser attack surface and
the security considerations of [RFC6716] apply unchanged.
The following additional considerations apply to the enhancement
itself:
* Memory safety on attacker-influenced input: the enhancement
operates on the decoder output and on internal parameters (such as
gains, LSFs, pitch and excitation) that are ultimately derived
from untrusted bitstreams. An implementation MUST handle
degenerate or extreme values, including out-of-range or non-finite
intermediates, without out-of-bounds access, and MUST run in
bounded memory.
Buethe & Valin Expires 26 April 2027 [Page 13]
Internet-Draft Opus Speech Coding Enhancement October 2026
* Bounded, data-independent complexity: the example enhancement
models (LACE, NoLACE and BBWENet) are of fixed size, so their per-
frame cost is essentially constant and independent of the packet
content. Future enhancement models SHOULD likewise avoid any
data-dependent unbounded work, so that crafted streams cannot
force excessive CPU usage; because the enhancement is gated by
decoder complexity, a decoder can also bound or disable it.
* Model integrity: the model weights are part of the implementation
and are not signalled in the bitstream, so they are not attacker-
controlled through Opus packets. When the weights are loaded from
an external artifact, their integrity should be ensured (for
example through a verified hash). A corrupted model can only
degrade audio quality, not compromise memory safety, provided
inference is bounds-checked.
8. References
8.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC6716] Valin, JM., Vos, K., and T. Terriberry, "Definition of the
Opus Audio Codec", RFC 6716, DOI 10.17487/RFC6716,
September 2012, <https://www.rfc-editor.org/info/rfc6716>.
[RFC7587] Spittka, J., Vos, K., and JM. Valin, "RTP Payload Format
for the Opus Speech and Audio Codec", RFC 7587,
DOI 10.17487/RFC7587, June 2015,
<https://www.rfc-editor.org/info/rfc7587>.
[OSCE-Compare]
"Opus Source Snapshot (opus-1.6.1-85-ga8b2545e.tar.gz)",
23 October 2026,
<https://media.xiph.org/opus/ietf/osce_material/opus-
1.6.1-85-ga8b2545e.tar.gz>. Source snapshot of Opus at
commit a8b2545eec71e0f53ecead2d583e7b6baa42684d, FIXME:
move to final location. SHA-256:
b2e92cbe5d1a443b8b489ce0b98c9dd48f106bb70c80f07cc9f6da04bd1aaebd
Buethe & Valin Expires 26 April 2027 [Page 14]
Internet-Draft Opus Speech Coding Enhancement October 2026
[p.800] ITU-T, "P.800 : Methods for subjective determination of
transmission quality", August 1996,
<https://www.itu.int/rec/T-REC-P.800-199608-I>.
[p.808] ITU-T, "P.808 : Subjective evaluation of speech quality
with a crowdsourcing approach", June 2021,
<https://www.itu.int/rec/T-REC-P.808-202106-I/en>.
8.2. Informative References
[lace-paper]
Buethe, J., Valin, J.-M., and A. Mustafa, "LACE: A light-
weight, causal Model for enhancing coded Speech through
Adaptive Convolutions", 2023 IEEE Workshop on Applications
of Signal Processing to Audio and Acoustics (WASPAA), New
Paltz, NY, USA, pp. 1-5,
DOI 10.1109/WASPAA58266.2023.10248150, 2023,
<https://doi.org/10.1109/WASPAA58266.2023.10248150>.
[nolace-paper]
Buethe, J., Mustafa, A., Valin, J.-M., Helwani, K., and M.
Goodwin, "NoLACE: Improving Low-Complexity Speech Codec
Enhancement Through Adaptive Temporal Shaping", ICASSP
2024 - 2024 IEEE International Conference on Acoustics,
Speech and Signal Processing (ICASSP), Seoul, Republic of
Korea, pp. 476-480, DOI 10.1109/ICASSP48485.2024.10448332,
2024, <https://doi.org/10.1109/ICASSP48485.2024.10448332>.
[bbwenet-paper]
Buethe, J. and J.-M. Valin, "A lightweight and robust
method for blind wideband-to-fullband extension of
speech", 2025 IEEE Workshop on Applications of Signal
Processing to Audio and Acoustics (WASPAA), Tahoe City,
CA, USA, pp. 1-5, DOI 10.1109/WASPAA66052.2025.11230955,
2025, <https://doi.org/10.1109/WASPAA66052.2025.11230955>.
Authors' Addresses
Jan (editor)
Meta Platforms Inc.
United States of America
Email: jan.buethe@googlemail.com
Jean-Marc
Google
Canada
Email: jmvalin@jmvalin.ca
Buethe & Valin Expires 26 April 2027 [Page 15]