Constrained Low Pass Filter
draft-midtskogen-netvc-clpf-01
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft whose latest revision state is "Expired".
|
|
|---|---|---|---|
| Authors | Steinar Midtskogen , Arild Fuldseth , Mo Zanaty | ||
| Last updated | 2016-03-18 | ||
| RFC stream | (None) | ||
| Formats | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-midtskogen-netvc-clpf-01
Network Working Group S. Midtskogen
Internet-Draft A. Fuldseth
Intended status: Standards Track M. Zanaty
Expires: September 19, 2016 Cisco
March 18, 2016
Constrained Low Pass Filter
draft-midtskogen-netvc-clpf-01
Abstract
This document describes a low complexity filtering technique which is
being used as a low pass loop filter in the Thor video codec.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 19, 2016.
Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Midtskogen, et al. Expires September 19, 2016 [Page 1]
Internet-Draft Constrained Low Pass Filter March 2016
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1. Requirements Language . . . . . . . . . . . . . . . . . . 2
2.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 2
3. Filtering Process . . . . . . . . . . . . . . . . . . . . . . 3
4. Complexity considerations . . . . . . . . . . . . . . . . . . 5
5. Performance . . . . . . . . . . . . . . . . . . . . . . . . . 5
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7
7. Security Considerations . . . . . . . . . . . . . . . . . . . 7
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 7
9. Normative References . . . . . . . . . . . . . . . . . . . . 7
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 8
1. Introduction
Modern video coding standards such as Thor [I-D.fuldseth-netvc-thor]
include in-loop filters which correct artifacts introduced in the
encoding process. Thor includes a deblocking filter which correct
artifacts introduced by the block based nature of the encoding
process, and a low pass filter correcting artifacts not corrected by
the deblocking filter, in particular artifacts introduced by
quantisation errors of transform coefficients and by the
interpolation filter. Since in-loop filters have to be applied in
both the encoder and decoder, it is highly desirable that these
filters have low computational complexity.
2. Definitions
2.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
2.2. Terminology
This document will refer to a pixel X and six of its neighbouring
pixel A, B, C, D, E, F ordered in the following pattern.
Midtskogen, et al. Expires September 19, 2016 [Page 2]
Internet-Draft Constrained Low Pass Filter March 2016
+---+---+---+---+---+
| | | A | | |
+---+---+---+---+---+
| B | C | X | D | E |
+---+---+---+---+---+
| | | F | | |
+---+---+---+---+---+
Figure 1: Filter pixel positions
In Thor the frames are divided into filter blocks (FB) of 128x128
pixels and each FB can be divided into a quadtree of coding blocks
(CB) which can range from 8x8 to 128x128. The filter described in
this draft can be switched on or off for the entire frame or
optionally on or off for each FB. CB's that have been coded using
the skip mode are not filtered, and if all CB's within a FB have been
been coded in skip mode, the FB will not be filtered and no signal
will be transmitted to indicate this.
3. Filtering Process
Given a pixel X and its neighbouring pixels described above we can
define a general non-linear filter as:
X' = X + clip(a*clip(A-X,-s,s) + b*clip(B-X,-s,s) + c*clip(C-X,-s,s) +
d*clip(D-X,-s,s) + e*clip(E-X,-s,s) + f*clip(F-X,-s,s),-g,g)
Figure 2: Equation 1
If a neighbour pixel is outside the image frame, it is given the same
value as the closes pixel within the frame. To avoid dependencies
prohibiting parallel processing, all neighbour pixels must be the
unfiltered pixels of the frame being filtered.
Experiments in Thor have shown that a good compromise between
complexity and performance is a=f=0.25, b=e=0.0625, c=d=0.1875 and
the filter strength s being 1, 2 or 4 signalled at frame level.
These values eliminate the need for the outer clipping to +/-g. The
rounding is to the nearest integer.
This gives us the equation:
Midtskogen, et al. Expires September 19, 2016 [Page 3]
Internet-Draft Constrained Low Pass Filter March 2016
X' = X + (4*clip(A-X,-s,s) + clip(B-X,-s,s) + 3*clip(C-X,-s,s) +
3*clip(D-X,-s,s) + clip(E-X,-s,s) + 4*clip(F-X,-s,s)) / 16
Figure 3: Equation 2
It can be noted that a=c=d=f=0.25, b=e=0 and s=1 give a slighly
simpler filter which is very similar to the one described in the
first version of this draft.
The filter leaves the encoder seven different choices for a frame:
1. The frame is not filtered.
2. The frame is filtered with s=1 and all non-skip CB's are
filtered.
3. The frame is filtered with s=2 and all non-skip CB's are
filtered.
4. The frame is filtered with s=4 and all non-skip CB's are
filtered.
5. The frame is filtered with s=1 and one bit per FB is sent to
indicate whether all non-skip CB's in the FB must be filtered.
6. The frame is filtered with s=2 and one bit per FB is sent to
indicate whether all non-skip CB's in the FB must be filtered.
7. The frame is filtered with s=4 and one bit per FB is sent to
indicate whether all non-skip CB's in the FB must be filtered.
The decisions at both frame level and FB level may be based on rate-
distortion optimisation (RDO), but an encoder running in a low-
complexity mode, or possibly a low-delay mode, may instead assume
that a fixed mode will be beneficial. In general, using s=2 and RDO
only at the FB level gives good results. Applying the filter to all
non-skip CB with no RDO at either frame level or FB level gives a
poorer result, and will not unfrequently lower the PSNR of the frame,
in particular if the frame already had near lossless quality.
However, because of the low complexity of the filter, fully RDO based
decisions are not costly. The distortion of the six configurations
of the filter can easily be computed in a single pass.
The filter is applied after the deblocking filter.
Midtskogen, et al. Expires September 19, 2016 [Page 4]
Internet-Draft Constrained Low Pass Filter March 2016
4. Complexity considerations
The filter has been designed to offer the best compromise between low
complexity and performance. A single pixel can be filtered with
simple operations as illustrated by this C function:
int clpf_sample(int X, int A, int B, int C, int D, int E, int F, int s)
{
int delta =
4*clip(A - X, -s, s) + clip(B - X, -s, s) + 3*clip(C - X, -s, s) +
3*clip(D - X, -s, s) + clip(E - X, -s, s) + 4*clip(F - X, -s, s);
return (8 + delta - (delta < 0)) >> 4;
}
Figure 4: C code
Also, these operations are easily vectorised in architectures
supporting SIMD instructions, such as x86/SSE4 and ARM/NEON. The
pixel difference is 9 bit, but it can be computed using adding an 8
bit offset and the use of 8 bit saturated signed subtraction. This
means that 16 pixels per core can be filtered in parallel on these
architectures. Clipping at frame borders can be implemented using
shuffle instructions.
A C implementation using x86/SSE4 intrinsics required 6.8
instructions per pixel to filter a single 8x8 block. The
corresponding number for ARM/NEON (armv7) was 4.9. The compiler was
gcc 4.8.4 in both cases.
Since the filter only needs to look up pixels in the line directly
above and below the pixel to be filtered, the line buffer requirement
in hardware implementations is very low.
5. Performance
The table below show filters effect on the bandwidth for a selection
of 10 second video sequences encoded in Thor with uni-prediction
only. The numbers have been computed using the Bjontegaard Delta
Rate (BDR). BDR-low and BDR-high indicate the effect at low and bigh
bitrates respectively. The effect of the filter was tested in two
encoder configurations: high complexity in which the encoder strongly
favours compression efficiency over CPU usage, and medium complexity
which is more suited for real-time applications. The bandwidth
reduction is somewhat less in the high complexity configuration.
Midtskogen, et al. Expires September 19, 2016 [Page 5]
Internet-Draft Constrained Low Pass Filter March 2016
+----------------+--------------------+--------------------+
| | MEDIUM COMPLEXITY | HIGH COMPLEXITY |
+----------------+------+------+------+--------------------+
| | | BDR- | BDR- | | BDR- | BDR- |
|Sequence | BDR | low | high | BDR | low | high |
+----------------+------+------+------+------+------+------+
|Kimono | -2.6%| -2.3%| -3.1%| -1.8%| -1.9%| -1.7%|
|BasketballDrive | -3.1%| -2.5%| -4.0%| -2.0%| -1.7%| -2.5%|
|BQTerrace | -7.0%| -4.9%| -8.4%| -5.1%| -3.6%| -6.0%|
|FourPeople | -5.5%| -3.9%| -7.9%| -3.7%| -2.6%| -5.3%|
|Johnny | -5.4%| -3.9%| -8.0%| -3.9%| -3.3%| -5.0%|
|ChangeSeats | -6.3%| -3.6%|-10.5%| -4.3%| -2.8%| -6.4%|
|HeadAndShoulder | -7.9%| -2.8%|-16.6%| -5.3%| -2.5%| -9.4%|
|TelePresence | -5.9%| -3.3%|-10.2%| -4.0%| -2.2%| -6.6%|
+----------------+------+------+------+--------------------+
|Average | -5.5%| -3.4%| -8.6%| -3.8%| -2.6%| -5.4%|
+----------------+------+------+------+--------------------+
Figure 5: Compression Performance without Biprediction
Whilst the filter objectively performs better at relatively high
bitrates, the subjective effect seems better at relatively low
bitrates, and overall the subjective effect seems better than what
the objective numbers suggest.
If bi-prediction is allowed, there is generally less bandwidth
reduction as the table below shows.
Midtskogen, et al. Expires September 19, 2016 [Page 6]
Internet-Draft Constrained Low Pass Filter March 2016
+----------------+--------------------+--------------------+
| | MEDIUM COMPLEXITY | HIGH COMPLEXITY |
+----------------+------+------+------+--------------------+
| | | BDR- | BDR- | | BDR- | BDR- |
|Sequence | BDR | low | high | BDR | low | high |
+----------------+------+------+------+------+------+------+
|Kimono | -2.1%| -2.0%| -2.4%| -1.3%| -1.4%| -1.3%|
|BasketballDrive | -2.4%| -2.6%| -2.2%| -1.4%| -1.7%| -0.9%|
|BQTerrace | -3.7%| -3.2%| -3.9%| -2.4%| -2.5%| -2.0%|
|FourPeople | -3.9%| -2.9%| -5.1%| -2.5%| -2.2%| -2.8%|
|Johnny | -3.4%| -3.2%| -4.0%| -2.2%| -1.7%| -2.7%|
|ChangeSeats | -4.2%| -3.2%| -5.7%| -2.6%| -2.2%| -2.9%|
|HeadAndShoulder | -3.9%| -3.0%| -5.4%| -2.4%| -2.1%| -2.7%|
|TelePresence | -2.6%| -2.0%| -3.6%| -1.5%| -1.1%| -2.1%|
+----------------+------+------+------+------+------+------+
|Average | -3.3%| -2.8%| -4.0%| -2.0%| -1.9%| -2.2%|
+----------------+------+------+------+------+------+------+
Figure 6: Compression Performance with Biprediction
6. IANA Considerations
This document has no IANA considerations yet. TBD
7. Security Considerations
This document has no security considerations yet. TBD
8. Acknowledgements
The authors would like to thank Gisle Bjontegaard for reviewing this
document and design, and providing constructive feedback and
direction.
9. Normative References
[I-D.fuldseth-netvc-thor]
Fuldseth, A., Bjontegaard, G., Midtskogen, S., Davies, T.,
and M. Zanaty, "Thor Video Codec", draft-fuldseth-netvc-
thor-01 (work in progress), October 2015.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>.
Midtskogen, et al. Expires September 19, 2016 [Page 7]
Internet-Draft Constrained Low Pass Filter March 2016
Authors' Addresses
Steinar Midtskogen
Cisco
Lysaker
Norway
Email: stemidts@cisco.com
Arild Fuldseth
Cisco
Lysaker
Norway
Email: arilfuld@cisco.com
Mo Zanaty
Cisco
RTP,NC
USA
Email: mzanaty@cisco.com
Midtskogen, et al. Expires September 19, 2016 [Page 8]