Minutes IETF98: netvc
minutes-98-netvc-00

NetVC
IETF98, 28/03/2017, 13:00

General
* Restart WGLC on requirements
* Start WGLC on Testing
* Milestone adjusted to May 2017.

-------- Agenda --------
No comments

-------- Requirements --------
draft-ietf-netvc-requirements
Jose Alvarez
https://www.ietf.org/proceedings/98/slides/slides-98-netvc-netvc-requirements-00.pdf

* Minor and Editorial changes, methodology not changed
* Added "efficient random access

Discussion:
* Mo: screensharing and bitstream compatibility requirements are NOT small
changes, which triggered restart of WGLC * Tim Terriberry: This doesn't say
much about when you have to be able to do this switching, is your estimation
that something like periodic aligned keyframes is sufficient or do you think
this will require S-frames. ** Jose Averez: I think this is sufficient. * Adam
Roach: Just to note, Tim [Terriberry] will be sending some proposed text to
this list for that. * Slide 4 screen sharing may be more than an "editorial
change", WG members should take note and respond on the mailing list. * Mo
Zanaty: There are some diffs for this text that is not changing which profiles
and levels are defined but is rather changing the definition of profiles and
levels. * Mo Zanaty: We are going to start a WG last call, but we need some
more reviewers on this document.  We don't have enough feedback on it to
progress.  Can we get a show of hands to see who will? * Notes that Tim
Terriberry, Steve (unknown) and Dave (unknown) have volunteered * Tim, Steve to
review

-------- Test and Evaluation Criteria --------
draft-ietf-netvc-testing
Thomas Daede
https://www.ietf.org/proceedings/98/slides/slides-98-netvc-netvc-testing-00.pdf

* two new test sets (obsoleting two existing sets)
* includes HDR material at all resolutions (HDR10)
* objective-2 slow is now a strict superset of objective-2 fast
* 240p Content added
* 720P and lower content has been extended to 120 frames in length (2x longer).

Discussion
* <Unknown>: I notice some clips are 4:3 and others are 16:9.  Any question to
test things that are weird aspect ratios? * Thomas Daede: All clips are
actually 16:9.  When the input video was not 16:9, it was clipped to 16:9.  The
weird aspect ratios, what they have are special edge effects.  What happens
when you have resolutions that are not multiple of 64, you have portions of the
image that need to be handled internally. * Steve Botzko: As we look forward to
WG last call, I wonder if it makes sense on this document to hold off on
publishing until we get closer to WG last call since we might need to do
differnet esting. * Thomas Daede: I do think that is a concern because this
document has been incrementally updated.  This document does have a provision
for other testing like MOS scoring. * Steve Botzko: I don't see any harm in
leaving it for a draft status for now. * Mo Zanaty: Any other opinions on if we
progress the testing document? * Tim Terriberry:  Basically i have the same
opinion, I don't mind us leaving it up, I don't see any harm in it. * Randal
Jessup: I think it's fine to keep it open, I agree with Tim.  Other thing on
the previous discussion on the siezs.  I was wondering if it makes any sense to
have it where the total widht or ttotal height is smaller than one 64x64 macro
block.  its a question to the codec people, I don't know what the answer is. *
Tim Terriberry: I think that is a reasonable thing to use in regression
testing, I don't think it makes sense to use for quality performance testing. *
Randal Jessup: That seems reasonable, I agree. * Mo Zanaty: Some support for
leaving the testing draft open and not progressing it.  Is there anyone who
thinks we should progress it? * Thomas Daede: I'm okay with leaving it open. *
Jonathan Lennox: The only counter argument I can think of is to spend less time
on this going forward if its closed.  The other counter argument is that you
might create a moving target for performance if we keep changing this. * Steve
Botzko: There is a middle ground, we could send it to WG last call but not
submit it for publication. * Mo Zanaty: If we intend to continually update it,
Thomas if that is your intent, then I don't see why we would push it to
publication.  However if you think it is 99% done, with just editorial changes
or other test clips, I think we should go ahead and do it now.  As editor (to
Thomas) what is the likely hood of substantive updates. * Thomas Daede: So far
for each of the last meetings, there have been substatial updates.  I would
actually think the addition of test clips would be a substantial update.  I
think the most likely thing to happen is a new set of test cips to be added for
some special case.  Also the subjective tests potentially changing.  If you
don't think adding testing clips are a substantial change we could publish it.
* Mo Zanaty: I don't think we want to publish stuff just to publish it.  I
heard arguments to leave it open and only weak arguments from Jonathan to move
it forward.  I suggest we leave it open and let Thomas continue to make updates.

-------- Thor Update --------
draft-fuldseth-netvc-thor
Steinar Midtskogen
https://www.ietf.org/proceedings/98/slides/slides-98-netvc-thor-codec-update-00.pdf

* added monochrome, 4:2:2, new constrained low-pass filter, misc fixes.
* Overall status:  AV1 outperforms Thor generally, but on some
videoconferencing test sequences Thor still has some advantages.

Slide 2
* Randal Jessup: I'll note that currently WebRTC code both in Chrome and
Firefox is very heavily oriented to 4:2:2. * Mo Zanaty: Do you mean 4:2:2 or
4:2:0? * Randal Jessup: I mean 4:2:2. * Mo Zanaty: That is probably a reporting
error.  I think it is 4:2:0 * Randal Jessup: Either way, it is not 4:4:4.  How
significant is the coding gain by using 4:4:4 [as the internal format for
Thor]? * Steinar Midtskogen: I don't know, we have not done the work to try
that.  It would require doing "the right thing" internally [it would require
changing Thor's internal representation] * Tim Terriberry: One way to test this
is to compare it to another encoder that does "do the right thing" with 4:2:0
and see how well it does. * Thomas Daede: I would be pretty comfortable with
adding one or two test clips that are 4:2:2, just for regression testing. *
Steinar Midtskogen: Does anyone remember what the requirements document says
about 4:2:2? * Jose Alverez: Yes, we do have one or two sentences about 4:2:2
in the requirements draft but it is to support legacy devices.  I agree with
what Thomas said, we should have one or two streams to make sure its not
broken, but we should be okay with having it sub-optimally supported. * Steinar
Midtskogen: OK, I'll move on.

Slide 10
* Mo Zanaty: One final point, since we have decided to keep the testing draft
open.  Other codecs have provisions for screen content, maybe it is worth
considering breaking out screen content, since the results can vary wildly. *
Steinar Midtskogen: There is a 1080p screen content test set on AWCY. *
Consider screen content and the testing document and how these factors could
produce differening test results.

Slide 11
* Tim Terriberry: I still have my patches to integrate the Daala EC into Thor. 
I can dust them off if that will help. * Steinar Midtskogen: That was a year
ago wasn't it? * Mo Zanaty: I'll remind the working group that the output of
this process is a single codec (we are not going to put forward multiple
implementations) so anything that can be done to merge efforts is encouraged.

-------- Daala Update --------
draft-midtskogen-netvc-clpf
Jean-Marc Valin

* Constrained Directional Enhancement Filter (Valin)
* General overview of current status
* Future work: perceptual distortion metrics, entropy coding strength, and
optimize interaction with other tools. * Mo Zanaty: Great to see a merge of
some of the ideas, I think that was the whole spirit of the work we intended to
have done here.  Good to see the teams are working together and getting the
efforts merged together.  And the resutls are good. * Mo Zanaty: This is what
surpirsed me (refering to slide 4).  The results are actually better when you
are trying to use faster settings, e.g., in real time.  If this lets the codec
take short cuts and make up for it here that helps.  This is the difference
between this group and others, the IETF cares about the real-time use case,
where others care more about decoder perforamce.  This is all about encoder
performance.