NetVC IETF98, 28/03/2017, 13:00 General * Restart WGLC on requirements * Start WGLC on Testing * Milestone adjusted to May 2017. -------- Agenda -------- No comments -------- Requirements -------- draft-ietf-netvc-requirements Jose Alvarez https://www.ietf.org/proceedings/98/slides/slides-98-netvc-netvc-requirements-00.pdf * Minor and Editorial changes, methodology not changed * Added "efficient random access Discussion: * Mo: screensharing and bitstream compatibility requirements are NOT small changes, which triggered restart of WGLC * Tim Terriberry: This doesn't say much about when you have to be able to do this switching, is your estimation that something like periodic aligned keyframes is sufficient or do you think this will require S-frames. ** Jose Averez: I think this is sufficient. * Adam Roach: Just to note, Tim [Terriberry] will be sending some proposed text to this list for that. * Slide 4 screen sharing may be more than an "editorial change", WG members should take note and respond on the mailing list. * Mo Zanaty: There are some diffs for this text that is not changing which profiles and levels are defined but is rather changing the definition of profiles and levels. * Mo Zanaty: We are going to start a WG last call, but we need some more reviewers on this document. We don't have enough feedback on it to progress. Can we get a show of hands to see who will? * Notes that Tim Terriberry, Steve (unknown) and Dave (unknown) have volunteered * Tim, Steve to review -------- Test and Evaluation Criteria -------- draft-ietf-netvc-testing Thomas Daede https://www.ietf.org/proceedings/98/slides/slides-98-netvc-netvc-testing-00.pdf * two new test sets (obsoleting two existing sets) * includes HDR material at all resolutions (HDR10) * objective-2 slow is now a strict superset of objective-2 fast * 240p Content added * 720P and lower content has been extended to 120 frames in length (2x longer). Discussion * : I notice some clips are 4:3 and others are 16:9. Any question to test things that are weird aspect ratios? * Thomas Daede: All clips are actually 16:9. When the input video was not 16:9, it was clipped to 16:9. The weird aspect ratios, what they have are special edge effects. What happens when you have resolutions that are not multiple of 64, you have portions of the image that need to be handled internally. * Steve Botzko: As we look forward to WG last call, I wonder if it makes sense on this document to hold off on publishing until we get closer to WG last call since we might need to do differnet esting. * Thomas Daede: I do think that is a concern because this document has been incrementally updated. This document does have a provision for other testing like MOS scoring. * Steve Botzko: I don't see any harm in leaving it for a draft status for now. * Mo Zanaty: Any other opinions on if we progress the testing document? * Tim Terriberry: Basically i have the same opinion, I don't mind us leaving it up, I don't see any harm in it. * Randal Jessup: I think it's fine to keep it open, I agree with Tim. Other thing on the previous discussion on the siezs. I was wondering if it makes any sense to have it where the total widht or ttotal height is smaller than one 64x64 macro block. its a question to the codec people, I don't know what the answer is. * Tim Terriberry: I think that is a reasonable thing to use in regression testing, I don't think it makes sense to use for quality performance testing. * Randal Jessup: That seems reasonable, I agree. * Mo Zanaty: Some support for leaving the testing draft open and not progressing it. Is there anyone who thinks we should progress it? * Thomas Daede: I'm okay with leaving it open. * Jonathan Lennox: The only counter argument I can think of is to spend less time on this going forward if its closed. The other counter argument is that you might create a moving target for performance if we keep changing this. * Steve Botzko: There is a middle ground, we could send it to WG last call but not submit it for publication. * Mo Zanaty: If we intend to continually update it, Thomas if that is your intent, then I don't see why we would push it to publication. However if you think it is 99% done, with just editorial changes or other test clips, I think we should go ahead and do it now. As editor (to Thomas) what is the likely hood of substantive updates. * Thomas Daede: So far for each of the last meetings, there have been substatial updates. I would actually think the addition of test clips would be a substantial update. I think the most likely thing to happen is a new set of test cips to be added for some special case. Also the subjective tests potentially changing. If you don't think adding testing clips are a substantial change we could publish it. * Mo Zanaty: I don't think we want to publish stuff just to publish it. I heard arguments to leave it open and only weak arguments from Jonathan to move it forward. I suggest we leave it open and let Thomas continue to make updates. -------- Thor Update -------- draft-fuldseth-netvc-thor Steinar Midtskogen https://www.ietf.org/proceedings/98/slides/slides-98-netvc-thor-codec-update-00.pdf * added monochrome, 4:2:2, new constrained low-pass filter, misc fixes. * Overall status: AV1 outperforms Thor generally, but on some videoconferencing test sequences Thor still has some advantages. Slide 2 * Randal Jessup: I'll note that currently WebRTC code both in Chrome and Firefox is very heavily oriented to 4:2:2. * Mo Zanaty: Do you mean 4:2:2 or 4:2:0? * Randal Jessup: I mean 4:2:2. * Mo Zanaty: That is probably a reporting error. I think it is 4:2:0 * Randal Jessup: Either way, it is not 4:4:4. How significant is the coding gain by using 4:4:4 [as the internal format for Thor]? * Steinar Midtskogen: I don't know, we have not done the work to try that. It would require doing "the right thing" internally [it would require changing Thor's internal representation] * Tim Terriberry: One way to test this is to compare it to another encoder that does "do the right thing" with 4:2:0 and see how well it does. * Thomas Daede: I would be pretty comfortable with adding one or two test clips that are 4:2:2, just for regression testing. * Steinar Midtskogen: Does anyone remember what the requirements document says about 4:2:2? * Jose Alverez: Yes, we do have one or two sentences about 4:2:2 in the requirements draft but it is to support legacy devices. I agree with what Thomas said, we should have one or two streams to make sure its not broken, but we should be okay with having it sub-optimally supported. * Steinar Midtskogen: OK, I'll move on. Slide 10 * Mo Zanaty: One final point, since we have decided to keep the testing draft open. Other codecs have provisions for screen content, maybe it is worth considering breaking out screen content, since the results can vary wildly. * Steinar Midtskogen: There is a 1080p screen content test set on AWCY. * Consider screen content and the testing document and how these factors could produce differening test results. Slide 11 * Tim Terriberry: I still have my patches to integrate the Daala EC into Thor. I can dust them off if that will help. * Steinar Midtskogen: That was a year ago wasn't it? * Mo Zanaty: I'll remind the working group that the output of this process is a single codec (we are not going to put forward multiple implementations) so anything that can be done to merge efforts is encouraged. -------- Daala Update -------- draft-midtskogen-netvc-clpf Jean-Marc Valin * Constrained Directional Enhancement Filter (Valin) * General overview of current status * Future work: perceptual distortion metrics, entropy coding strength, and optimize interaction with other tools. * Mo Zanaty: Great to see a merge of some of the ideas, I think that was the whole spirit of the work we intended to have done here. Good to see the teams are working together and getting the efforts merged together. And the resutls are good. * Mo Zanaty: This is what surpirsed me (refering to slide 4). The results are actually better when you are trying to use faster settings, e.g., in real time. If this lets the codec take short cuts and make up for it here that helps. This is the difference between this group and others, the IETF cares about the real-time use case, where others care more about decoder perforamce. This is all about encoder performance.