Minutes interim-2017-rmcat-01: Thu 17:00
minutes-interim-2017-rmcat-01-201704271700-00

RMCAT virtual interim meeting

27 April 2017

Minutes reported by Colin Perkins

Attending:
  Anna Brunstrom
  Colin Perkins
  Martin Stiemerling
  Xiaoqing Zhu
  Bernard Aboba
  Ingemar Johansson
  Jiantao Fu
  Jonathan Lennox
  Mirja Kuelewind
  Sergio Mena
  Stefan Holmer
  Varun Singh
  Zahed Sarker


# Introduction

  Anna Brunstrom reviewed the draft status and proposed milestone updates.
  Will extend WG last call for NADA until we have some reviews on the list. 


# Update on RMCAT Video Traffic Model

  Xiaoqing Zhu presented the updated version of their video test sequences
  generated using a modified version of the Mozilla browser running WebRTC
  with the H.264 codec at various rates. Results presented showed both the
  steady-state and transient behaviour during rate changes. The draft uses
  these traces to derive and parameterise a statistical model of the
  traffic, and will be updated based on an Laplacian distribution and the
  trace parameters. The traces have been uploaded to the syncodes repo on 
  github, and the draft will be updated to match. 

  Jonathan Lennox asked how VP8 traces would differ from the H.264 traces. 
  There were questions about the VP8 traces at IETF 97, but these have not
  yet been resolved, and the current results are for H.264 only. It seems
  clear that the models are suitable for both codecs, but will need to be 
  parameterised differently for different codecs and codec implementations.

  Zahed Sarker asked how much do we think this model has to match real
  codec behaviour. The goal is to model frames sizes and intervals, since
  these are the parameters that impact congestion control. 

  Not expecting any major updates to the model, and believe it nearly ready
  for working group last call. Reviews from the working group are solicited
  once the draft has been updated (likely in a couple of weeks).


# Update on ns3-rmcat open source module 

  Sergio Mena presented an ns3 module that can model RMCAT traffic sources
  (as in the previous presentation) in wired and wireless test cases. This
  is being made available as open source by Cisco shortly. They have been
  using it to model the behaviour of NADA, but the authors hope is that it
  can become a reference implementation with pluggable congestion control
  to evaluate different candidate congestion control algorithms.
  
  The code is being updated to better model changes in physical bandwidth,
  jitter, and to implement the design team feedback format, but this will
  not be in the initial release. 

  Zahed Sarker and Stefan Holmer thanked the authors for doing the work,
  and will review and send comments.


# Evaluation of NADA in ns3-rmcat 

  Xiaoqing Zhu presented evaluation results for NADA collected using the
  ns3 module described by Sergio. This is intended to be a reference
  implementation for the NADA algorithm. 

  Known issues with NADA, based on these evaluations:
   - may occasionally get stuck in loss-based mode;
   - when working with trace-based traffic couses, may under-utilise the
     available bandwidth; and
  - in presence of fully utilised bottleneck, new flows can take a long
    time (~60 seconds) to converge to equilibrium rate.
  The authors do not consider these major problems.

  Ingemar Johansson noted that he'd expect the problem with being stuck
  in loss based mode. SCReAM has similar issues in some cases. It's a 
  known problem with delay-based algorithms in general, and all these 
  algorithms have delay-based components. Some discussion of possible
  ways to resolve this, but performance-complexity trade-off is unclear.

  Anna Brunstrom noded that slow convergence is also a trade-off. Xiaoqing
  agreed: fast convergence is desirable, but fast convergence that impacts
  existing traffic possibly less so. 

  Jonathan Lennox asked for clarifications on the impact of bi-directional
  traffic, and there was some discussion. The test case is designed to show
  the sensitivity to loss of feedback messages.  Jonathan also asked about
  the impact of aggregation of feedback messages.  NADA has been shown to
  be stable up to ~100ms feedback interval, but further experimentation is
  needed. Sergio Mena believes it behaves well at feedback intervals up to
  ~1 second, but this is based on an old study that needs updating.


# Progressing the evaluation drafts

  Anna Brunstrom, as chair, asked about the status of the evaluation drafts.

  eval-test:
  - Discussed at IETF 96, should test cases 5.1, 5.2, and 5.3 be modified
    to separate tests for changes in available bandwidth by introducing
    cross traffic or by changing link bandwidth? Is it required to run
    both sets of test cases?
  - Zahed Sarker says both test cases are needed ("a very strong SHOULD")
    by the current draft, and is happy with the draft as written.
  - Varun Singh has only done the bandwidth change tests, and hasn't run
    the UDP traffic tests, but doesn't seem to have issues with running
    both.
  - Xiaoqing Zhu believes both test cases are needed.
  - General feeling seems to be that the current draft is okay, and both
    sets of tests need to be run.

  eval-criteria:
  - Zahed Sarker noted that there was some discussion about choice of TCP
    model in eval-test, but this has been moved to the eval-criteria draft.
  - Varun Singh has implemented the draft, but is seeing inconsistencies 
    with the TCP results. Investigation ongoing, to see if this is a
    problem with the draft or their implementation.
  - Zahed Sarker asked if ECN should be added to this? Anna noted that the
    consensus was to leave ECN to a separate draft, and Varun hasn't added
    ECN tests to the draft.
  - Colin Perkins noted that, due to the ongoing alternate backoff and L4S
    work, it makes sense to leave further ECN tests to future work.

  It seems that the choice of TCP model is the only open issue before these
  drafts can go to WG last call. The chairs asked Varun Singh to propose an
  update to address this, so the drafts can progress as a pair. 

  The wireless tests don't seem to have open issues, but perhaps need more
  implementation experience before they can progress. Reviews of the draft
  are also solicited - the authors haven't received much feedback.


# Status of feedback design team 

  Zahed Sarker presented an update from the feedback format design team
  and the proposed feedback format.

  Is this the right information to report? Yes.

  Is encoding this using RTCP XR and transport layer feedback appropriate?
  Yes. There will likely be some nits to resolved when reviewed by AVTCORE, 
  but there's nothing fundamental wrong.

  There has been some discussion on the benefits of optimising the format.
  Two proposals: use RLE for the loss reports to reduce size, and separate 
  out the ECN reports into separate packets (since ECN not widely used). 
  Stefan Holmer has suggested scenarios to use to evaluate the format and
  the optimisations, and has produced data on how Chrome sends feedback.
  Discussion is ongoing to understand the differences between the Chrome
  implementation and the format used in the cc-feedback draft, and hence 
  to understand further whether optimisations are needed.

  The issues of how to adapt the RTCP reporting interval to changes in the
  session bandwidth has been put aside for now, while the feedback format 
  is finalised. 

  Jonathan Lennox asked whether we expect to take the feedback format to
  AVTCORE for IETF 99. Zahed hopes this will be the case. 

  Varun Singh noted that he has packet loss statistics that can potentially
  be used to optimise the packet formats. The benefit will depend on header
  overheads, of course. Concrete suggestions for optimisations are needed,
  so the benefits can be evaluated.

                                   - + -