Minutes for RMCAT at IETF-89
minutes-89-rmcat-2

Notes for RMCAT minutes, as heard by Brian Trammell (thanks!)

Magnus Westerlund (RTCWeb update):
==================================

Trying to finish core specification before end 2014. Mandating circuit
breakers. Trying to provide tools in RTP to deploy proprietary solutions until
RMCAT complete. Two interesting thigs coming up: interaction with the data
channel, prioritizations. Need input from people working on CC.

Evaluating Congestion Control for Interactive Real-time Media
Varun Singh -- draft-ietf-rmcat-eval-criteria-00
=============================================================

Lars: Does the working group feel it is okay to split out appendix B? (Yes.)

Reviewers: Mo, Zahed, Markku

Test Cases for Evaluating RMCAT Proposals
Varun Singh -- draft-sarker-rmcat-eval-test-00
==============================================

(Media source description)
Varun: Anything to add to media source description?
Mirja: Did you integrate all input from last time? (Yes.)
Lars: Have you run this by RTCWeb? (Just the video encoder.)

Rong Pan: How quickly does it produce a new rate -- sometimes the encoder can't
conform to the rate wanted to Varun: This is modeled...

Gorry: Haven't read the draft but surely there's a difference between
increasing and decreasing the rate? Varun: Just specify in media source
characteristics.

(Test cases)
Varun: Any more test cases you'd like? Fill out using guidelines in previous
slides

???: Are you planning to consider data channel test cases?
Varun: Considering data channel cross traffic. Do we need to do this beyond TCP?
Lars: One thing to talk about TCP/SCTP is whether channel is sender limited
Varun: Goes into desc. of competing traffic.

Zahed: Ask the working group: is this what happens in the Internet? Please
provide input. Lars: Send input soon. List is pretty exhaustive.

???: SCTP congestion control has CWV so less aggressive than TCP.

Randell (Jabber): I can help with DataChannel details
[Lars: more action items for Randall?]

Michael Ramalho: Hybrid of 4+5, long lived data pull from a source, full out
and bursty alternating.

Rong Pan: Any plan to test larger scale, as in a conference situation?
Varun: Scope is unicast. We chose multiple of 10 media sources.
Rong: When there's enough multiplexing, some characteristics change...
Lars: Each scenario we put on the list we commit to evaluating later.

Brant Hirshnen: Consideration of different QoS on the flows?
Zahed: Not really in the basic test cases.
Mirja: We're chartered for e2e congestion control, no network-based things
Rong: We run the *** evaluations, it's easy for us to run 100 flows.
Lars: These are the basic. You can do extra credit.

Randall (Jabber) via Lars: Agree with Michael R - NetFlix does that too even
without explicit DASH protocol (up to 10 seconds on, then off).  They'll be
moving to DASH (in JS) using MediaSource Extensions

Matt Mathis: Shouldn't be too much TCP vs RMCAT. If you're competing with TCP
you can't meet first order requirements for RMCAT. We have to avoid this
problem through other mechanisms Mirja: We have a coexist with TCP requirement
Matt: The only way to do that is push on the queue and increase delay beyond
TCP. Lars: We should maybe evaluate in AQM context Matt: Only helps with a
short RTT TCP flow. Bernhard Aboba: Requirements document talks about these two
contradictory things Mirja: Network mechanisms out of scope in this WG. Would
solve problem but not here.

Gorry: The aim of building this framework is to use it on the in-ter-net. We
will compete with TCP. We will have FIFO. It's okay to look at this. Everything
Matt said is true. I like this list.  People can do other stuff.

Varun: AQM is there in the topology description

Dave Taht: AQM and WebRTC are required for each other to work.

randall jesup: you can compete to a degree with short TCP flows.  We do include
AQM in the requirements.  And yes, we can't wave a wand and make TCP play nice,
but we can evaluate how *badly* it gets hurt, and how well it recovers.  We
need to look at it.  We will lose to low-RTT TCP flows over enough time.  We
know this.

Matt: need to play nicely with on-off flows. Big difference if TCP has always
data to sent

Arat: Just because we have a test case doesn't mean that a problem is a fatal
error. randall jesup: we need to know how badly each alg gets hurt

Colin: RTCweb usage draft doesn't motivate support for ECN.
Varun: Remove from draft?
Colin: If it's useful, put in RTC draft. Otherwise remove
Gorry: ECN important for the future. But we don't need it here.

Zahed: I know we're focusing on RTCweb
Lars: We added ECN before we knew what RTC was
Colin: Need manageable scope

Mo: Balancing act with reqs and evaluation: candidates shouldn't fall apart,
want to make sure alg survives new deployments, whether or not we have test
cases

Lars: Phase one, experimental candidates, phase two, one standards track. Don't
have to do everything in phase one. Does this give implementors a good idea as
to how to evaluate? Mirja: everyone happy with this? (hands go up)

Matt: Social engineering path: users discover apps don't work together. Charter
has requirement for instrumentation. If it doesn't work when competing with
TCP, tell the user that.

Lars: we seem to believe this is a pretty good list of tests. I would propose
that people implement their own algorithm as well as competing algorithms. Do
this and get more time to talk.

Bernhard: WiFi test case? Does it do rate adaptation? Typical models in
simulators do a bad job of this.

Michael Welzl: Something on the wishlist: something we also had in the old days
of TCP, don't just run each other's schemes, also provide an online unified
simulation/evaluation environment Mirja: please send a link when you've done
this

Evaluation Test Cases for Interactive Real-Time Media over Cellular Networks
Zahed Sarker - draft-sarker-rmcat-cellular-eval-test-cases-00
============================================================================

How many have read this? 5

Varun (on slide 9, description structure): clarification on radio environment,
how do I model this? Zahed: You need an LTE simulator Varun: Is there an open
source one? Zahed: No Varun: How do I do this test? Zahed: I guess the tools
exist, many papers. Varun: Do we use traces? Can we decompose this into
something simple? Paul ???: Will need to define the entire test processing
chain. Zahed: I agree. Lars: I suspect everyone has different evaluation
capabilities. This is why I want you to run other people's algorithms, this
would get us better comparability.

Varun: We need clarifications on mobility modeling
Zahed: in 3gpp spec, I can link to them.

Vijay: Re simulator, ns3 has LENA.

Varun (on slide 10): Best effort bearer, is that ack mode or unack mode. Does
RTC say something about this? Zahed: I want unack, there are deployments with
ack. I expect unack.

Mo: In general, are there aspects of LTE that would make a solution candidate
work better if the app knew it was on LTE. Is there benefit to knowing? Lars:
Want to push back on link-type-specific rmcat algorithms. We want an algorithm
that works reasonable well everywhere. LTE networks have lots of knobs, not
clear results portable between operators.

Paul ???: Cell throughput, useful for operator but not useful for user. Raises
issue for one-size-fits-all solution: this metric is only useful in wireless
environments.

Michael Ramalho: haven't dealt with link layer, but we talk about channel
delay. Mobile is incredibly important going forward. If we can infer mobile is
bottleneck, we'd be doing the industry a disservice by not adapting to that.
Lars: Solution candidates optimized for one case are not good.

Glor.. Ti...: this is repetition of section 5.1 or previous draft, why is this
separate document Zahed: in basic test, we're looking at internet behavior,
here we're looking at a single link type, these are two different discussions.

Randall Jesup (jabber): I struggle with the complexity too; I agree we need to
know how well it does on "generic" LTE/etc cases

Magnus: Very important to find things that work in most cases. LTE has knobs.
Wifi has variants. We're going to have to pick cases and see how well they work
Lars: agree, would be unhappy with three algs, one good for lte, one for wifi,
one for fixed. premature optimization is the root of all evil.

??? (Qualcomm): Definitely support scenario, maybe unify with other document,
and have basic/advanced test cases, third stage could be *** quality? Lars:
let's not talk about how many docs.

Mo: bad idea for any link specific candidates, but useful to abstract out
differences in these different secnarios, let the app know that. propose for
wireless networks to distill their environment down to a few factors.

Jorg: for which you need a lot of statistics... trying to simplify things, 3g
eval of CC had scenarios such as elevators, buildings, outdoors... we shouldn't
go into models, but similar abstractions as building blocks for looking at
mobility between regimes... multiple documents should use one set of traffic
models. Zahed: traffic models are same as other doc.

Mirja: This document is useful. But not adopting it soon. People can already
use this scenarios for testing.

RMCAT Application Interaction [slides]
Mo Zanaty -- draft-zanaty-rmcat-app-interaction-00
==================================================

Zahed: I don't get what you're saying with the conceptual model, the config
should go everywhere, not just to codec. Mo: Implementation model, typically
congestion control is in the RTP stack. Zahed: Is the codec controlling the RTP
stack? The config only goes to the codec? Mo: First thing to get agreement on
is model Z: what is goal of the document Lars: it's a milestone, mo wrote a doc
(Brian clarifies) Mo: config should branch out to all components, not just
through the RTP stack

Colin: the way I thought this would work, I would put CC between RTP and codec
so it can mediate Mo: it's on the bottom because it does transmission shaping
Colin: clearly implementation is fuzzier. wise to split RTP and RTCP
conceptually. algs will treat feedback differently. Mo: separating feedback
from media useful

Bernard: there's feedback you get and your response, those are distinct things.
Varun: having pacer/shaper separate is useful.

Brad Hirschmann: draw this in separate control and media planes with CC in
control plane Bernard: codec also does control.

Varun: (on config-codec interactions) this is a config for the whole
application, not just for codec...

Colin: you want to negotiate RTCP XRs.
Mo: good point

Michael: what we'll need for coupled congestion control is the rate the codec
is able to use, will send to list.

Mirja-from-the-floor: this seems like a list of all things you could exchange,
what I'd like to see is a smaller list of things you really have to exchange,
there is a risk people might misuse too much information

Colin: we want a way to tell the codec "i can tolerate a burst of this size",
for iframes etc.

Zahed: we discussed responsiveness in the usecase, this is maybe more
important, how frequently can you change or how much time you need to adjust.
Mo: captured in elasticity, maybe have as separate.

Roni: not just from codec to cc, also from app.
Mo: codec is fuzzy boundary, might not be a codec at all, e.g. in switching
mixer

Lars: isn't most of the CC-UDP interaction here just something the CC should do?
Mo: if the underlying stack provides this features cc can use them.

Mirja: feedback on the list from app designers and cc developers
Lars: good first start, going broad, feedback should include which features to
flesh out more

Jorg: is there a list of three or four interaction that are most important?
Mo: already in prio order on codec-cc slide.

Progress on Practical Shared Bottleneck Detection for Coupled Congestion Control
David Hayes
================================================================================

Michael Ramalho: your delay plot looks like a 2-sided dist, packet nets have a
1-sided dist.

Mirja: how did you generate the different traffic loads?
David: using Tmix traffic generators

Matt: can't imagine these dual bottlenecks being common
David: these should be harder than we'd normally find

Zahed: how fast can this react?
David: Slower at startup. If the flows are already up and running, the change
in the stats come out more quickly. Order of seconds, not milliseconds. Much
faster with smaller queues. Zahed: Cross sources? David: None are greedy
sources. Trace-based Tmix sources. Lots of short flows, big flows, all tcp
controlled. Packets sent at varying rates.

Mo: Any faster convergance if bottleneck is closer?
David: No. Stats faster with shorter queues. Can be faster with higher error
rate. MPTCP might be willing to make this tradeoff

Rudiger: Isn't the number of datapoints very important? You're measuring
correlation. David: Take summary stats, if different by threshold, divide into
group, not quite correlation, grouping problem.

Varun: For specific video source, are the packet sizes equal?
David: Not going out of my way to model video.

Update on Google Congestion Control Algorithm for Real-Time Communication
Stefan Holmer -- draft-alvestrand-rmcat-congestion-02
=========================================================================

Zahed (on gcc vs tcp): what kind of TCP source is this? is this one object
fetch? How does TCP adapt? Mirja: i.e. these are greedy sources? Stefan: yes

Michael Ramalho: bottleneck queue is 350ms?
Stefan: I'm not sure. These are RTTs.
Michael: The dynamics of TCP are going up and down consistent with 400MS RTT.
Alg is workig as designed. Did you try on other TCPs, with different RTTs?
Mirja: More eval next time. Michael: With a lot of TCPs, you'll fill the link
and tail drop. Do you lose bandwidth again Stefan: DV will be small

Lars: Folks that have alg proposals, talk to Varun and Zahed, pick scenarios
with quick result generation, more presentation time if you simulate other
people's algorithm Mirja: No candidates without external evaluation
(half-joking) Lars: Will summarize to list.