Thursday, March 26, Morning Session I 0900-1130 Room Name: Venetian Transcript (notes taken by Brian Trammell ) WG Status & Agenda Bashing: Chairs ================================== On draft-ietf-rmcat-eval-test ----------------------------- Zahed: status of eval, small changes to RTT fairness, but there are missing wifi test cases, please have a look. Varun: Traffic models (short-lived TCP) from appendix moved to the other draft Asking for additional reviews Randal (via Magnus): Will talk to Patrick McManus about models, on short-lived TCP On merging test cases into eval-test: Zahed: this could be done. Not sure about video traffic sources, still listing different models, that needs to proceed as individual. Mirja: At people using the wireless test cases? If you're the only one working on it, worries me a little. Varun: On cellular test cases, not all of us have simulators Mirja: Can these be simplified so they can be implemented, e.g. in NS3 Varun: Yes; on level and granularity in the draft, hard to implement Zahed: Can't promise to contribute on ns3 Karen: Disussion on list to resolve this point -> potential adoption. Mirja: If we need these, we can Zahed: For RMCAT, wireless test cases is a MUST Mike Ramallo: We normally ignore link layer at transport. Link layer characteristics are very important for delay CC. WACs and schedulers make modeling difficult. Use case is incredibly important. How to codify that in the charter? Contrast between shared and per-user queues. We need more than just mailing list discussion. Do we need to add link-layer issues directly to the charter? Mirja: Not excluded from charter, recharting doesn't make sense. Clear we can work on it. The group deciding is very small. Xiaoqing Zhu: Let's do something based on wifi first. Similar to cellular draft. Decide whether to merge when we have more experience Mirja: Did you implement any wireless test cases? Xiaoqing: For simulations, no. Mirja: Now I've heard two people who say they can't implement. Zahed: Why not? If you have comments as to why not, please say so. Mirja: Can we simplify the cellular model? Zahed: Of course. But we can't pick just one scheduler to model. Want to have this discussion though, what can we simplify for eval implementation. Mumble, covered badge: Maybe an option is to have real-world traces fed into a simulator Magnus (as Randall): Strongly agree need to test wifi. Eval criterion 4.5 says must handle heterogeneous environments Magnus (as ???): [missing, check jabber log] Mike Ramallo: Hard to drive characteristics off data traces, not a stationary process. Xiaoqing: as evaluators, we want something we can build on top of. don't want to spend lots of time implementing. In terms of networks, wifi/cellular are merging, but delay characteristics are still difficuly Zahed: I don't think we'll be able to provide traces. We could talk to 3GPP. Varun: Re-echo what [missing] said, we have to do both. What stephan said: perhaps using a trace is easier, could go down that path with cellular case. How to get them? Another problem. magnus as Randall: We don't need perfection magnus as Stephan: We can record traces ourselves Mirja: who thinks draft-sarker-rmcat-cellular-eval-test-cases is a good starting point? Hum now (hum quietly pro + randall) -> will take to the list. Zahed: we need another revision to determine whether we are going to merge the models Mirja: Can we merge trace-based into test cases now? If we can merge now, we don't have to procees Varun: I prefer the models stay in one place Colin: Video traffic modeling should perhaps be separate Mirja: We're doing this work for evaluation. It's easier to proceed less documents Brian: What just happened for the record? Mirja: We will have two documents at the end: test cases and eval criteria. We adopted cellular test cases to move it forward (Not clear about merging wifi at this point). Not adopting video traffic source now, but may merge parts into eval criteria. on splitting rmcat-app-interaction terminate existing, three new: CC-App interations -> W3C (Varun) CC-Codec interactions -> RMCAT (Mo) RMCAT framework Mike Ramallo: Procedural question: this is a WG document, will all three become WG drafts? Or should they be submitted as individual drafts (first)? Karen/Mirja: yes (will be individual drafts) On feedback extensions ---------------------- Mo: Other areas in RAI where we may need to switch media in conference scenarios, can we use feedback techniques here since we need them elsewhere anyway Varun: The idea of the holmer draft is good, not sure this is the right group Karen: Not the question. We're not doing etension here, question is does the arch need feedback extensions. document says no now. Colin: Characterization of cc-feedback is not how I see it. There are two kinds of RTP extensions: those that fit the frame and those that change it. CC-feedback shows that framework is enough. Draft said nothing about feedback type, or new types of message. Expect each CC alg may need new messages, this draft asks whether RTP needs to be extended. For the purpose of this document, I think the answer is no. I don't believe feedback messages are extensions. Zahed: Even if we recommend new feedback message, we need to say why we need them. We have to wait on CC to know that. Mirja: In this group? Zahed: I think so Magnus as Magnus: The CC alg should specify info it needs, sufficient for testing. Colin: Agree. Possibly we'll need to change framework. Probably not. But we need to look at the outcome of the CC exercise. Actual reqts for feedback messages in the alg drafts. Geert: From our 3gpp point of view I think it works with current messages. Zahed: I don't think 3gpp has an alg? Karen: cut discussion Shared Bottleneck Detection for Coupled Congestion Control for RTP Media: David Hayes ===================================================================================== Brian: (on clock drift slide) How much drift have you really seen in your experiments David: (basically) not very much, simple method is sufficient Mirja: We had a lot of feedback, who's read version 2? any version? seems enough Mirja: Hum for adoption (hum pro, no hum against) -> we'll confirm on list Update on Scream and implementation: Zahed Sarker ================================================= Varun: clarification: (changes in alg 2/2 slide) when you do fast start, do you have false positives? Zahed: We look at different variables, don't think there is a chance of false start. Haven't seen this. Colin: You have some new feedback messages (slide changes 1/2): look at 6679, a lot of this information is already included. Zahed: No timestamp Colin: But everything else. May want to try and use it and piggyback a timestamp, or align with it. Xiaoqing: (RTT fairness) Are these diverging? Zahed: No Xiaoqing: Are you planning on testbed experiments? Zahed: Working on now Mirja: To have an experimental RFC we only need to show simulation results that it won't break the Internet Karen (individual): Have you tested with reordering? Zahed: Yes Karen: There is no requirement for reordering required, but we need to think about whether we specify this in our documents. Especially WRT wireless. Zahed: We have to discuss, not sure we have to stress it Karen: Not a showstopper, we have to talk about it though Magnus as Randal: contact Mozilla for test integration in Firefox. Mirja: Shall we adopt? I think it's ready. Hum for adoption (Hum for + Randal, no hum against) -> confirm on list Update on NADA on Evaluation Results : Xiaoqing Zhu ================================================================================ Varun: WHen you modeled TCP in test case G, how was it modeled? Xiaoqing: FTP source on for a certain amount of time, not an amount of data Xiaoqing: in test case H, restarting the flow starts as a completely new flow Brian: May want to change VM hosts to ensure you don't have systematic timing error [missing exchange with Zahed] Mirja: Which TCP are you using in NADA vs TCP Xiaoqing: whatever is in Windows 7 Mirja: In the simulations? Xiaoqing: TCP [missing], in ns2 Mirja: In this scenario, NADA looks more aggressive, in contrast to the simulations Xiaoqing: Not side-by-side comparable Mirja: Maxrate is config parameter? Xiaoqing: Yes Mirja: Are you going to recommend a set of parameters? Xiaoqing: Application input Mirja: Are the NADA aggression parameters the same as in the simulation? Xiaoqing: Parameter is a function of video... how much weight do you want to add to loss? Mirja: Same set of parameters as [...] Xiaoqing: Not changed Mirja: In your draft you discuss AQM mechanisms. We don't have that on charter. If we adopt this, we would ask you to remove these from the draft. Xiaoqing: Performance doesn't depend on AQM. We include results to demonstrate that AQM helps. Karen: Is this AQM spec something we want in the evaluation criteria or test cases? Xiaoqing: We do have a place for UCs with AQM in the test cases draft Zahed: Yes, we discussed this at IETF 91, we found no need, but if it pops up as really important we can add it. Magnus as Randall: We had AQM in reqts: 11, be stable and maintain low delay when using AQM Matt: We can't set requirements on the network, but this can provide insight on how to make things better. No AQM, long flows, cross traffic -> hopeless. Mirja: We should integrate AQM in test cases. But before we adopt the doc we can move this somewhere. Question on AQM testing is whether we specify parameters. Mirja: call for adoption for next revision in which AQM is removed. Hum for, not against -> confirm on list. NADA AQM: Xiaoqing Liu ====================== Mirja: on two flows with PIE: Here, PIE has a target rate of 20ms, but you have 100ms? Xiaoqing: We're showing total OWD. Queueing delay is loose. Coupled congestion control with NADA: Michael Welzl =================================================== Michael: Would like not to have to track CC alg changes in each CCC implementation. Zahed: WOuldn't it be nice to not have to change CCC? Michael: That's impossible, have to at least look at things that changed, may have to make minor tweaks. I would like the call for adoption not to require the support of both scream and nada .... Michael: example NADA updates two rates based on feedback, if I only change one rate, not adjusting both rates, have to wait for next feedback michael: next rev will have guidelines for how to make CC work with CCC. Mirja: How complicated is it for someone to update CCC to handle a new CC? Would it be valuable to document how to apply it to a new CC alg? Michael: We're going to do that. Mirja: Do you need a separate doc for CCC for NADA, CCC for SCReAM? Michael: No, it's one paragraph. Stein: Could we make a generic interface between CC and CCC? Michael: Offline. Mirja: Are you planning to apply to scream? How far have you looked into it? Michael: We initially found the code complicated and hard to use. Zahed: What's complicated? Michael: We'll have a deeper look at it. Mirja: I understand you can't prove it works for every alg, but I would feel better if it works for two rmcat algs. Other possibility is to specify CCC for a specific scheme then add text to recommend how to use a different CC alg, instead of describing a completely general mechanism. Reorganize the docmuent so it specifies on a single CC alg. Michael: OK, I see. David: NADA and SCReAM were designed without CCC. New algs will have to specify CCC support from the start. Varun: Don't set the bar too high for generic mechanisms. If CCC has a requirement for CC, makes this easier. Michael: Planned for next rev. Varun: Other algs eg scream have multiflow support, these may conflict with CCC. Zahed: I don't want something to constrain how I design my alg.