Thanks to Kevin Fall and Mat Ford for taking notes!
led by co-chairs Brian Trammell (BT) and Mirja Kuehlewind (MK)
MK: This question was originally raised by people who wanted to be able to present results in the meeting without having those results appear in the proceedings.
Lars Eggert (LE): Understood. Then there's two groups - those willing to submit to disclosure rules, and everyone else. Acknowledge that the problem might exist in the future and burn the bridge when we come to it. Need to distinguish between identifying conditions in which a researcher can see data (NDA, etc.), and the work of this group which can take output from that research and think about what it means for the evolution of the stack.
Aaron Falk (AF): Not all RG meetings have to be open, have to have open list and maybe annual open meeting. But we should not have closed meetings unless we have clear articulation that it will help with data sharing.
Leslie Daigle: This is a hard question to answer until we're clear about scope and purpose.
Andrew Sullivan: If results are also closed then what is persuasive value to rest of Internet? Need to be able to persuade people about need to make changes to the stack.
BT: It makes no sense to close without benefit. And aggregated or otherwise non-attributed information would have to be an open outcome.
Spencer Dawkins: in discussions of security information sharing.. non-attribution is quite useful in some of these cases
Natasha Rooney: We've had this conversation with mobile operators - here there are similar problems. We're trying to formalise a plan to collect data in GSMA and then distribute to this group in some anonymised fashion that keeps everyone happy; basically a list of operators who are doing it, so really nothing right now. I could use help to make progress.
LE: It would help if speakers could say whether they can make raw data available to this group, and whether they would have problems sharing findings. This will help assessment of whether or not this should be formally chartered as an IRTF RG.
Jana Iyengar (JI): Don't formalise requirements too much in advance.
Summary: Sense of the room: we can be less than completely open, but only if necessary, and only if the need arises.
MK: Question is whether people are interested to continue turning up for this or if there is only transitory interest.
LE: Discussed this somewhat during RAIM workshop. Facilities already exist for data aggregation - don't need to reinvent those. I can't buy petabytes of storage for IRTF, but webpages are cheap.
AF: Focus on measurement infrastructure questions is a huge distraction from what important questions are and how to answer them.
BT: Can continue working informally for a while.
Roland van Rijswijk-Deij: What would IETF community need to have research that flows into the standardisation process - what does IETF community expect from academic research community - input from IETF would be useful.
MK: Need data to do work in IETF - less clear about whether or not work here could lead to any standardisation activities.
Summary: Future activities should focus on creating a forum for exchange on results potentially relevant to IETF work across areas; measurement infrastructure is less interesting.
AF: Hiccups paper and Honda paper motivated me - small quantities of locations to do active measurements from clients - interested in understanding if middleboxes are interfering with transport protocols, need diversity out at the edge- is this an area of focus for you?
KC: we're not aiming for 1000s of measurement points - are aiming for 100s. We do say 'no' to hosts when we already have probes in a network. www.caida.org/projects/ark has map of current probe distribution. Needs for people who have requirements for visibility into specific regions to come and tell us what they need, see www.caida.org/projects/ark, email ark-info@caida.org
KC notes that AIMS workshop, in February 2016 after NANOG in San Diego, may be of interest to the room.
IS notes that Packet reordering data does not include mobile - only desktop.
IS notes in response to question from Karen Nielsen that TBLD is compared to FACK.
Dave Plonka: How would we use the web to the server could collect statistics?
IS: I'm not sure - good question. We're still developing our own metrics gathering. Also adding more logging - hope to share that data. There is also data we'd love to have that we don't have, e.g. UDP rate limiting on mobile.
AMM notes that measurements were taken on port 80 in response to question from Jana Iyengar
Dave Oran: how is 6% a 'low' failure rate? that's hundreds of millions of flows/day. Is a 6% failure rate usable?
AF: from a service providers PoV... no.
AF: how much did it cost you? How long did it take you to collect?
AMM: 2k users in 3 days
Stephane Bortzmeyer: can we run DNS on TLS? Don't see port 53 on your list. Potential issue for middleboxes. So we're moving to another one. Can you advise?
JI: thank you for the data.. Seems trying to do something unusual over port 80 is difficult. how about bias in your data set?
AMM: you can select which countries you want to use for data collection
Brandon Williams (Akamai): curious about using Tor for this purpose? Did that affect your results
PA: no, didn't seem to
JI: you showed middlebox tampering as a thing, but do you have numbers on this?
PA: all these measurements are to see middlebox effects..lots tampering with headers
Richard (?): did you look at other upgrade methods? just h2c or do they hate upgrades in general?
PA: seems they hate upgrades in general.
Yuchung Cheng: go back to delay affecting TCP congestion control.. makes deploying non-loss based cc difficult; not clear how much backlog to use
JF: you need several packets to determine what you will get
JI: thanks; this is all just packet sizes.. are there quantization issues wrt to the scheduling alg being used? I don't see history, but these graphs don't show that.
JF: they show it in that you have replicated patterns.. capacity is allocated to you. when there's more load you get migrated to a higher bw state; requires additional rt
JI: right... these are time signatures, so I get this as a scheduling issue but not a middlebox problem
JF: there is lower layer information
JI: note that any bursty MAC will do this..
Bob Briscoe (BB): I don't think is based on increased queues-- seems to be that you get more capacity for more offered load... PA: check out the LTE graph
TP notes we're not going to share a lot of raw data here, but want to share and discuss results in aggregate
JI: is this something only for ECN, or something for other items (reordering) [ref to slide 14]
Stuart Cheshire (SC): the RTX jumps the queue so this messes the PAWS computation; partly caused by hashing-based load balancing
Matthew Ford: how to fix this?
SC: "it is Apple's policy to not comment"
Jason Weil: explain your slide 19 more please
TP: we're seeing different performance, possibly due to load balancing
Joao Tavera Araluo (): we find you can, in anycast, get ECN negotiated, then you get an RST. did you see this? (due to hashing on ToS)
SC: how is this RST problem happening? (if no client is asking)..
JTA: AWS instances try to do ECN negotiation
SC: which client devices?
JTA: Linux boxes?
SC: where the users are setting this explicitly (?) So who is explicitly asking?
JTA: I've seen several
Andrew Sullivan: so you have workarounds; how are you measuring and knowing when to not do this anymore? When is it 'safe'?
TP: great question; SYN fallback maybe; we are reporting fallback statistics
BB: you need to give some warnings to these companies that they should be given a date
JI: is maybe something doing poor ToS marking reponsible for some of this?
Dave notes he'll be around the IETF all week if people are interested in collaborating.
15 min
Allison Mankin (AM): do you have a plan to start with the existing observatories?
BT: we have an idea about doing that but not a plan
MK: we're focusing on mbox impairment
AM: I could imagine many players are interested in this from one angle or another... such an important problem
BT: We hope that a forum for discussion of information sharing issues could grow around such an observatory