Minutes for PLUS at IETF-96
minutes-96-plus-1

Birds of a Feather session:
PLUS: Path Layer UDP Substrate
THURSDAY, JULY 21ST, 2016
10:00-12:30
Chairs: Aaron Falk, Natasha Rooney

Scribe: tale, krose

Notes are by the main presenter unless stated otherwise.

------------------------------------
Slides: https://www.ietf.org/proceedings/96/slides/slides-96-quic-0.pdf
Presenter: Aaron Falk
------------------------------------

Overview of agenda

Goals
Working group-forming BOF
Is the problem well-defined and are people willing to do the work?

Non-goals/off-topic
Implementation details
Protocol design
SPUD
Focus is overall very meta, not on the specifics of any implementation
proposals or prior work.

[ some background:
    Re: wrt quic, plus, and taps?
    https://www.ietf.org/mail-archive/web/ietf/current/msg98993.html
]

------------------------------------
Slides: https://www.ietf.org/proceedings/96/slides/slides-96-plus-4.pdf
Title: Impediments to Transport Innovation
Presenter: Joe Hildebrand
------------------------------------

TCP Firewall behaviour is based on implicit signals, not explicit design
features of TCP to allow firewalls. UDP doesn't have as much going on, because
it can't have as much going on w.r.t. protocol design.  "Both signaling and
control impaired with respect to TCP."

Need to look at why network operators want to block and/or rate-limit UDP. 
Basically, attack traffic that firewalls can't effectively mitigate. Try to
reach feature parity with TCP (for traffic management).

Questions to Joe:
    dkg @ mic.  Question was re: on-path vs off-path, Joe misspoke.
    jana said something I didn't catch

------------------------------------
Slides: https://www.ietf.org/proceedings/96/slides/slides-96-plus-3.pdf
Title: Ernestine might like PLUS too.
Presenter: Natasha Rooney
------------------------------------

Mobile networks underprovisioned, so operators try to optimize use.
"Network and endpoints not working together on flow control."
Slide 4 ("ACCORD 1") contains a good, high-level description of why mobile
networks and TCP seem to have interoperability issues. Slide 5: Mobile networks
often(?) use "bearers" to attempt to encapsulate different traffic flows and
assign varying QoS constraints. Aaron Falk in jabber: "today, I think, a
customer gets a single bearer for INternet traffic.  all traffic is carried by
it." Slide 6: How operators have classically dealt with these issues (for TCP)

Looking into what mobile networks need.
Desired abilities: give a flow best balance or resources, manage resources
sensibly, future-proof, classification based on as little info as possible.
Want to be able to trust the overall trust model without traffic
prioritisation, DPI or traffic inspection.

Questions to Natasha:
Jana: what does it mean to have a trust model not based on prioritization vs.
one that is? Natasha: If you are loss-sensitive traffic and say you're
latency-sensitive, your users are going to have a worse experience. ...?

Ekr: Assumes there's an aligned interest between the one sending the data and
the carrier. Not always the case. Assumes that I want the carrier to know what
I'm doing.  Won't be able to avoid traffic prioritisaton; carriers will always
want to do it.

SzilveszterNadas (on jabber): clarify last bullet point

Lorenzo: Found out that transparent proxies were actually what was making
things slow, and now that theey're being upgraded to support v6, we can expect
v6 to get slow. (Laughter.) Natasha: Transparent proxies are used to save
money, not to improve performance. There may be other justifications, but she
doesn't have that information. Lorenzo: Trust us (the standards community) to
do the right thing to help all internet users.

Jana: "Ability to trust in the trust model". Users also need to have trust in
the trust model, as well. Natasha: Oh yes of course.   Story time example of
bad trust model:  web sites that would fire up 3d object rendering even though
they didn't need it, just to get the GPU going to render the flat site faster.
Jana: Thank you for bringing the operators' view point here.

Joe on Jabber: "talking about this in terms of "aligned incentives" is
potentially useful."

------------------------------------
Slides: https://www.ietf.org/proceedings/96/slides/slides-96-plus-2.pdf
Title: Architecture Viewpoint "Alice and Bob look at PLUS
Presenter: Ted Hardie
------------------------------------

There is an implicit layer between transport and internet, he's calling the
"path layer". It is used to maintain state in the network, which is now being
impaired by encryption of some critical transport layer signaling information.
We normally ignore this as an extant layer.  Signals that are in Internet or
Transport layers (particularly Transport) are now being impaired through, f.ex,
encryption. Options: Use Internet Layer to send the signals. Bolt the signals
onto each transport. Do nothing. Restore implicit signal by encrypting less.
      OR
Make the implicit layer explicit.

Questions to Ted:
Christian licks the mic. Then roars at it. And literally dropped the
mic.(Without a shoulder brush.) Christian: Layers tend to translate into blocks
of headers that are applied in order to the data. Each header layer adds
overhead. Maybe these are functions rather than protocol layers, because
information only needs to be used at certain points in time, not all the time.

Alain Durand: ... unclear comment on ptcp, not clarifying question.

Tom Hebert: Not a path layer, but multiple paths. Made point about future
internet evolution including multi-homed clients whose packets (e.g., SYN and
FIN) don't pass through the same points in the network. (Also no clarifying
question; chairs ask for focus on clarifying questions.) Ted: Not limited to a
single path.

Joanna: Do you think that having an explicit path layer will help with the
ossification problem? Won't encrypted signaling lead to a different kind of
ossification?

Stewart Bryant: Where does routing fit into this?
Ted: Internet layer alone.
Aaron:  lets get to proposals and we'll talk about that later.

Eric Klein: Should information be changeable?
Ted: no

------------------------------------
Slides: https://www.ietf.org/proceedings/96/slides/slides-96-plus-1.pdf
Title: Technical considerations
Presenter: Brian Trammell
------------------------------------

We're talking about explicit co-operation, versus the existing, widespread
implicit co-operation.  (Some objections to the term "co-operation" since it
can be quite hostile.) Part of desire for explicit co-operation is to encrypt
everything on the path that intermediate devices don't need to see, to prevent
future implicit co-operation without send authorisation.

Three (and a half) mechanisms to make path layer explicit:
* Sender to path signalling
* Path to receiver signalling by including work space for the path to use for
its purposes, to be relayed back by receiver * Direct path to sender signalling
for info about dropped packets

This slide deck is going to be difficult to understand without hearing Brian's
talk.  Agreed.

Paul Hoffman:  clarifying question about whether we're convering all 3.5
mechanisms Brian: yes, in part that's why it took 2 years to get here from spud

Back to preso, "Is this a user tracking and network neutrality violating
machine?"  Ans: no, unless the client requests it to be.

Ekr: First bullet point ("Will it be possible for a middlebox to use PLUS to
insert user identifiers in the server-bound stream of a client-server protocol?
No") only true if the MAC verification is enforced at the server. Otherwise, if
it's just advisory, then it's trivial to violate.

Goran: Is this about parental control?
Brian (+ room): No!
Goran: Can you list what it is meant to be used for then?

Christian: Do you have an estimate of the overhead per-packet?
Brian: In the SPUD prototype, we were looking at 12 bytes of overhead per
packet in order to do the recognition that this was a PLUS packet plus the
subflow ID. Part of the problem is that the ID needs to be a random number from
a relatively large space. Aaron: we're jumping down the implementation rathole
and I'm going to have to cut that off. Christian: We need to make sure we're
not exceeding some threshold of overhead. Any solution needs to account for
that overhead.

Lorenzo: "IETF: Designing 'how' before 'should' for last 25 years."  Where is
the problem statement?  Can we clarify that? Brian:  The problem statement is
to solve some subset of these known use cases where encryption increases but
operators still need traffic management signals.

Eliot Lear:  Where are drafts?
Aaron Falk: Drafts are in the agenda. 
https://www.ietf.org/proceedings/96/agenda/agenda-96-plus

Lee Howard on Jabber:
    Relevant drafts:

  Requirements for the design of a Substrate Protocol for User Datagrams
  https://tools.ietf.org/html/draft-trammell-spud-req-04

  Use Cases for a Substrate Protocol for User Datagrams
  https://tools.ietf.org/html/draft-kuehlewind-spud-use-cases-01

Mirja Kühlewind on Jabber:
See all relevant information on the bof page:
https://trac.tools.ietf.org/bof/trac/ plus is at the very bottom

Brian on draft-herbert-transports-over-udp: "breaking middleboxes is a feature."
Makes transport innovation possible in a crypto world.

Can we use v6 extension headers?  Yes, possible.  And ignore v4 in future.
DO and HBH are already supported in most socket APIs, but more impaired than
UDP. Yoav on Jabber: "Doesn't HBH force the packets into slow path in many
routers?"

What about udp options (draft-touch-tsvwg-udp-options)?  Cool hack, but seems
less flexible for all the places we want to use it, like in user space.

slide 25: conclusion: Things we need
• A mechanism for making widespread cooperation
between endpoints and middleboxes explicit
• Endpoint control over explicit cooperation
• A clear boundary between what the path can see
and what it cannot, enforced by encryption
• A design for this facility that deploys on the
endpoints from day zero
• All this without requiring a trust relationship
between the endpoints and middleboxes

Tim Shepherd: would you be adding an RTT?
Brian: no - that's a detail. anything that gets exposed before crypto is set up
doesn't have a mac.  and then once you have the mac you can repeat the info.

Jana: Does this mean you can have a path layer not over dtls, no shim?
Brian: Yes.

Jana: Old mechanism wasn't "implicit cooperation": rather "co-optation"
Joe on jabber: "There's always coopetition."

Zahed Sarker:  Does the client know who's sending the reply in the "path direct
to sender" case? Brian: No, only receiver/sender in IP layer.  Could define it
as payload in path layer though, so I'm changing my answer to yes as I design
something while up here.

Joanna: How is this all going to work when partially/initially deployed? If the
middleboxes give preferential treatment to flows that have this marking, in
order to incentivize use of PLUS, how is this going to work? Brian: It's likely
that the initial phase 0 of deployment will have completely opaque traffic
(DTLS/UDP directly over IP).(Like slide 20.)

phase1 looks like slide 9?

phase 2 looks like slide 11?

dkg: Said you designed 6 kinds of ways to do this. This is a way to do that
with transparency? So we don't think this solves the problem of non-transparent
signaling? Brian: Yes, because we don't think that problem is solvable.

A question about how this will work with older middle boxes. Answer is that it
will just look like UDP to boxes that aren't PLUS-aware.

Dan Druta: We should be clear that we don't base this on a trust model. (?)

Muthu: Is the problem with UDP or with encryption?
Brian: Return to this in open mic.

Tiru Konda: Is there some competition with existing off-path signaling like PCP?
Brian: This is in-band, PCP is OOB. PCP only works within a single
administrative domain. Eventually, we can get to a point where we're talking to
middle boxes you don't know exist. Tiru: PCP has mechanisms to make sure the
endpoint and firewall have some explicit trust between them. How does this
offer better security? Brian: talking about a fundamentally different
architectural pattern; declarative vs imperative.

------------------------------------
Title: BoF general discussion starting
Speakers: BoF Proponents
------------------------------------

Aaron: Charter Scope, key points:
    1. Shim-layer protocol above UDP, that is implementable in userspace
    2. Supports ubiquitous deployment of encrypted protocols
    3. Transport-independent exposure of TCP-like flow semantics to on-path
    devices. 4. Permits other explicit signalling to be exposed from apps to
    path 5. Enables devoices on path to signal to sender

Jana: Good that there's a shim for senders to add info that receivers can read.
Jana: I am literally kissing the mic right now.  This is why I am at the head
of the line right now. Jana: Something fundamental here not yet pointed out:
(and I didn't really grok his epiphany; sorry :( ) Ted: The WG will focus on
TCP-like semantics because that's what middleboxes understand and require to
manage their own state. You are concerned about the network adding requirements
that would not make sense.  We're starting with flow semantics is because those
semantics are clear.  The registry for this model should be as small as can be.
Mirja: The reason we're not talking about more specific use cases is that
everyone has their own. It's a separate discussion. We're focusing on the
larger meta point with simple use cases; the rest will need to go to the WG for
further discussion.

Aaron: Charter deliverables:
    1. Experimental protocol spec
    2. Consider vulnerabilities
    3. Initial registry of vocabulary/types of info that can be exposed
    4. Implement
    5. Running code with webRTC data channel, quic, tcp, etc

Alain: how will this impact other things, like existing ICMP messages

David Black: What are the phases of this project, and what has to be
accomplished for the WG to be successful? Brian: Basic idea is that we're
fixing the ossification problem by re-ossifying, which we get to do exactly
once. We want to be in a situation where we define a system that supports all
of these interactions (scope key points) and be able to define them later, but
we don't want to wait for a routing layer hardware cycle for every change, so
we want something comprehensive so we only need to do it once.

Subodh: What happens when we get to phase 2 and there are protocols that never
explicitly send a SPUD signal? Is there any possibility that middleboxes will
treat this traffic as suspicious and start dropping it? Joe: In some networks,
all UDP is suspicious now. And in the future, there is potential for any
markings exposed to the network to be misused. PArt of this is getting the
incentive strucutre right, so operators recognize the downsides to misuse so we
don't wind up moving back to TCP and making user experience worse.

Zahed: How many vendors are going to be willing to do this?
Ted: The more people that recognize the path layer explicitly, the better. 
(Sounds like a "if you build it they will come" answer.) Mirja: This is a huge
change, so we have the usual tussles with deployment.  But since it should make
it easier for new transport protocols to get deployed, there are additional
incentives beyond just fixing existing path management. Zahed: I should hope to
have something in 7 years?  Should I be hopeful? Aaron: You should always be
hopeful

Slide:  Every BoF gets driven off the rails when someone says ... IPv6,
Multicast.

Eliot chanelling Martin from Jabber: How does the charter recognize and value
existing IETF work in this space, namely MIDCOM, NSIS, PCP ,etc? espcially NSIS
as this is in-path/path-coupled?

Harald Alvestrand: Is it decided that the extra information goes into the same
UDP packet as the payload? Brian: Noooooooooooo.....? But if it doesn't, then
you have to build a transport protocol for it? Harald: Is this ICE version 2? 
I am also worried about your MTU. Brian: So am I.

Christian: channeling star wars, "I have a really bad feeling about this
thing." I worry about overhead, at 1% of available bandwidth. Joe: Instead of
thinking of this as additional, could replace some information in other layers.
Christian: I think this is a terrible idea. Once you split information between
two layers, you ....? duplicate information? (I understand what he was getting
at, I think.) Brian: Not necessarily.  PLUS could basically be defining what
the first few bits (...bytes) of the transport need to look like.

Christian: You start with a set of use cases, starting with cases that are
widely recognized. Good idea for endpoint to cooperate with routers to prevent
DoS attacks. Widely recognized as a bad idea is for an endpoint to tell a
router which movie i'm watching. Brian: We don't recognize content revelation
as a legitimate use case Christian: Need a protocol narrowly-focused on the
problems we all recognize as good. Brian: Not proposing a content ID, or even
video vs non-video information.  What the network needs to know is the shape of
the traffic.

Lorenzo: coming back to "re-ossifying but only once".  Lots of things get
ossified, not only protocols.  Thoughts can ossify too, and we seem to be
ossifying on single path tcp semantics.  Thinking multipath is a fundamental
change that could invalidate some of the use cases, not really addressed just
by adding "s" to the existing uses of the term "path".  Needs to be addressed
up front. Joe: Agree, this does need to be done now. Aaron, to BoF proponents:
do you think multipath needs to be considered as part of requirements? Mirja:
Yes Lorenzo: So we all agree multipath is a go?  Excellent.

Erik Nygren: Given that we're looking at this as experimental, shouldn't the
charter be more explicit about just what the experiment(s) is/are? Erik: One
dimension is: what is the bare minimum we can do to cover use cases everyone
really agrees on? (As IPv6 options analysis shows, we'll get ossification
around the stuff that people *actually* use.) Erik: Other dimension is: part of
the experiment is what are we ossifying onto? We call this "experimental", but
the reality is that this will end up proposed standards track. The experiment
is "How quickly do people ossify onto an experiment?" Will we just end up with
an extra mandatory header? Erik: While we're calling this experimental, we have
to be prepared that this will be the new standard and we might rapidly ossify
onto it as part of the experiment.  (Editor's note:  this is what happened with
EDNS Client Subnet in DNS.) Brian: Yes, I would like to put that in the
experimental considerations section.

Tom Herbert: Goal of PLUS is to signal to middle boxes, both known and unknown.
What if I trust a particular middlebox, but don't want to reveal information to
middle boxes I don't know about. Do I really need to signal every device on the
network that I'm watching a cat video? Ted: Tom Herbert: I have a device on a
mobile network. I want to give information only to the immediate upstream node
(1st hop) because that's probably where the information is needed. Ted:
emphasize that this is declarative, under sender control.

Stewart Bryant: is this a new layer or what layer would it be in? routing
layer? other? mirja: routing out of scope here...

Aaron moving things along at a brisk pace: patience ebbing.

Wendy Seltzer: More restrictive language is better; fewer privacy issues.
Remove some extensibility bits at this point to reduce potential for networks
to compel clients to send signaling or packets will be dropped.

Randall Stewart: Would like to be implemental in kernel too, not just userspace.
Proponents: agree
Eliot (via Jabber): Did it with sctp and had good results

Randall: How does this work interact with quic?
Ted: Orothogonal.  Quic work can continue, and we expect to benefit from their
experience Brian: they'll co-evolve.

Joanna: If this ossifies even more, how many more shim layers are we willing to
keep wrapping?  tcp? one protocol I'm really worried about ossifying is TCP.
What are your thoughts on TCP? I'd like to be able to hide all my TCP headers
and stick one of these shim layers on. Joe: We're really trying to avoid that
by digging deep into use cases and goals and getting it all right this time...

Aaron: closing discussion moving on to hums
see also: "Re: wrt quic, plus, and taps?"
https://www.ietf.org/mail-archive/web/ietf/current/msg98993.html

dkg: I'm not convinced that in any of this the analysis has been done to decide
who has the power in the relationship of the endpoint to the network and how
this impacts it.  Despite the language of openness and explicitness, it seems
like we're giving even more power to the network, power for coercion not
co-operation.  It looks like opt-in, but could very likely be in
opt-in-or-else. Natasha: we have a problem, with the problem being network
management and not business-enabling goals. dkg: I understand that and
appreciate it.  I don't think this architecture is the right one to prevent
abuse. Lorenzo: Can I give an example of how this can make things worse?  If
you don't opt-in with an identifier, I will deny service.  Customer complains
oob, but if we build a clean framework to communicate the denial that's way
worse. Mirja: How can you possibly enforce this?

Ted: Would like a hum: If this problem were limited to flow state semantics
ONLY, would more people be interested?

Aaron: If scope were restricted to flow state semantics and multi-path, would
that be agreeable?

Lorenzo: OBJECTION!  I don't think that's possible.

Room hum: seemed pretty even, with slight bias toward the finer scope focus.

Mark Nottingham: By standardizing this, we might make new things possible. That
worries me, because I don't understand how this impacts end users. Often on
mobile networks, the carrier owns the device. So it can do things that the user
isn't even aware of. So the possibilities for abuse here frighten me. Worried
that we're going to launch into something we don't really understand. Look at
cookies: they seemed like a great idea in 1995, and now look at where we are.

Phil Hallam-Baker: THEY SUCK! [cookies]

ekr: SO.... [applause] Aside from the middle box people, is anyone else
interested in even transmitting these bits?

Georg Mayer: there are things that people can be afraid of but we are really in
need of such mechanisms.  we now have 15 years of experience.  we should build
on this to fulfill requirements but address people's fears and we need a
starting point.

Ian Swett: Not clear which parts middlebox vendors are excited about. When I go
through the goals, I am excited about some of them, but many of them are
unachievable. Re: FIN, we're already in a world in which close happens on
timeout, so we already have that problem.

Tom Polly: I'm not sure we can solve everything desired here but there are
concrete problems that Apple would like to solve that would make use of some of
this.  I support WG work on it.

Colin Perkens: (garbled first part) and helping me find a transparent path

Yoav: as a middlebox vendor, would like to see these aspects of protocol
clarified if it were as simple as just being able to understand when flows
start and end.  This seems too extensible, perhaps beyond usefulness.

Spencer Dawkins (transport AD): please continue discussions on spud
mailinglist.  Thank you all.