TAPS Interim May 8 2019

Attending (add your name so colors are noted)
* Aaron Falk
* Gorry Fairhurst
* Tom Jones (notetaker)
* Max Franke
* Michael Welzl (MW)
* Colin Perkins
* Jake Holland
* Tommy Pauly
* Philipp S. Tiesel
* Jonathan Lennox
* Brian Trammell
* Anna Brunstrom
* Spencer Dawkins
* Kyle Rose
* Mirja Kühlewind
* Eric Kinnear
* Magnus Westerlund (MaW)

Agenda:
1. Framing - Tommy
  * See https://github.com/ietf-tapswg/api-drafts/pull/323
  
TP: (see above thread on pull request). This pull request is reworking the text around the framer in the API document. It is hard to represent the different parties in the pseudo code in the document. 
AB: Would a picture help?
TP: Aaron made a comment to include a railroad diagram.
AB: and which of the descirbed calls related to the API to the application. A figure might help. 
PT: my main concern is that the framer api and the send and recv calls are looking quite different. If we could align them it could be easier to understand how framers fit the picture.
CP: I thought they did radically different things. This makes framers more visible that I expected to see. I expected them to be more implicit, part of the stack. We need to be clear about the model.
TP: in many cases an application doesn't need to know about this, i.e. a websocket doesn't need to know about the http interaction. for the interface document we need to document when they do interact.
CP: it makes the interface look very complicated. 

NOTE: add more text saying "most of the time you don't need to do this"

BT: we need to be explicit that an app might want to do this, but most of the time the selected stack does this. The app needs to be part of this choice, but the choice be which stock framer is used from the library. part of this is implicit in the implemnetation. We need to call this out in the interface document. 
TP: when not provided, we need some sort of common set for protocols. 
AF: an example would help here
PT: is there a way to specify for transport X can I have framer A?
TP: that is missing now, dns over tcp vs dns over udp is a different framer. 
CP: isn't this an internal detail? 
JL: that is great if the stack knows what dns is. pressumably you want to use this for your propietary protocol.
CP: maybe there are two apis, one for a user and one for an implementer
JL: what is the difference between these?
CP: we can make the api much simpler with clear distinction between these two.

JL: back to PT's point, I am concerned that this model is different to the application API. This sort of feels fragile if I am doing exactly the same thing as the data api. Maybe we need a taps api above and an api below. 
CP: you send and recv objects, the framer turns those into a raw bitstream
PT: if you stack framers you can get meta operations that only deal with messages or parts of the object. 
JL: I feel there are cases where, depending on the protocol, you might or might not want to be a framer. In websockets or start tls, something might happen and I have to change the protocol to use framing. I want the stuff before that to use the standard taps consumer api. 
TP: if you are on top of the connection you are the thing that is driving flow control. you get the events. in the framer model, you get the message, it must go now. you can't add a flow control layer. on the recv side, you have to parse or interpret the stream. You don't control when the app consumes these.
JL: there needs to be a buffer between the application and framer, where is that?
CP: the app doesn't say I want these bytes, it gets an event from the framer.
TP: the framer delivers message, the connection api has to manage the world and deal with what is available and what is delivered to the application. 
JL: tls presents a stream not a message
TP: 1 big message
TP: the doesn't need to be aware of what the application wants in terms of read size. It just have to deliver.

TP: to clarify, a message framer is either a library object or something from your app and the framer instances this object to help you interact with that one connection.
MK: can we provide a little bit more support in the transport? do we more than detecting a delimiter.
TP: i agree most things are header footer or delimiter. There are extra things where there is some encoding to do a little more. You can't simply extract a length field from a certain spot in the header.
JL: I want TLS to be a framer
TP: you should be able to encrypt
MK: isn't tls only encrypting the payload right?
TP: yes, tls's records are fairly easy to parse. getting ap application record needs more of a transformation.
MK: is there a middle ground to provide both?
JL: it would be easy to have just a header stripper on top of a flexible interface. 
MK: if you implement taps in kernel you get a context switch for every packet. 
TJ: if this is a problem we can do larger copies between kernel and userspace. We don't need to fix this in this API.
  
2. Implementation draft - ??
  * See https://datatracker.ietf.org/doc/draft-ietf-taps-impl/
  
TP: I guess we want to cover restructuring
AB: one main part is to pull out protocol specific parts
TJ: I went through this and brought it inline with the other documents. we should create issue now if we can
AB: we have two pieces now, the policy part and the protocol part to see if we should pull the text out per protocol.
TP: in 248 we had two proposals, I am leaning to option 2.  1 - we have a section to go through all the protocols and figure out the mapping. 2 - sprinkle the protocol text throughout the document.
  https://github.com/ietf-tapswg/api-drafts/issues/248
TP: we need working group concensus on how we want to do this
GF: my 2 cents is to do it like option 1
MK: I am not sure 1 is possible, lots of things aren't specific to one protocol.
GF: sure, you conver it in the general section.
MK: I am not sure you can do a specific section for happy eye balls
TP: happy eyeballs is covered in the generic section now. 
AF: thinking in terms of the bulk of the document, how much protocol specifc text is there?
TP: not so much protocol specific stuff now, but we could add more. We should look at how much (de)duplication there should be. If we go through each protocol they have to point out generic things.
AF: you could go either way, I think you get both.
JL: you should do the refactor as you go. As you write pull out the common things to common sections.
MK: The advantage to having a per protocol basis is to help implementers if they are only implementing taps for 1 protocol. The other order makes it less reduntant. 
GF: my argument for the first one is what Mirja said but for what Aaron said.
MaW: I agree with gorry, it makes it easier to implement more protocols.
MK: if it is all mixed up it makes it look like you have to implement everything.
TP: in the generic sections we could define common protocol functions for sets of protocols. it gives us a definition we can have references back to. We still have per protocol sections, but they call back to a common directory of functionality.
AF: for n+1 not in this draft we want a template to support taps with it
GF: i love what was just said for that reason.
AF: Tommy has a good proposal, it is hard you now need to come up with a good set of approaches. 
GF: it might be doable. We can do it by section. When doing connect we can say it is connection oriented or not. I think it is doable.
JL: I think the organisation is hard, connection oriented is one group, multiplexed is another these are orthogonal.
AB: it is already a bit like that in the document. 
JL: having it all over the place makes the n+1 problem harder.
TP: we do have section 9 (specific transport protocols) it kind of does it, but it isn't clear enough.
AB: it is quite different now what is described for the different protocols. 

TJ: I think with moving the redevous text around we have enough of the considerations to enable protocols. I don't want to have to directly talk about proxy protocols.
TP: maybe we need a little more text after the new text to say .... (can someone fill this in?)

3. Parameters & Defaults - Gorry or Phil
  * Default for Use 0-RTT session establishment #314
  * Default for Control checksum coverage #315
  * The recommended default and can we specify rather than recommend? #317
  * Section 7 default #319
  
GF: why don't we agree?
JL: is this issue 319 ?
MW: I see you make these proposals about various defaults where you say you think there are too brave. Shouldn't the taps system try to be more brave than the normal protocol defaults? We are in the space of trying and falling back.
BT: some of these cases arethings that are very tied into the application semantics. Most applications will never case about this. Most of these are here becauae it is in minset because it is in 8095. 
AB: most applicatiosn don't care about this. in the section with ignore that has to do with what we do during selection. 
BT: ignore means use the system defaults for this
PT: ignore means - where the protocol does it or not don't use this for ordering.
BT: if you have deteremined UDP-lite is usable on the path and the other side is litenign for udp and udp-lite, when you initiave you are going to ramdonly choose between udp and udp-lite.
AB: you are still going to make this choice in your policy?
BT: in the case where these are basically equivalent. the taps system should know which to use. It seems reasonable to select both. We get more udp-lite on the internet. I could buy that argument, the problem is if you are in a situation when you are not 100% confident on the udp-lite prober then having it as ignore rather than avoid creates a connectivity risk.
PT: that is an implementation issue and not the API.
MW: it is fair to implement a taps system that in the face of ignore it doesn't choose udp-lite.
GF: I don't get it, you are saying the stack can choose in this case?
BT: only if udp-lite doesn't present connectivities issues. if it knows the other side is listening.
GF: you can't know what on the path might cause udp-lit to not work. If makes it harder for hte application developer.
MW: do we want to allow random choice of things we don't expect, or do we want a system that is prone to working?
GF: running a udp(ish) app, I don't want to jsut get udp-lite because a connectivy mechanism says it works. This turns me off taps
BT: how do you know as taps that udp-lite is safe by sending a udp-lite packet? 
GF: we would use sctp and everything else in this manner.
BT: the reason to do this is to actually exercise the network. we are testing the network to create more traffic to make it used to not screwing us. 
GF: I love the measurement campaign. As an app developer I now hate taps.
BT: the selection process should never select something that creates an availabilty risk. Unless you have said you want it. 
PT: if you have ignore here it means the taps system can decided to choose udp-lite instead of udp. The implementations probably won't do this now. That is implementation not api. In 10 years we want to have enabled this. 
GF: to make that happen a server must exist to pick this up. When they did this they probably set a client policy to try this. Why do we have to change the system to do that. 
PT: if you have new deployment, no one will change the existing clients. The taps api in the operating system may do the update. Your vendor business app won't do this.
MW: udp-lite is a bad example. it is an application change and not just a policy change. They have to implement some support for it anyway. Do we have an exmaple where a policy update will make things better?
AB: policy can override the default
PT: the api defaults should always let policy decide. Lets not restrict the policy from controlling the api.
GF: I am okay with policy doing what it needs to do. 
PT: if the default is prohibit the policy changing that to ignore is unexpected behaviour.
GF: I don't think so. Maybe we are seeing the difference here.
AB: prohibit is different. Avoid is better. prohibit you might want to rely on for an application. I don't think you can override this
BT: avoid is better than prohibit
AB: we never pick this unless asked to. 
BT: if you absolutely need it you can change it to require. If you can handle not getting it your can preffer. 
AB: does avoid seem reasonable
GF: I am trying to work out what that means
TP: should ignore be named allow? 
PT: I am not sure this make sense for all properties that are preferences. I think allow can get awkward
GF: can I check how this works. When an app says it wants to do something it gets the choice. If it doesn't want it, it never gets it
PT: when it sets forbid it never gets it
GF: this is the api, what happens in the system default
BT: we have handwaved this away, this is going to be implementation specific.
PT: I think require and prohibit are things you can't hand wave away.
MW: we should have some text that says what the policy manager can override - from "prefer" to "avoid". 
GF: we have to be careful, are you saying prefer and avoid can be changed to any setting?
MW: yes. I don't think it makes sense for a policy manager to force something on an application that says "prohibit", and if an application says "I require this, or fail on me", then it should fail if it can't get what it needs. 
JH: it seems to be that a lot of this can be solved, if you seperate the systems listing of protocol. By default you would pass in the system selection and the actual connection is working with a list of protocols that the application dev can change if they want. 
BT: that is kind of what we are trying to do here. Specifying the precendence between these is hard. We have punted a little bit on this and said it was implementation and deployment specify. We are aguring now about the defaults in the document for how application fall back to reasonable default values. We all realise that the default will be what is used. 
JH: as long as there is an easy work around for the app dev it isn't really a point against using taps. 
BT: when you know there is no risk to using udp-lite vs udp, should the system sometimes use that if someone sets all default. That is the discussion we are having now. This seemed the easiest one to discuss.
GF: did we make progress?
BT: I took the action to write text for this issue, I wrote now "switch from ignore to avoid" - is that right?
(someone): yes!

GF: What about 302? I argue it gives unexpected results if you allow idempotent data through.
BT: MW is correct. UDP is well understood to be idempotent. Do you MW have text you want to write?
MW: just because I brought it up I thought it was probably my duty to do it. This needs to be somehow in the api for everything on a taps connection. For UDP it would be everything on the connection.
GF: udp everythign would be idempotent
PT: you can set idempotent in preestablishment and you are done.
BT: is it well enough understood that udp messages are idempotent
MW: is this explicit or not
BT: there needs to be explicit language in the document.
JL: you shouldn't have to know to do this for udp
BT: udp is alway idempotent. If you aren't willing to accept that you taps needs to not select UDP.
MW: the text now just covers 0RTT establishment
GF: we need a different name for this. 
TP: if we had another protocol like udp with a different name, would it not have this property? If you allow something to be unreliable you get this, so I check that with unreliable I get idempotent. 
MW: with sctp this isn't the case. You can play tricks with PR
`JL: there are protocols that can be unreliable 
MW: unreliable means unreliable, with UDP you must handle duplicates
CP: raw udp is pretty uncommon without some sort of framing layer.
GF: I always liked profiles, it seems the standard udp profile is idempotent, unreliable...
AB: we need some way to specify it in the document.
GF: my concern is that if you have to specify it, it won't be clear that you need it. What default would you expect?
AB: I would expect it to be avoid?
GF: just avoid UDP as a whole?
AB: right now we have reliablilty as the first choice. If you do not specify anything you get reliablity.
GF: I think it is good to have a bundle that represents raw udp without having to pass all of these parameters.
TP: if someone things unreliable means udp needing something more to do it will be annoying.
JL: I worry there will be cookbooks for doing udp in taps on stackexchange. 
GF: that is probably enough for an issue to be raised
MW: I agree with what I heard, but I am not sure how to solve it. Have a set of profiles that are a group or properties.
GF: are there many of these, the default seems clear. It is a solid basis.
MW: we could have a basis for tcp udp and other protocols.
AB: there could be some conveincance paramaters that cover a few of these.
PT: I think it makes more sense to have something like a profile added to the api that sets a bunch of defaults. On the otherhand we might want an explicit "I want protocol x" for some reason.
JL: I agree. If you don't do the latter people will do the former to do the later
GF: I would write require udp as a shortcut. I don't want this in the api. I want to say "datagram" and datagram is this reasonable set of stuff that requires these features. Usually this gets UDP, but in the future it gets something else. They might be smarter, but at least they haven't bound to using UDP. 
MW: TCP is probably like the defaults. So it is just a matter of making a property that gets dataa.
JL: is the default TCP or does it include TCP
GF: the default includes TCP. It includes things that look enough like tcp and you can add properties that get you close to what you want. 
PT: I would ask for this profile mechanism to be distict from regular properties
BT: I would caution against having profiles at all and stick this in an appendix. "This is how you translate from what you are used to to taps". profiles should be a thing to illustrate how to do things.
GF: I disagree, this was a good thing in the neat stack. it gives you an extra layer of abstraction.
MK: isn't this just an example, people will just use examples when they can.
JL: SOCK_STREAM doesn't have the word tcp in it and when it was written it probably bound to more. 
CP: we want a feature that allow "I want something TCP like" and the document should help
MK: isn't hte thing with taps making it hard to select I want TCP only.
MW: there seem to be people in one camp and people in the other.
--cut off by chair --

AF: there are too many ascpects to resolve in this call.
PT: I might be able to provide two pull requests for this. Going either way.
GF: I will happily trash them or support them.

ACTION -> GF will try writing up the profile case
ACTION -> PT will write an API PR to select a specific potocol, e.g., TCP
ACTION -> GF+TP+PT try and make a table of defaults

AF: that will help the discussion

  
4. Yang Model - Jake
  * See https://github.com/GrumpyOldTroll/python-asyncio-taps/blob/master/PyTAPS/modules/ietf-taps-api.yang 
  
JH: I put up some examples. There are some parts I don't hate. I have a question, the idea is that when you have multiple sets of remote and local addresses you do a permutation explosion and race them all?
TP: I don't have sets of local addresses. I require a specific address or a type of address. If you allow certain types of interfaces to be used, it is up to the implementation how many you try. Systems normally have a preference and chose the best one. It is more useful to have the explosion on the remote side.
JH: if you do specify two different ports and it maps to four different things you will end up with 12 racing attempts.
CP: if you are using something like ICE that is what it does.
TP: even normal happy eye balls priorities things. You might have 12, but you will only try all 12 if the first 11 don't work.
JH: great, that helps me tidy up the things I don't like about taps now, which is an endpoint id in the object. I will fix and resubmit. There are aspects of yang geared towards netconf, we are not netconf so it gets awkward. When you have a list you need to have a key in the list if it is config. the key in the list has to be unique and requires atleast one element of the object. You can have multiple elements have the key, but the combination has to be unique. I think I can fix that.
JH: does anyone have advice on the next steps? 1 was submit the yang model as an appendix in the interface draft the other was to have conformance tests. I am not sure what to do.
GF: did you decide not to have a seperate document?
JH: no strong decision one way or the other. I think seperate document will be hard to maintain as the other drafts change.
JH: mostly I want this to be in good shape, that is what I am focusing on now. I hope it is useful and someone can look at it. Review welcome
TP: on the conformance test apsect, that sounds useful, is this description you have what would go into a format. Would we be able to reuse the same objects? 
JH: that would be useful, it would be good to make a 'here is what we ended up with' output. I haven't done that yet. Some of the trace stuff from the impl draft would be good. 
TP: would that be part of the same model?
JH: yes, two seperate containers in the same model.
AF: maybe put what you have in a draft and we can discuss this in montreal and see if we want to take this as a working group draft. 
JH: that sounds fine

ACTION -> add to Montreal agenda "why do people resist specific procotol selection"

GF: what is your position?
JH: my position is based on the adoption rate amoung developers and operators. I would like to understand the real pros and cons that people see and if everything has been considered. I don't understand the real objection here.
GF: I think you might find there is disagreement here.
AF: great topic for discussion on the mailing list if someone wants to take it there.

ACTION -> GF will try the datagram profile on github then "why we can't do specific protocols" on the mailing list


5. ARCH & API issue scrub & assign - Brian
  * See https://github.com/ietf-tapswg/issues
  
BT: I suggest we start oldest first and catch those without an assignee
PT: I can take 301 and you can close 222
CP: it wasn't clear we could always reuse preconnections.
BT: in the case of listening we can.
CP: for things like opening connections you can't reuse it, it will fail as it is in use.
TP: if you use a preconnection you can reestablish from a preconn a similar thing

BT:
PT: I think we should leave it as is.

BT:
TP: this may be easier to deal with once we peel off the listener object. A lot of discussion was about how the listener object works.

BT: 
    286 interface should describe transport level padding
BT: it is a privacy and security thing. If we pad at one layer we should pad at all layers.
GF: I think it is silly and opens a can of worms. Padd at the app layer and you are done.
TP: if you pad at the app layer and you are quic, you are potentially adding a lot of overhead when the transport could do it.
BT: I would have a transport property taht says 'transport provides constant sized network layer datagram'. That is out of charter.
JL: once quic is published is this in charter?
TP: we could say we won't do it now, but can do it later
JL: padding is just one way to do this.
MaW: why isn't quic supported in this? quic has the protocol functionality to do it
BT: some implementation of quic could do this. As long as we don't have a definition for how to do this we can't include it. 
CP: you could do this with RTP
MW: for no larger than 256 bytes
TP: we should put this to science fiction
TP: this seems like a great experiemental extension to taps

BT:
MW: can someone explain what that means for TAPS?
TP: I think that relates to some of the other pooling stuff. We can close it
GF: lets close it and see if it comes back

BT:
    303 more expressive multipath transport property
TP: we removed and were talking about adding it again
TP: this was discussed at the multipath group. 
BT: can you write up this property?
ACTION -> TP write up mp property
GF: are we going to set a default?



Etherpad:
https://etherpad.ietf.org/p/nots-interim-2019-taps-02?useMonospaceFont=true

Information about remote participation:
https://ietf.webex.com/join/taps | 316 448 940