TAPS Interim 2018-09-12
AGENDA
1. Partial Send/Receive, Tommy
- https://github.com/taps-api/drafts/pull/200 (merged, but there was some discussion at IETF 102 and on Github)
- interface needs to work for streaming and very long messages, when you want to start processing before the entire message has been received
- define simpler interface without partial messages, then have partial interface be an extension, with message context
- receive parallels the send call
- currently the interface does not support receiving chunks of a single message out of order
- Mirja: We need this mechanism, but we may not need the same functionality on the sender and the receiver side. Receiver side we may not know the boundaries and we need to signal that
- Brian: Document should cover that with the deframer. And "out of buffer"
- Mirja: That's a different signal.
- Brian: File that as an issue.
- Tommy: On the sender side you could totally ignore partial sends. But on the receiver side you cannot totally ignore it.
- Brian: You can supply a deframer that will always split the stream and then implement it in your application. Or you trust your application to never send anything larger than your buffer, which I wouldn't recommend.
- Tommy: If I have a framer, I assume that it's bidirectional, so I also have a deframer on the other side.
- Brian: I wouldn't say that every application will meet that guarantee.
- Mirja: Deframer needs to be the same on both sides, but you can either tell the transport or not.
- We probably do not need a signal whether message boundaries are handled by the TAPS system.
- Mirja: We should signal Ordered and which chunk of data needs a framer.
- Brian: A framer per message?
- Mirja: Instead of partial, we want to tell the transport whether to add a framer
- Brian: That's a different problem.
- Mirja: What would TCP do with it? Just send the data, as it always does.
- Tommy: But assume we have a framer, like TLV. If we have a partial message, it cannot send, because it doesn't know the length. But with a delimiter it can.
- Brian: For transport protocols with a framer, partial is basically a signal to the framer. Do we have that in the document? We should talk about the interface to the framer, that's another issue.
- Tommy: Another issue we need to address is whether or not the data you are sending is semantically interesting for boundaries. We may have layered protocols, we care about the top-level protocol. Like "This message is in terms of protocol X".
- Jonathan: Do not assume too tight coupling
- Brian: Anything that TAPS selects will be underneath the lowest framing level. A framer has to live on top. It is an invariant.
- Jonathan: I don't think that's true.
- Brian: Then they're not raceable.
- Jonathan: HTTP2/QUIC and HTTP2/TLS/TCP. The framer could either be on top of what we're racing or part of what we're racing.
- Tommy: We may have to serialize differently over QUIC or ...
- Mirja: There's also a difference in if they expose streams.
- Colin: The application should see the same thing no matter if it is using HTTP2/QUIC or HTTP2/TLS.
- Zahed: Let's propose something on the mailing list or discuss on Github, especially the framing issue.
- Jonathan: Is it possible to change the framer on a connection? Use case websockets.
- Brian: We don't have that yet. But we might want that.
- Jonathan: You have to make sure the old framer doesn't consume more than it's supposed to.
- Brian: Can we agree that you can't switch the framer during racing?
- (general agreement)
2. 0-RTT, Brian
- https://github.com/taps-api/drafts/pull/214 (merged, but there was discussion)
- Sending at the same time when initiating for 0RTT is special
- New call on a preconnection, ugly name to force people to understand that this is idempotent send, as it's a different contract
- alternate proposal by Colin with explicit Start() and Initiate(), but this is complicated if we have to do this every time for every connection.
- Tommy: We handle this like in Colin's proposal, you Initiate(), then Send(), then Start(), so you can do multiple Sends. Also during racing you can Send more during later racing attempts. Siri is using this. Start is the beginning of the racing, and then further sends may be part of idempotent data.
- Brian: But you can do the same with the PR that landed (#214) if you specify Idempotent in the Message context. There was another proposal to implicitly wait from Initiate() to the first Send() to see if it is idempotent.
- Tommy: But that's just another version of Start()
- Tommy: I like PR 214. But with our threading model we needed Start() to pass a connection, add callbacks.
- Brian: InitiateWithSend() doesn't change the events that you'll get. I want to avoid that someone who doesn't know about 0RTT has to call Initiate() and Start(). That's weird.
- Tommy: How does the TAPS system know who to deliver the Connected event to?
- Brian: Initiate() gives you an object and it's on that object. Registering the handler is language-specific. But that might be a reason to use the Start() thing. The section will stay but its content changes.
- Tommy: If I call InitiateWithIdempotentSend() and there's some error, who then gets the error?
- Brian: The object you get the callback on is an argument to initiate()
- Mirja: I think it's right to give data to the preconnection because you want to add it to the racing.
- Colin: I don't think that's right. Racing happens when you have a Connection.
- Mirja: Are we not doing Idempotent data on Rendezvous?
- Colin: Rendezvous is tricky. You might want it when you have your candidates, but I'm not sure how it works with this.
- Brian: The idea of my PR is that we defer how it works with Rendezvous.
- Zahed: The PR holds?
- Brian: We can bikeshed the name on the list.
3. Connection & Message Properties, Michael Welzl
- Message Dependencies: https://github.com/taps-api/drafts/issues/220 - Should we have a Property that encodes dependency between data, so a partially reliable transport could make a decision which data to drop?
- Con: Added complexity.
- Pro: Useful to have.
- Opinions?
- Colin: If you're sending partial reliable data with deadlines, you need to know if it makes sense to discard some data. For real-time video for example. If not, the stack might discard data that you then need.
- Brian: TAPS is an exercise to get application developers to new and scary things. Handing over dependencies to the transport is just one more. The cost is that the stub implementation will not do anything useful.
- Jonathan: If the implementation doesn't do much for me, why would the application provide this information?
- Brian: You already have the information
- Gorry: If you have a deep buffer at the sender that you can't control, this is cool. But in TAPS we expect the buffer to be shallow. Why bundle this functionality with our interface?
- Jonathan: Even with small buffers you might have long RTTs.
- Gorry: I would love to see such things in an app, but does TAPS need to do it?
- Colin: If you're writing a TAPS interface to RTP you'd want that.
- Anna: SCTP will not even support this, right?
- Colin: RTP has done this for 20 years. The socket API pushes this complexity up to all applications. You can include this above the API, but you can implement everything above the API, like congestion control. That doesn't mean we should.
- Tommy: TAPS allows us to change this model, we don't need this small buffer. Right now you need a shallow buffer and potentially not have the pipe as full as it could be. But with TAPS we can enqueue e.g. screensharing frames with deadlines and not have to worry about it. The TAPS system can figure out which ones to send. We have implemented this, you can drop with dependency trees etc, even without protocols underneath that support partial reliability.
- Tom: At what point do you commit data to a buffer where you cannot cancel? Like the rest of the protocol stack?
- Tommy: When you have enqueued send(), there is a socket buffer equivalent of data that has not been consumed yet. And in that queue you can apply deadlines etc.
- Gorry: But why not implement it in a library above TAPS?
- Anna: I'm worried about adding a lot of functionality in the TAPS layer. I thought this was not in charter, I thought we would only expose services in minset.
- Tommy: But we also have 0RTT data and the TAPS system will have to queue it somewhere.
- Tom: You can write a lot of data to the socket and then invalidate the data.
- Tommy: With QUIC these deadlines can apply all the way down into the protocol stack, as we develop transports that do this in a smarter way.
- Colin: My mental model is that the deadline is when the data will be received. The sender keeps an RTT estimate so it can know when to send.
- Gorry: This sounds a lot like what is needed for something like an RTP server.
- Michael: I don't sense convergence. We need to discuss this more.
- Mirja: You could also put it to the front of the transmission queue.
- Anna: I thought Niceness on Connection also has to do with Congestion Control.
- Michael: That's Capacity profile. Or the priority of the stream.
- Mirja: If you have different Connections that are grouped and they are mapped to different streams, the priority would just decide to reorder the sending buffer. If they are not grouped then you have separate sending buffers and then you might use this to configure Congestion control.
- Anna: Maybe we need different names for these. I think these are different and we should keep them.
- (general agreement)
- Brian: Let's file an issue.
- Fragmentation prohibited: https://github.com/taps-api/drafts/issues/226
- Michael: We wanted to prohibit network fragmentation. But maybe we also want to prohibit fragmentation above that, for Path MTU discovery.
- Mirja: The only case you want to do it in the application is if you implement your own transport on top of UDP.
- Brian: And if you want to implement Path MTU discovery and not have a competent TAPS implementation do it for you.
- Jonathan: But what if we do TAPS right over UDP?
- Mirja: Yes, if you write your own transport over UDP, but then it's a protocol-specific property.
- Brian: Segmentation prohibited is not useful.
- Tom: The DF is probably a property in your framer.
- Brian: We don't have properties for framers yet.
- Jonathan: If you implement something in TAPS you probably want to use the TAPS API too.
- Brian: I'm sold on segmentation prohibited.
- Tommy: I feel like this should be a per-message protocol specific property, agree with Mirja. So we should agree whether this is generic or specific?
- Philipp: Agree, we should have fragmentation and segmentation prohibited, an segmentation as UDP-specific.
- Jonathan: Not sure I agree. (...)
- Brian: There is only one generic, fragmentation, which will be implemented differently. I think we only need one of these. UDP will implement it as IP-DF.
- Jonathan: Are transports just ignoring meaningless protocols?
- Brian: Yes, then it's a NOOP.
- Mirja: ... TURN...
- Tom: How is your TURN implemented with TAPS?
- Michael: I suggest we get text in about segmentation and IP layer fragmentation and see how people react.
4. draft-pauly-quic-interface-00.txt, Tommy
- https://tools.ietf.org/html/draft-pauly-quic-interface-00
- We'll talk about this at the upcoming QUIC interim. It defines the transport services mapping if you're not doing HTTP. At some point we will probably start a discussion on the QUIC mailing list, would be useful to have TAPS people in that discussion.
- Interesting question on whether every TAPS Connection is a new stream or not. Gets interesting for unidirectional streams.
- The other mapping is letting the Connection be an entire QUIC Connection, so every Send is a new stream ID. This maps well for HTTP. For short requests it might be overkill to have a new stream for each one.
- Please read it and give feedback.
- Jonathan: In the stream model, is there an event for new stream?
- Tommy: You can create a protocol-level listener which accepts streams associated with this. Also interesting if every stream is a message, if you want to explicitly receive a reply to that message.
- Jonathan: It should be possible to implement the connection model on top of the stream model, if you have a stack that only exposes the stream model. Making sure you can do that it probably good.
- Mirja: It's two different use cases. HTTP over QUIC you want the stream interface. But a random application that sends independent messages doesn't care.
- Zahed: We need to think about our priorities. QUIC doesn't have prioritization.
- Tommy: This is interesting related to the Niceness discussion.
- Zahed: Can you do Send() and then SendIdempotent?
- Tommy: SendIdempotent is the same as Send() with Idempotent marked, and it just means that it's eligible to be sent as 0-RTT.
- Philipp: About transport as QUIC connection, is there a way to map receive on the same stream ID?
- Tommy: You need a way to specify "receive a reply to another message".
- Philipp: My work with connection pools brought up the same problem, solution goes in the same direction. Either have a parameter for Receive() or specify a receive handler directly with the Send().
- (...)
- Brian: This seems to violate one design assumption that you can use this over all transport protocols. We tried this in Post and found that we need a message reference on the wire.
- Tommy: We could add this to a framer, but then it's not so useful directly with QUIC.
- Michael: Transport Connection as QUIC Connection - what's the benefit, other than looking convenient in the API?
- Brian: It works well for the HTTP/2 mapping. Right now the stream identifiers are semantically entangled. Have it on a per-message basis makes this better.
- Tommy: You can have a framer that sits on top of multiple connections and streams that merges them back together.
- Brian: We need access to Stream IDs, but you could have this as a Message Context Property. Maybe this is a third option.
- Mirja: This goes back to Ordered. I proposed we need Message Groups, which is kind of like a stream ID.
- Tommy: You could also make the groups Connections.
- Brian: The Received event is on the Connection. This document is good to talk about the space of discussion.
- Jonathan: I think the Connection model is the lowest level and everything else is syntactic or semantic sugar on top. What are the useful simplified interfaces for the most important use cases?
- Michael: I agree, but if I call it a shim, then nobody wants it.
- Mirja: With HTTP/QUIC, where do you implement the fallback to TCP? In the application, TAPS, ..?
- Michael: We want a uniform behavior above what we are currently defining.
- Zahed: If it selects TCP and not QUIC, the application needs to be notified.
- Mirja: It's not bad to implement the fallback in the application, it's not the same HTTP2 over QUIC or TCP.
- Zahed: One point to have this API is to have some synergy in implementations.
- Tommy: I think this goes back to the protocol stack equivalence in the Architecture. We may require a framer on top to make QUIC and TCP equivalent.
- Gorry: Are you planning to do H2 over TAPS and migrate between QUIC and TCP?
- Eric: Yes. There is another document in the httpbis group. Essentially defines how to do transport over H2 the same as QUIC transport without HTTP semantics on top. ((Bi)directional arbitrary bytestreams)
5. Implementation & testing, Chair
- Zahed: Last IETF, we decided to do Standards Track, but to make sure that it is implementable, that implementations are there and tested. But how will we do that?
- Gorry: Is there a new requirement from the IETF?
- Zahed: No, but this was consensus in the Working Group.
- Gorry: I didn't hear the "tested" bit in the meeting.
- Zahed: If somebody uses your API, I think that means it's tested.
- Gorry: That's a Working Group decision, not an IETF decision, so we can revisit that?
- Brian: I'm also not clear what tested means. You can test the implementation. But it's not an interoperability test. What are the criteria for Pass or Fail?
- Jonathan: This isn't strictly a test, more of a review by implementers.
- Tommy: This is not interoperability, but there could be a standardization test for the interface. If I'm calling Send, does the created packet match the expectation? We could validate the mappings.
- Brian: For that we would have to publish the mapping.
- Tommy: It reminds me of things we have in the implementation document. It might just be an appendix to the implementation document.
- Brian: A criterion is how much people complain about the API. Would it be possible to get stuff out of Apple's developer relations, like in discussion forums?
- Tommy: I think there's interesting stuff there and we can share that. But it'd be good to have at least one other open source implementation. We still have problems with language-specific implementation.
- Colin: We want to go through features of the API and ask if there are two different implementations that are interoperable.
- Brian: Seems like Internet Standard, not Proposed Standard.
- Colin: Maybe we should publish as Experimental first and then skip Proposed Standard and go directly to Internet Standard?
- Gorry: Yes, you can. But it doesn't make any sense, you should publish it as PS.
- Zahed: I didn't want to have this discussion again. I think you want to test the mapping and whether it is implementable, whether somebody uses it. I think it's possible to do this as part of the implementation draft.
- Zahed: Other issue: Github. There's a lot of discussion there, all authors are there. But I think some discussions need to be on the mailing list. As a Working Group we never decided our best practices. If we want application developers, Github is good. But this is for the Working Group to decide. I think we will have some discussion in Bangkok - When to use Github, when to put things on the mailing list.