Minutes interim-2020-core-04: Wed 16:00
minutes-interim-2020-core-04-202005131600-00

CoRE Virtual interim - 13-05-2020 - 14:00 UTC

Chairs:
* Marco Tiloca, RISE
* Jaime Jiménez, Ericsson

Material: https://datatracker.ietf.org/meeting/interim-2020-core-04/session/core
Webex: https://ietf.webex.com/ietf/j.php?MTID=m94be8bc01b7b3c034a5bf3106d4376ae

Minute takers: Christian, Marco and Jaime (helping)
Jabber scribes: Jaime

Participants:
1. Marco Tiloca, RISE
2. Jaime Jiménez, Ericsson
3. Barry Leiba, FutureWei
4. Jim Schaad, August Cellars
5. Jon Shallow
6. Carsten Bormann, TZI
7. Mohamed Boucadair, Orange
8. Christian Amsüss, –
9. Rikard Höglund, RISE
10. Peter Yee, AKAYLA
11. Klaus Hartke, Ericsson
12.
13.
14.
15.
16.
17.
18.
19.
20.

* New blockwise and asynchronous non-traditional responses

   - Follow-up of CoRE/DOTS discussions
       - https://mailarchive.ietf.org/arch/msg/core/H4NM0doO_ZzaHdkm5Op9UukP3-4/

Jon and Mohamed presenting

Jon Presenting draft-bosch-core-new-block

Challenges in DOTS. Network lossy due to DDOS attack. (p2) showing a use case.
Server provides mitigation service, client is in a network under attack.
Client's "I need help" is NON mitigation request, server updates client with
NONs due to full pipe where data may be lost. pipe might be overloaded due to
DDOS attack. Mitigation then kicks in and gets traffic to a usable level.
general operation for DOTS is to use CON for configuration during "peace"time,
mitigations and responses need to be NON. A mitigation request can be in a
single packet, works fine with packet losses. To make the protocol robuist
there are application heartbeats. The serve component is capable to identify
that the client is alive even with some loss and during the attack (?). Client
can carry on blind.

Extending telemtry (p4): Telemetry (eg. for when client figures out something
about the attack to help the server do better mitigation; also back from server
about how it is mitating). Block[12] works fine when no package loss is
present. With loss, there is no easy recovery and things stall, which is
unacceptable under attack.

Looking into how to handle oversized packets (p5):
    * IP fragemtnation
    * Application data break up into chuncks. Requires chunks to be full JSON.
    * Use Blockwise transfer, which has limitations on the performance.
 Performance limited due to turnaround times, and falling apart under loss.

Introducing Block[3,4] (p6). Analogous to Block[1,2] with addtitions:
    * Can send all Block3 simultaneously, also Block4 (without incrementing
    NSTART by doing NON; NSTART is unexplored territory) * Block ID. ETag not
    sure usable because "resource-level identifier" (?).

Block3 vs Block1 changes: 1 is limited to probing rate, 3 subjects the whole
body to probing rate. Both can use 4.08 responses, but the response needs to be
extended with data body including an array of missing blocks or using repeat
option with block 3.

Likewise with Block 2 vs 4, the server has to wait for next block. With Block 4
the entire set of blocks can be sent without waiting, thus better performance.
It made sense to them for caches to keep the data.

There is a challenge with tokens; token matches are not possible as there is no
longer the same request going upstream, thus how are they supposed to use the
token? Question on RFC7252, 7641, 7959. There are also OSCORE implications.

-- Discussion --

CA: Smaller items first. Etag, the etag should not impede this particular use
case as it does not apply. Jon Shallow: Under server control; ETag may be the
same for different sets of body blocks. CA: The etag should not work like that,
I wonder how the blockid would be any better anyway. JS: for each block set
back we wil have a new blockid that would be incremeting. CA: Also etag could
do that. JS: Agree. CA: On a general level, did you have the chance to check
email? JS: Saw previous one. CA: [reads email] Ok, does not apply, wrong email.
CA: My impression is that it can be done without changing the block system, by
finding concrete extensions or using other things in the pipeline. For the
Block3 (and 1) it should be straightforward if there was a response from the
server indicating which blocks are missing. JS: From your email earlier this
week. (choppy audio?) CA: For block 2 is not different than 4, the main point
ios to get the server to send the blocks in mass at the beginning. Non
traditional responses can solve that. JS: Concern was about possibly changing
the semantics of Block2, more out in one go somehow. This is a modifications
that brought up a new option. CA: I think it's an extension to the blockwise
token mechanism. JS: The server needs to know that rather than sending Block2
individually, it has to do that in blocks, this has to be negotiated. If the
client iniates the transfer using Block4, the server understands to take that
way. CA: Agrees. Two mechanisms, one is a small extension that could be enough
and that it would be usable for other purposes as well. The only thing that
remains is that we would not have a mechanism to specify specific missing
blocks. Is this something that would happen frequently? JS: We may randomly
miss a block and have to request that block, we are likely to miss more in a
flooded situation. CB: (Unrelated WebEx comment) CB: Restructure into items:
    * Congestion control -- what are we doing here, are we doing something DOTS
    specific? Do other applications have similar requirements? (Applying
    congestion control to a DDOS situation won't work well, we *need* to cut
    through excessive congestion). * What is the proper way to extend the CoAP
    model, which was not designed to be optimized for performance. Of course
    CoAP over TCP would improve the situation, but it does not apply here. We
    are designing something somewhat specific to DOTS.
"If you need performance for large objects, use CoAP over TCP" is the
traditional response. This indicates what we need here is DOTS specific.
Progress on non-traditional responses would be great when generally applicable,
maybe with some congestion control added. JS: Agrees that it is DOTS specific.
Other people might use UDP over lossy networks too. Having the ability to send
the data quickly might be a general case than just DOTS. TCP is just a no-no,
cause it can potentially give up, we can't afford the client to be chocked by a
tcp connection. CB: We have to make sure that we don't sell this to IESG as:
"if you want to ignore congesiton control, coap is what you want to use". JS:
From DOTS pov, we want to get chunks fast but the body will be subject to
congestion control. Otherwise we will be using DOTS to create a DDOS attack.
CA: That's confusing me, if DOTS can send large amount of packages, how is this
better than having no PROBING RATE? JS: You have a set of blocks, that entity
is allowed to be sent, when the enxt entity of the same size is coming that is
subject of congestion control. CA: Yes, but can't you just increase the probing
rate, if you want a buch of packets/bytes to go forward and then respect that
rate on average? probing rate is meant to followed on average not byte by byte.
JS: Yes, then that could be something to be used here cause we do not want the
DOTS protocol to be creating DDOS attacks. CB: we need to consider the two
directions separately. The one for DDoS, you need licence to cut through, not
in the TCP way. You don't want to send your packets faster than your links. The
other direction does not have DDoS to worry about, so there are incentives to
do it using congestion control even being alone on a traditional path. How do
we get feedback if the packets are lost in the other direction? JS: it's a way
for the client to detect there's a loss of traffic, again that's not in the
general CoAP, it's application specific. It could be a coap option in its own
right. CA: could be part of the telemetry that you are sending no need to have
a coap option. JS: Absolutely, it could also be done that way. We are able to
configure the different pipe sizes, and we can pass that informaiton btw client
and server, which helps mitigation processes. We do have the application layer
capable of handling that. If we had the concept of blockid for the client would
be able to know if it is missing blocks and is able to feed that back up to the
server. That would be something useful for coap too, having congestion control
information in coap. CB: that can be one component of the solution, a way for
CoAP to run with external congestion control information (e.g. from the
application). Instead we can focus on the protocol issues. CA: I like the
separation of flow control (DOTS specific) to free up from the particular
situation. KH: different perspective: Now we have CoAP over UDP/DTLS, over
TCP/TLS. We tried to keep CoAP as simple as possible and to get away without
even CONs and ACKs or congestion control over UDP; reluctantly we took them in
(at cost of additional messenging layer). For congestion control we followed
RFC on UDP usage guidelines ("if you have roundtrip times available"), and
modelled things around "low volume applications" case. For some applications,
this doesn't work (large payloads), CoAP-over-TCP has that. Idea was "if you
need more than simple retransmissions, use over-TCP". 10 years later, it's not
that much about 802.15.4, constrained devices have grown. Great to see a
protocol adapt to use cases not originally envisioned. In some cases, use cases
get far away from initial use cases to the extent of hurting a bit due to
initially designed limitations. Dilemma: Just add this one tiny thing to
support the use case, but many one-more-tiny-things get away from initial
design goal of simple protocol. Web browsers "really need this blinky text". WG
is called *constrained* restful environments, don't have that luxury.
Desparately want to make CoAP useful for more use cases, but am conflicted
about each "one more tiny thing". On this draft: Initial reaction was that if
over-UDP is too weak and over-TCP is a no-no, wouldn't it make sense to bolt
something inbetween? Selective ACKs, receive windows, DOTS-specific congestion
control etc ... ask transports area about somethign inbetween,
CoAP-over-"DOTSTP"?

JS: Part of challenge is that standard DOTS is almost at hitting final RFC
stage. It's not that easy to go back that far. We stumbled into this on
questions on telemetry for sending larger data.

CB: What Klaus said is true but this is also about adding features to CoAP that
become part of the standard protocol we recommend outside DOTS. Have some
flexibility to add options that are really specific for a use case with
suitable applicability warning ("not use in general Internet unless DOTS-like
situation"). Still interested in optimizing this to have as much to take home
from this for more global CoAP community. Should not design another transport,
keeping it simple is important as well. In the end, needs to be useful for
DOTS; if that can not be transferred 1:1 to other applications that's fine too.

MT: Have you explored in a good anough way for you the alternative usage of
non-traditional responses as they are now or as they may develop? CA: I think
the proposals are on the mail, discussing them through is not in the agenda for
today anyway. Jaime: Considering that blockwise seems to be limited and we need
Block 3 and Block 4, would be worth the while to go back to the drawing board
and update blockwise-transfer rfc? Med: on probing rate, but not related to
JJ's 'question CB: We have two things that hurt here: block is entirely
lock-step, and doing something related to observe (eliciting new blocks) is
probably useful to have. The other is the need for some form of selective
acknowledgement. Can't work with one ACK per data packet (inverse of the bitmap
which blocks made it). CA: in one direction we have selective acknowledgment
CB: how does that work? CA: NON a few blocks No-Response:2.xx, then CON to see
whether anything is missing.

CA: it will need some payload saying "I don't have all I need to process this".
CB: Need fleshed-out example.
CA: Can provide.

CB: Something like a selective acknowledgement will be needed, and we'll need a
data structure for that -- but also know how the protocol flow will make use of
this, and who sends this when and why.

KH: CoAP request-response layer is not the place to do selective
acknowledgements. CB: Agree with that from a theoretical protocol-purity PoV,
but think we should explore whether it can be done, and whether the shape of
what we get has actual problems beside being ugly. Much of the Internet is
built out of hacks like this.

CB: Seems to make sense to write up some of those flows in a litle more detail.
Couldn't quite make out of new-block what the protocol machinery behind this
would be. With more detail, we can find out whether we can solve it that way.
Med: Which details? CB: How do the two ends of the communication know what they
are supposed to do? Do we have all the protocol progress information available
so you can actually write code that does this? (And that's preferably
compatible with the rest of CoAP) JS: no coding of Block3 and Block4 yet, but
not expecting issues CB: There were a few opaque statements about tokens I
could not quite understand. JS: One needs to provide a token, had discussions
about empty token ... The token provided in Block4, should it be privded by the
client? That's why in the draft it's all on the same token. Follows how
existing CoAP-type stuff does things, client initiates request and received
back to be assimilated into a single response. CB: Extended lifetime of the
token would be a lot like Observe tokens. But when is that token finally
retired? JS: If GET does observe at same time, it is remembered for a period of
time. Client may cycle through lots of request with potential of clash, same
situation even with single-packet observe. CB: when do you really "retire" a
request? JS: All 10 results have the same token. 9 come back. Can re-request
10th or not, but after that can reuse the token. We know we're in a Block4
situation and expect more blocks to come back. CB: the request sent out to get
the rest of the blocks... JS: needs to use a different token CB: OK. Think this
can be made to work, may not be beautiful but yeah. Problem is more with the
current text that doesn't explain all the things we seem to have in mind. JS:
we'll revise the draft based on today's discussion covering points missing now
CB: we need applicability statement, congestion control info from out of CoAP,
definitions of Block3/4 options, and data structure for SACK and re-requests
Med: somethilg already in the draft, we'll make it clearer CA: I still think
this may be solved using a more generic phrasing of non-traditional responses.
If that's more complicated to do for the good of this use case, then this can
be instead considered. CB: this can be a narrowed solution for now but with a
broader scope of applicability. How much time is fine to use for this, from a
DOTS pov? JS: some weeks for the options, need to think about for the SACKs. It
does not cover sending multiple blocks of data now, need to look back at
non-traditional responses too. CA: if the client can indicate something more to
the server on what blocks it wants, that's needed. Can this be done in a way
that the proxy does not necessarily understand it? JS: sure, we'll come back on
this. CA: I would need feedback on my comments on non-traditional responses
sent to the list. CB: do you want to be a co-author? CA: happily so. CB: On
time: Will DOTS universe explode if we don't come with something by date X? Do
we have a year? Everyone wants everything ASAP, but what is a realistic
timeline? Med: Depends on whether you want to have it in base telemetry spec.
Current version targets WGLC for July. We know this is a problem we need to
solve, so far there is no normative reference to blocks we defined. Don't want
to add a dependency on telemetry spec on something I'm not sure I'll get soon.
Maybe can have update to DOTS telemetry if we're [...] continue both tracks and
know whether we go with blocks or unsolicited responses. Ideally, I want
something to integrate into DOTS telemetry. Blocks is more focused and can be
progressed into publication process faster than unsolicited. Don't have more
precise milestones. That's the current situation. We have pending issue about
large unsolicited notifications. Ideally, we'd have blocks specified.
Understand WG can take more time to take various proposals. Won't be the end of
the world, but if by July no progress declare it in as open issue in current
telemetry spec. CB: Thanks, that's very detailled, think I understand. Have
about 6 week to play Lego with various solution elements. Then should be
starting to make decisions.

MT: Ob probing rate, you had a comment
Med: When setting up, we're also negotiating a probing rate. Increasing the
default over the CoAP default, because lots overhead. [...] Not being
aggressive about that. Only in attack situation; apart from that we follow
probing rate and not bombing the network. Also one comment on CB's comment for
feedback: As mentioned by JS we have heartbeat on application level [...]. In
congestion case heartbeat will go through in a single direction, and receiver
is aware about loss due to telemetry. This is not sufficient for all
situations, but that's what we have now. CB: Probing rate negotiation is part
of what already went through IESG. So we don't have to negotiate that part.
Having heartbeats for add'l congestion control, we'd need to check whether
that's enough of a measurement to use here. For under-attack situation, will
want a little bit of information about flow in non-attacked direction. A bit
like a receive report in RTP. CB: At CoAP protocl level, we don't have to do
much, just say that there's external input that allows us to do this in a
proper way, and just make sure we stay within those limits from the external
input, just needs to be written up, probably in DOTS. Med: For next version of
draft.

CB: In current implementations, how do you pace sending those NONs? If you do
Block3/4, how do you know how fast you should send them? JS: Currently like IP
fragmented packets. [..] CB: That's probably needed if you want reasonable
delivery rate. CB: Should also be addressed. Probing rate gives you pace for
that. [...] JS: [...] average [...] CB: How big are these? JS: Up to 5 packages
in my tests. Some active data reduction done with prenegotiated mapping tables.
Not likely to go above 10. CB: Argument can be made that you can send 10
packets without paces, due to the initial-window-10 discussion. If we have a
way to ensure we stay within the ballpark, won't need much mechanism. If you
occasionally go beyond that, will still need some pacing. JS: Can do queries in
telemetry to do data reduction.

CB: When next? Was useful.
CB: Target interim in 4 weeks?

   - Related documents:
       - https://tools.ietf.org/html/draft-bosh-core-new-block-00
       - https://tools.ietf.org/html/draft-bormann-core-responses-00