Block-Wise Transfers in the Constrained Application Protocol (CoAP)
draft-ietf-core-block-21

[Ballot comment]
Remaining issues from my AD review: :

In 3.2:

  A stateless server that simply builds/updates the resource in place
  (statelessly) may indicate this by not setting the more bit in the
  response (Figure 8); in this case, the response codes are valid
  separately for each block being updated.

What is the behaviour of both the client and the server if PUT on a particular block fails? Is this clear enough in the document?

[Ballot comment]
Remaining issues from my AD review: :

In 3.2:

  A stateless server that simply builds/updates the resource in place
  (statelessly) may indicate this by not setting the more bit in the
  response (Figure 8); in this case, the response codes are valid
  separately for each block being updated.

What is the behaviour of both the client and the server if PUT on a particular block fails? Is this clear enough in the document?

Other questions I have after reviewing the document. If I missed where they are answered, please point me out to existing text in the document or another RFC:

Is there a special error for block size mismatch between multiple requests? (Possibly no.)

As block2 is a critical option, how can a server know if a particular client supports this option?

[Ballot comment]
Remaining issues from my AD review: :

2.10:
  A response with a Block1 Option in control usage
  with the M bit set invalidates cached responses for the target URI.

Can you explain the reason for this?

In 3.2:

  A stateless server that simply builds/updates the resource in place
  (statelessly) may indicate this by not setting the more bit in the
  response (Figure 8); in this case, the response codes are valid
  separately for each block being updated.

What is the behaviour of both the client and the server if PUT on a particular block fails? Is this clear enough in the document?

Other questions I have after reviewing the document. If I missed where they are answered, please point me out to existing text in the document or another RFC:

Is there a special error for block size mismatch between multiple requests?

As block2 is a critical option, how can a server know if a particular client supports this option?

[Ballot comment]
Thanks for adding new text at the beginning of the doc. This addresses my previous concern and makes it usage more clear from my point of view. Sorry for my rather long delay!

[Ballot comment]
I moved the following from a DISCUSS to a COMMENT, pending the next revision:

-- START:
This should be easy to clean up, and it's entirely possible I am misreading something. But Section 3.4, 2nd and 3rd paragraph seem to be inconsistent in SHOULD vs MUST for block size.  I _think_ I'm reading the following:

1. If the client requests a specific block size, the server MUST use that size or a smaller one.

2. Any subsequent requests from the client MUST indicate the same size that the server used

3. But paragraph 3 then says all further requests SHOULD indicate the same size. But if they already MUST indicate the same size as in the initial response, then this  SHOULD seems non-constraining at best, and at worst in conflict with 1. (ignoring the last-block size issue for the moment.)

4. Then paragraph 3 goes on to say the server SHOULD use the block size indicated in the request or smaller. This seems to conflict with the MUST in 1.

5.  Then it again asserts that the client MUST indicate the same size in subsequent requests, conflicting with the SHOULD in 3., but agreeing with the MUST in 1.

-- END


Substantive:

- General: The draft dives into detail without giving much context until the examples. The reader is left to infer the forest from the trees. It would be very helpful (to me, at least) to add a high-level overview of operation early in the document, including definitions for "descriptive" vs " control" usages. (I know it's late in the process to ask for a whole new section, so I won't push the point.)

- The distinction between the option names Block1 and Block2 seems almost actively non-mnemonic. Is that intentional? (I won't push this point, either.)

- 3.4: Does the eTag apply to the "payload" or the whole "body"?

- 2.5, 2nd paragraph, last sentence:
Should I read this to mean that the reduction in block size is retroactive? That could use some elaboration. (I thought I read comments in the examples suggesting such a reduction would _not_ be retroactive).

- 7: Does "object security" apply to the "payload", or the "body"?
Can you describe or add a reference for the "usual considerations"?

Editorial:

- Abstract: That’s a really long abstract, and serves more as an introduction than an abstract. Please consider combining the first and last paragraph and leaving the rest to the introduction.

- 2.4, paragraph 5: a definition for "reassembler" would be helpful.

- Figures 5 and 6:
There's no discussion of either figure. Is that intentional?

[Ballot comment]
  CoAP is based on datagram transports such as UDP, which limit the
  maximum size of resource representations that can be transferred
  without creating unreasonable levels of IP fragmentation.

CoAP is based on UDP, right? So isn't it?
NEW:
  CoAP is based on UDP datagrams, which limit the
  maximum size of resource representations that can be transferred
  without creating unreasonable levels of IP fragmentation.

And ...

OLD:
  Using fragmentation (either at the adaptation layer or at
  the IP layer) for the transport of larger representations would be
  possible up to the maximum size of the underlying datagram protocol
  (such as UDP),

NEW:
  Using fragmentation (either at the adaptation layer or at
  the IP layer) for the transport of larger representations would be
  possible up to the maximum size of the underlying datagram protocol
  (UDP),

Note that they might exist other instances.

[Ballot comment]
  CoAP is based on datagram transports such as UDP, which limit the
  maximum size of resource representations that can be transferred
  without creating unreasonable levels of IP fragmentation.

CoAP is based on UDP, right? So isn't it?
NEW:
  CoAP is based on UDP datagrams, which limit the
  maximum size of resource representations that can be transferred
  without creating unreasonable levels of IP fragmentation.

And ...

OLD:
  Using fragmentation (either at the adaptation layer or at
  the IP layer) for the transport of larger representations would be
  possible up to the maximum size of the underlying datagram protocol
  (such as UDP),

NEW:
  Using fragmentation (either at the adaptation layer or at
  the IP layer) for the transport of larger representations would be
  possible up to the maximum size of the underlying datagram protocol
  (UDP),

Note that they are maybe other instances.

[Ballot discuss]
This is only a minor point, requesting to spell out implicit assumptions explicitly. However, I think it's important to address this before publication.

It is not clear in the main part of the doc that this extension to does not change the message transmission method as specified in RFC7252 (Stop-and-wait retransmission). With my initial ready I assumed that this extension would allow the sending of back-to-back messages. Only by looking at the examples, it got clear to me that this is not the case.

Further, this document does not say anything about reliability. Do block message need to be transmitted reliable (as Confirmable)? If not, this extension could still lead to back-to-back sending and then further guidance on congestion control would be needed.

[Ballot comment]
I agree with others that reduncy makes the doc harder to read, especially regarding SHOULDs and MUSTs. Would it be helpful to have all SHOULDs and MUST in one section and combine the Usage guidance with the examples?

Further, please also add a reference for ETag in section 2.4.

[Ballot comment]
Is there a specific reason that the 4.08 response code is overloaded for use for two different kinds of issues?

a) Mismatching Content-Format options in different blocks
b)  not all previous blocks are available at the server at the time of processing the final block of an atomic operation

Section 7.1:

Have you thought about how this text can be made actionable, especially in the case of atomic PUT or POST requests without maintaining state? If so, it would be good to elaborate here.

"Wherever possible, servers should therefore minimize the opportunities to create state for untrusted sources, e.g. by using stateless approaches."

[Ballot comment]


The intro and early section 2 has quite a bit of argument
as to why this design is good or better than some other.
For this reader, that's not really needed (I assume the WG
had those discussions). I think this is an example of a
recurring problem with the text here - with too many words,
clarity suffers. For example the Implementation note on p7
is, just by itself, the best and would be a sufficient
explanation of, the encoding, the description of which is
otherwise pretty opaque. There's also a good bit of
repetition that makes this a harder read in general. That's
all just IMO of course, and there may be history in the WG
that provides good cause for so many words to be needed.
All that said, I assume that it's too late to reduce the
size of this document, so no action is required unless the
authors/WG/chairs would themselves like to try remove
words.

[Ballot discuss]
This should be easy to clean up, and it's entirely possible I am misreading something. But Section 3.4, 2nd and 3rd paragraph seem to be inconsistent in SHOULD vs MUST for block size.  I _think_ I'm reading the following:

1. If the client requests a specific block size, the server MUST use that size or a smaller one.

2. Any subsequent requests from the client MUST indicate the same size that the server used

3. But paragraph 3 then says all further requests SHOULD indicate the same size. But if they already MUST indicate the same size as in the initial response, then this  SHOULD seems non-constraining at best, and at worst in conflict with 1. (ignoring the last-block size issue for the moment.)

4. Then paragraph 3 goes on to say the server SHOULD use the block size indicated in the request or smaller. This seems to conflict with the MUST in 1.

5.  Then it again asserts that the client MUST indicate the same size in subsequent requests, conflicting with the SHOULD in 3., but agreeing with the MUST in 1.

[Ballot discuss]
This should be easy to clean up, and it's entirely possible I am misreading something. But Section 3.4, 2nd and 3rd paragraph seem to be inconsistent in SHOULD vs MUST for block size.  I _think_ I'm reading the following:

1. If the client requests a specific block size, the server MUST use that size or a smaller one.
2. Any subsequent requests from the client MUST indicate the same size that the server used
3. But paragraph 3 then says all further requests SHOULD indicate the same size. But if they already MUST indicate the same size as in the initial response, then this  SHOULD seems non-constraining at best, and at worst in conflict with 1. (ignoring the last-block size issue for the moment.)
4. Then paragraph 3 goes on to say the server SHOULD use the block size indicated in the request or smaller. This seems to conflict with the MUST in 1.
5.  Then it again asserts that the client MUST indicate the same size in subsequent requests, conflicting with the SHOULD in 3., but agreeing with the MUST in 1.

[Ballot discuss]
This should be easy to clean up, and it's entirely possible I am misreading something. But Section 3.4, 2nd and 3rd paragraph seem to be inconsistent in SHOULD vs MUST for block size.  I _think_ I'm reading the following:

1. If the client requests a specific block size, the server MUST use that size or a smaller one.
2. Any subsequent requests from the client MUST indicate the same size that the server used
3. But paragraph 3 then says all further requests SHOULD indicate the same size. But if they already MUST indicate the same size as in the initial response, then this SHOULD seems non-constraining at best, and at worst in conflict with 1. (ignoring the last-block size issue for the moment.)
4. Then paragraph 3 goes on to say the server SHOULD use the block size indicated in the request or smaller. This seems to conflict with the MUST in 1.
5.  Then it again asserts that the client MUST indicate the same size in subsequent requests, conflicting with the SHOULD in 3., but agreeing with the MUST in 1.

[Ballot comment]
Substantive:

- General: The draft dives into detail without giving much context until the examples. The reader is left to infer the forest from the trees. It would be very helpful (to me, at least) to add a high-level overview of operation early in the document, including definitions for "descriptive" vs " control" usages. (I know it's late in the process to ask for a whole new section, so I won't push the point.)

- The distinction between the option names Block1 and Block2 seems almost actively non-mnemonic. Is that intentional? (I won't push this point, either.)

- 3.4: Does the eTag apply to the "payload" or the whole "body"?

- 2.5, 2nd paragraph, last sentence:
Should I read this to mean that the reduction in block size is retroactive? That could use some elaboration. (I thought I read comments in the examples suggesting such a reduction would _not_ be retroactive).

- 7: Does "object security" apply to the "payload", or the "body"?
Can you describe or add a reference for the "usual considerations"?

Editorial:

- Abstract: That’s a really long abstract, and serves more as an introduction than an abstract. Please consider combining the first and last paragraph and leaving the rest to the introduction.

- 2.4, paragraph 5: a definition for "reassembler" would be helpful.

- Figures 5 and 6:
There's no discussion of either figure. Is that intentional?

[Ballot comment]
Please consider whether you need to say more about UDP usage for this extension than what the base CORE specification says - RFC 7252 has only one mention of RFC 5405, and that only points to guidance about reducing ACK_TIMEOUT below one second. I understand that the CoAP story includes "most of these nodes aren't capable of generating a lot of packets in a short timeframe", but does this extension make it easier to send multiple UDP packets back-to-back?

I'm reading this text,

  In most cases, all blocks being transferred for a body (except for
  the last one) will be of the same size. 

and then this text

      *  The block size implied by SZX MUST match the size of the
        payload in bytes, if the M bit is set.  (SZX does not govern
        the payload size if M is unset).  For Block2, if the request
        suggested a larger value of SZX, the next request MUST move SZX
        down to the size given in the response.  (The effect is that,
        if the server uses the smaller of (1) its preferred block size
        and (2) the block size requested, all blocks for a body use the
        same block size.)
       
and realizing that I'm confused about why all the blocks for a body might NOT use the same block size. Maybe this doesn't matter (because an implementation would need to be prepared for the case when all the blocks don't use the same block size)?

The examples were helpful to me. Thank you for doing that work.

(Via drafts-lastcall-comment@iana.org): IESG/Authors/WG Chairs:

IANA has completed its review of draft-ietf-core-block-18.txt. If any part of this review is inaccurate, please let us know.

IANA understands that, upon approval of this document, there are two actions which IANA must complete.

First, in the CoAP Option Numbers subregistry of the Constrained RESTful Environments (CoRE) Parameters registry located at:

http://www.iana.org/assignments/core-parameters/

three new option numbers are to be registered as follows:

Number: 23
Name: Block2
Reference: [ RFC-to-be ]

Number: 27
Name: Block1
Reference: [ RFC-to-be ]

Number: 28
Name: Size2
Reference: [ RFC-to-be ]

Second, in the CoAP Response Codes subregistry of the Constrained RESTful Environments (CoRE) Parameters registry also located at:

http://www.iana.org/assignments/core-parameters/

two new response codes are to be registered as follows:

Code: 2.31
Description: Continue
Reference: [ RFC-to-be ]

Code: 4.08
Description: Request Entity Incomplete
Reference: [ RFC-to-be ]

IANA understands that these two actions are the only ones required to be completed upon approval of this document.

Note:  The actions requested in this document will not be completed until the document has been approved for publication as an RFC. This message is only to confirm what actions will be performed. 

Thank you,

Sabrina Tanamal
IANA Specialist
ICANN

The following Last Call announcement was sent out:

From: The IESG
To: "IETF-Announce"
CC: draft-ietf-core-block@ietf.org, kovatsch@inf.ethz.ch, core-chairs@ietf.org, core@ietf.org, barryleiba@gmail.com
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Block-wise transfers in CoAP) to Proposed Standard


The IESG has received a request from the Constrained RESTful Environments
WG (core) to consider the following document:
- 'Block-wise transfers in CoAP'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2015-12-04. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  CoAP is a RESTful transfer protocol for constrained nodes and
  networks.  Basic CoAP messages work well for the small payloads we
  expect from temperature sensors, light switches, and similar
  building-automation devices.  Occasionally, however, applications
  will need to transfer larger payloads -- for instance, for firmware
  updates.  With HTTP, TCP does the grunt work of slicing large
  payloads up into multiple packets and ensuring that they all arrive
  and are handled in the right order.

  CoAP is based on datagram transports such as UDP or DTLS, which
  limits the maximum size of resource representations that can be
  transferred without too much fragmentation.  Although UDP supports
  larger payloads through IP fragmentation, it is limited to 64 KiB
  and, more importantly, doesn't really work well for constrained
  applications and networks.

  Instead of relying on IP fragmentation, this specification extends
  basic CoAP with a pair of "Block" options, for transferring multiple
  blocks of information from a resource representation in multiple
  request-response pairs.  In many important cases, the Block options
  enable a server to be truly stateless: the server can handle each
  block transfer separately, with no need for a connection setup or
  other server-side memory of previous block transfers.

  In summary, the Block options provide a minimal way to transfer
  larger representations in a block-wise fashion.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-core-block/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-core-block/ballot/


No IPR declarations have been submitted directly on this I-D.

1. Summary

This specification extends the CoAP protocol (RFC 7252) with a pair of
"Block" options to enable the transport of large payloads through
multiple request-response pairs. In constrained environments, these
blockwise transfers have an advantage over fragmentation at lower layers
(e.g., IP), since the server can be truly stateless with no need for a
connection setup or other server-side memory of previous block
transfers. The specification is intended for standards-track, as it is a
widely implemented extension of a standards-track protocol.

Matthias Kovatsch is the document shepherd.
Barry Leiba is the responsible Area Director.

2. Review and Consensus

The specification was developed along with the base CoAP specification
and received significant review by active WG members and implementers.
There were two major discussions while finding concensus: One concerned
an interoperability issue that occured in early test events was tracked
down to the interaction between the Block options and the Token. The
issue was fixed in version -08 and editorial clarification added in -16.
The other discussion circled around the possible combinations of the
Block1 and Block2 options as well as the combination of Block2 and
Observe. The proposal for server-side "Initiative," where the server is
responsible for managing the transfer of Block2 messages after a
block-wise request (Block1) or a notification (Observe), was dropped in
favor of normal client-driven transfers with a clear definition of the
message sequence. Version -13 added the corresponding text and
sufficient examples for implementers.

Several implementations of this final version have completed an
interoperability test event with a very good score. Further implementer
feedback also was positive.

3. Intellectual Property

There is no IPR disclosure referencing this document.  The authors have
indicated recently that they are not aware of patent claims relating to
this document.

4. Other Points

This document references draft-ietf-core-observe, which is currently in
the RFC Editor queue. The intention is to define the combination of the
Block and Observe options solely in this document,
draft-ietf-core-block.

There is currently a discussion in the WG to use one of the reserved
values in the Block option to provide an optimized mechanism for
reliable, stream-based transports such as TCP. This change will have no
impact on datagram-based transports, which is the target of this
document.

1. Summary

This specification extends the CoAP protocol (RFC 7252) with a pair of "Block" options to enable the transport of large payloads through multiple request-response pairs. In constrained environments, these blockwise transfers have an advantage over fragmentation at lower layers (e.g., IP), since the server can be truly stateless with no need for a connection setup or other server-side memory of previous block transfers. The specification is intended for standards-track, as it is a widely implemented extension of a standards-track protocol.

Matthias Kovatsch is the document shepherd.
Barry Leiba is the responsible Area Director.

2. Review and Consensus

The specification was developed along with the base CoAP specification and received significant review by active WG members and implementers. There were two major discussions while finding concensus: One concerned an interoperability issue that occured in early test events was tracked down to the interaction between the Block options and the Token. The issue was fixed in version -08 and editorial clarification added in -16. The other discussion circled around the possible combinations of the Block1 and Block2 options as well as the combination of Block2 and Observe. The proposal for server-side "Initiative," where the server is responsible for managing the transfer of Block2 messages after a block-wise request (Block1) or a notification (Observe), was dropped in favor of normal client-driven transfers with a clear definition of the message sequence. Version -13 added the corresponding text and sufficient examples for implementers.

Several implementations of this final version have completed an interoperability test event with a very good score. Further implementer feedback also was positive.

3. Intellectual Property

There is no IPR disclosure referencing this document.  The authors have indicated recently that they are not aware of patent claims relating to
this document.

4. Other Points

This document references draft-ietf-core-observe, which is currently in the RFC Editor queue. The intention is to define the combination of the Block and Observe options solely in this document, draft-ietf-core-block.

There is currently a discussion in the WG to use one of the reserved values in the Block option to provide an optimized mechanism for reliable, stream-based transports such as TCP. This change will have no impact on datagram-based transports, which is the target of this document.

1. Summary

This specification extends the CoAP protocol (RFC 7252) with a pair of "Block" options to enable the transport of large payloads through multiple request-response pairs. In constrained environments, these blockwise transfers have an advantage over fragmentation at lower layers (e.g., IP), since the server can be truly stateless with no need for a connection setup or other server-side memory of previous block transfers. The specification is intended for standards-track, as it is a widely implemented extension of a standards-track protocol.

Matthias Kovatsch is the document shepherd.
Barry Leiba is the responsible Area Director.

2. Review and Consensus

The specification was developed along with the base CoAP specification and received significant review by active WG members and implementers. There were two major discussions while finding concensus: One concerned an interoperability issue that occured in early test events was tracked down to the interaction between the Block options and the Token. The issue was fixed in version -08 and editorial clarification added in -16. The other discussion circled around the possible combinations of the Block1 and Block2 options as well as the combination of Block2 and Observe. The proposal for server-side "Initiative," where the server is responsible for managing the transfer of Block2 messages after a block-wise request (Block1) or a notification (Observe), was dropped in favor of normal client-driven transfers with a clear definition of the message sequence. Version -13 added the corresponding text and sufficient examples for implementers.

Several implementations of this final version have completed an interoperability test event with a very good score. Further implementer feedback also was positive.

3. Intellectual Property

There is no IPR disclosure referencing this document.  The authors have indicated recently that they are not aware of patent claims relating to
this document.

4. Other Points

This document references draft-ietf-core-observe, which is currently in the RFC Editor queue. The intention is to define the combination of the Block and Observe options solely in this document, draft-ietf-core-block.

There is currently a discussion in the WG to use one of the reserved values in the Block option to provide an optimized mechanism for reliable, stream-based transports such as TCP. This change will have no impact on datagram-based transports, which is the target of this document.

1. Summary

This specification extends the CoAP protocol (RFC 7252) with a pair of "Block" options to enable the transport of large payloads through multiple request-response pairs. In constrained environments, these blockwise transfers have an advantage over fragmentation at lower layers (e.g., IP), since the server can be truly stateless with no need for a connection setup or other server-side memory of previous block transfers. The specification is intended for standards-track, as it is a widely implemented extension of a standards-track protocol.

Matthias Kovatsch is the document shepherd.
Barry Leiba is the responsible Area Director.

2. Review and Consensus

The specification was developed along with the base CoAP specification and received significant review by active WG members and implementers. There were two major discussions while finding concensus: One concerned an interoperability issue that occured in early test events was tracked down to the interaction between the Block options and the Token. The issue was fixed in version -08 and editorial clarification added in -16. The other discussion circled around the possible combinations of the Block1 and Block2 options as well as the combination of Block2 and Observe. The proposal for server-side "Initiative," where the server is responsible for managing the transfer of Block2 messages after a block-wise request (Block1) or a notification (Observe), was dropped in favor of normal client-driven transfers with a clear definition of the message sequence. Version -13 added the corresponding text and sufficient examples for implementers.

Several implementations of this final version have completed an interoperability test event with a very good score. Further implementer feedback also was positive.

3. Intellectual Property

There is no IPR disclosure referencing this document.  The authors have indicated recently that they are not aware of patent claims relating to
this document.

4. Other Points

This document references draft-ietf-core-observe, which is currently in the RFC Editor queue.