Deterministic Networking (DetNet) Bounded Latency
draft-ietf-detnet-bounded-latency-10

# Internet AD comments for draft-ietf-detnet-bounded-latency-09
CC @ekline

### S2

Super-nitty, but I feel like the acronym should be PREOF, given the
expansion and the use of PREOF later on.  If it should be PROEF then
perhaps update the expanded text and the use of PREOF later on.

### S6.4.1

"an interleaved regulators" -> "an interleaved regulator"

Thank you for the work on this document.

Many thanks to Robert Sparks for his ART ART review: https://mailarchive.ietf.org/arch/msg/art/IVotuLgxpTSyiLdkNk9O1RdkrVI/ , and thanks to the authors for addressing it.

I agree with Robert (and with Murray, who has brought it up in the context of the shepherd write-up) that it is not completely clear to me why this document is Informational. This is not a major comment that should stop publication, nor it rises to the level of a blocking DISCUSS, but it might be good to have a short discussion during the telechat on its intended status, to understand its purpose and audience. Is it expected that this document will be referenced by Standard Track RFCs in the future?

Francesca

The shepherd writeup doesn't answer the question of why Informational is the right type of RFC.

Just wanted to check here with the sponsoring AD that we're OK with more authors than the guidelines recommend.

Section 2 defines "PROEF", but that term appears nowhere in the document.

Well written document. I have some comments that can be answered or ignored as the authors see fit.

   If an already-established DetNet flow would be pushed beyond its latency
   requirements by the new DetNet flow, then the new DetNet flow can be
   refused, or some other suitable action taken.

This seems like a great power and should come with great responsibility. If being a member of a DetNet
is voluntary, that seems fine. But if such membership is forced on the enduser, the concerns of RFC 8890
come into play.


[BennettDelay]

This seems to be a broken reference in section 4.2.2

   As illustrated by numerous
   implementation examples, some of the "Layer 3" mechanisms described
   in documents such as [RFC7806] are often integrated, in an
   implementation, with the "Layer 2" mechanisms also implemented in the
   same node.

But shepherds write up said:

   While there seems to be interest in this specification from multiple
   vendors, there are no publicly known implementations of this
   specification.

Which leaves me wondering what "implementation examples" refers to.


The security considerations mostly (and rightly so) point to RFC 9055 and RFC 8655.
Although those seem to mostly focus on attacks against DetNet and there is no
mention of abuse by the DetNet. For example, if a mobile phone is considered to be within
a DetNet, the enduser has no real choice but to be part of this. If a common service
like a speedtest service the user can run to evaluate their internet connection is
also within or linked to a DetNet, it can present the user a very false sense of
low latency and network speed by simply reserving an excessive amount of network between
the endusers and the speedtest service. This leads to a more generic concern of
DetNet placing us in the "user vs network" debate on who should have a say in how
packets flow, which is clearly not a problem this document needs or can address.

Thank you to Watson Ladd for the SECDIR review.

** Section 8.  It wasn’t clear to me that outright control of a resource (e.g., relay) was needed impact the latency bound calculation.  Recommending a bit more flexibility on what an attack would take:

OLD
   but may have the
   ability to control some resources within the boundary of the DetNet
   domain.

NEW

but may have the ability to control or change the behavior of some resources within the boundary of the DetNet domain.

** Section 8.

An example of such attacks is to make some
   traffic sources under the control of the attacker send more traffic
   than their assumed T-SPECs.  This type of attack is typically avoided
   by ingress conditioning at the edge of a DetNet domain

If a traffic source _in the DetNet_ is induced to send more than their T-SPEC, how does ingress conditioning at the edge help with mitigation?  This DetNet node is already in the DetNet domain.

** Section 8.  Figure 1 presented a taxonomy of delays (#1 – 6) for the timing model.  It would be helpful to explicitly extend this taxonomy to a threat analysis, or to rule out that mitigating certain threats are out of scope.  For example:

-- 1: Output delay: ?
-- 2: Link delay: somewhat covered by the discussion of timing; and violating T-SPECs.
-- 3: Frame preemption delay: ?
-- 4: Processing delay: ?
-- 5: Regulation delay: ?
-- 6: Queuing delay: covered by “Some queuing mechanisms require time synchronization and operate correctly only if the time synchronization works correctly ...”

Thanks for working on this specification, it is well written. Thanks to Yoshifumi Nishida for the TSVART review.

Thank you for the work put into this document. It was a fascinating read and most of the text is easy to understand. Good job ! Could even end up as educational material ;-)

Please find below some non-blocking COMMENT points (but replies would be appreciated even if only for my own education), and some nits.

Special thanks to:

- Lou Berger for the shepherd's write-up including the absence of implementations 

- Ralf Weber for his IETF last call INT directorate review at: https://datatracker.ietf.org/doc/review-ietf-detnet-bounded-latency-08-intdir-lc-weber-2022-02-06/. Ralf's review was acted upon by Mohammadpour Ehsan ;-)

I hope that this helps to improve the document,

Regards,

-éric

Please note that I am not a DetNet expert, hence some comments are possibly not relevant. The use of "regulator" instead of "shaper" was also surprising to me ;-) There may even be a semantic difference between a shaper and a regulator that I have to learn.

More generally, the document is about a bound on the latency, but would it be possible to compute an aggregate latency curve, which could also be interesting?

## Abstract

The document goes beyond "it provides a methodology to compute end-to-end latency" as it describes how admission control is important and gives formulas to compute the latency bounds in many technologies. The content is great but the abstract should reflect the actual content.

## Section 1

"DetNet flows are characterized by" is followed by only 2 characteristics of DetNet and does not mention low packet loss. Even if this I-D is about latency, why not adding "notably" in the text ?

## Section 2

As noted by other ADs, either the acronym "PROEF" or its expansion is wrong ;-)

## Section 3.1.1

Perhaps some examples of "static latency" could be added to ease the reading ? E.g., does it include serialisation and propagation times ?

Have the authors considered the use of "constant" rather than "static" ?


## Section 3.2

Does DetNet support some link having some flow control / congestion at layer-2 or layer-1 ?

Are there any DetNet relay able to start forwarding packet after receiving just a couple of bytes of the frame w/o having to wait for the full frame reception ? Similarly, should the deserialisation delay be taken into account ? Or any congestion inside a relay node (e.g., contention to a backplane bus)

## Section 4.1

Is "end-to-end delay" defined in DetNet architecture ? I.e., is it from first bit out from the source to the last bit received by the destination ? Or other variation ? Or the difference is so small that it does not care ?

## Section 4.4

Sorry couldn't resist in suggesting to rename "DetNet-unaware island" into "DetNet-unaware swamp" ;-) (comment to be ignored of course)

## Section 5

Should some assumptions be listed ? E.g., no packet replication/generation/encapsulation/discard by the relay node ?

## Section 6.1

Is the list at the end of this section exhaustive ? If so, then why not adding "IP protocol (e.g., UDP or TCP)" ? Else, please let's be clear that it is not exhaustive.


# NITS  

AFAIK, XML2RFCv3 has features for mathematical formulas that could be used for this document to increase readability.

Usually, "e.g." is followed by a coma. Same for "i.e.".

## Section 1

Multiple lists in the section would benefit from a correct markup to put one bullet per line.

In "It disregards the in-band packets", and accept all my apologies for not being a native English speaker, but "disregards" sounds really negative.

## Section 6.3

s/Time Aware Shaper/Time-Aware Shaper/ ?

Section 11.2. , paragraph 7, comment:
>    [IEEE8021Qcr]
>               IEEE 802.1, "IEEE P802.1Qcr: IEEE Draft Standard for Local
>               and metropolitan area networks - Bridges and Bridged
>               Networks - Amendment: Asynchronous Traffic Shaping", 2017,
>               <http://www.ieee802.org/1/files/private/cr-drafts/>.

Please link to a public version of this document, if possible.

The document has six authors, which exceeds the recommended author limit. I
assume the sponsoring AD has agreed that this is appropriate?

Thanks to Gyan S. Mishra for their General Area Review Team (Gen-ART) review
(https://mailarchive.ietf.org/arch/msg/gen-art/yMbUVrUjPZYqHTfNd8XwqMwg0FI).

-------------------------------------------------------------------------------
NIT
-------------------------------------------------------------------------------
All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via https://github.com/larseggert/ietf-reviewtool), so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

Document references draft-ietf-detnet-controller-plane-framework-00, but -01 is
the latest available revision.

These URLs in the document did not return content:
 * http://www.ieee802.org/1/files/private/cr-drafts/

These URLs in the document can probably be converted to HTTPS:
 * http://ieeexplore.ieee.org/document/8403927
 * http://www.ieee802.org/1/

Section 6.4. , paragraph 6, nit:
> cussed in Section 4.2.2; an interleaved regulators does not increase the dela
>                          ^^^^^^^^^^^^^^^^^^^^^^^^^
The plural noun "regulators" cannot be used with the article "an". Did you mean
"an interleaved regulator" or "interleaved regulators"?

Section 6.4.2. , paragraph 9, nit:
> T, CQF with more buffers can be used and a cycle identification label can be
>                                     ^^^^
Use a comma before "and" if it connects two independent clauses (unless they
are closely connected and short).

Section 6.6. , paragraph 3, nit:
> to entrance of the first node in sub-network 2, and d3_p the delay bound of p
>                                  ^^^^^^^^^^^
This word is normally spelled as one. (Also elsewhere.)

I don't understand this sentence in the introduction:

"It disregards the in-band packets that can be part of the stream such as OAM and necessary re-transmissions"

Are you referring to retransmissions of user data? If so, that sounds like an important consideration for latency bounds!

Thanks to Yoshi for the TSVART review.

Nits:

(2) s/PROEF/PREOF

(6.6) s/is the same cycle/in the same cycle (or maybe I just don't understand this sentence?)

Thanks for this document.  I found it to be a pleasant read and informative.

Regards,
Rob