Explicit Host-to-Network Flow Measurements Techniques
draft-ietf-ippm-explicit-flow-measurements-07

Request for posting confirmation emailed to previous authors: Alexandre Ferrieux , Fabio Bulgarella , Giuseppe Fioccola , Igor Lubashev , Isabelle Hamchaoui , Massimo Nilo , Mauro Cociglio , Riccardo Sisto

Request for posting confirmation emailed to previous authors: Alexandre Ferrieux , Fabio Bulgarella , Giuseppe Fioccola , Igor Lubashev , Isabelle Hamchaoui , Massimo Nilo , Mauro Cociglio , Riccardo Sisto

Request for posting confirmation emailed to previous authors: Alexandre Ferrieux , Fabio Bulgarella , Giuseppe Fioccola , Igor Lubashev , Isabelle Hamchaoui , Massimo Nilo , Mauro Cociglio , Riccardo Sisto

Request for posting confirmation emailed to previous authors: Alexandre Ferrieux , Fabio Bulgarella , Giuseppe Fioccola , Igor Lubashev , Isabelle Hamchaoui , Massimo Nilo , Mauro Cociglio , Riccardo Sisto

[Ballot comment]
# Éric Vyncke, INT AD, comments for draft-ietf-ippm-explicit-flow-measurements-03

Thank you for the work put into this document.

Please find below some non-blocking COMMENT points (but replies would be appreciated even if only for my own education).

I strongly support Jim Guichard's original DISCUSS point on squatting on bits that may be used by the network. I read the follow-up email conversation and then I now wonder what it the point of publishing this I-D as a RFC *in the IETF stream* (as opposed to the *IRTF stream*) to describe a potential plan. Hence, my current ABSTAIN ballot about the stream selection.

Other thanks to Pascal Thubert, the Internet directorate reviewer (at my request), please consider this int-dir review:
https://datatracker.ietf.org/doc/review-ietf-ippm-explicit-flow-measurements-03-intdir-telechat-thubert-2023-05-11/ (and I have seen the email exchange with Giuseppe)

Special thanks to Marcus Ihlar for the shepherd's detailed write-up including the WG consensus and the justification of the intended status (and the explanation for 8 authors)

I hope that this review helps to improve the document and possibly trigger a change of stream,

Regards,

-éric


# COMMENTS

## Section 1

`can be prevented because of the encrypted transport-layer headers (e.g. QUIC, TCP)` since when TCP is encrypted ? ;-)

## Section 8

While it is not really related to privacy, if there is some experimental traffic over an ISP network with those bit sets, the ISP could put this marked traffic in a priority queue to deliver a better service and so 'cheat' on an experiment that would benchmark ISP (e.g., speedtest.net and others).

[Ballot comment]
# GEN AD review of draft-ietf-ippm-explicit-flow-measurements-03

CC @larseggert

## Comments

### Too many authors

The document has eight authors, which exceeds the recommended author limit. Has
the sponsoring AD agreed that this is appropriate?

### Inclusive language

Found terminology that should be reviewed for inclusivity; see
https://www.rfc-editor.org/part2/#inclusive_language for background and more
guidance:

* Term `crippling`; alternatives might be `broken`, `damaged`, `defective`,
  `deformed`, `impaired`

## Nits

All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via https://github.com/larseggert/ietf-reviewtool), so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

### Outdated references

Document references `draft-ietf-tcpm-accurate-ecn-23`, but `-24` is the latest
available revision.

Document references `draft-ietf-tsvwg-udp-options-19`, but `-20` is the latest
available revision.

### Grammar/style

#### "Table of Contents", paragraph 1
```
headers (e.g. TCP sequence and acknowledgment numbers) to allow for quantitat
                              ^^^^^^^^^^^^^^
```
Do not mix variants of the same word ("acknowledgment" and "acknowledgement")
within a single text.

#### Section 1, paragraph 2
```
ination. Each technique uses a small number of bits and exposes a specific m
                            ^^^^^^^^^^^^^^^^^
```
Specify a number, remove phrase, use "a few", or use "some".

#### Section 1, paragraph 2
```
this document proposes adding a small number of dedicated measurement bits to
                              ^^^^^^^^^^^^^^^^^
```
Specify a number, remove phrase, use "a few", or use "some".

#### Section 2.1, paragraph 7
```
on. The delay sample lifetime is comprised of two phases: initialization and
                              ^^^^^^^^^^^^^^^
```
Did you mean "comprises" or "consists of" or "is composed of"?

#### Section 2.2.1, paragraph 5
```
used to produce a delay measure. Therefore the value of T_Max must also be k
                                ^^^^^^^^^
```
A comma may be missing after the conjunctive/linking adverb "Therefore".


#### Section 3.3.2.2, paragraph 1
```
Therefore, recommendations in section Section 3.2.3.1 apply equally to impro
                              ^^^^^^^^^^^^^^^
```
Possible typo: you repeated a word.

#### Section 3.4.3.1, paragraph 1
```
unting packets in this direction with a E bit equal to 1. The upstream CE-ma
                                      ^
```
Use "an" instead of "a" if the following word starts with a vowel sound, e.g.
"an article", "an hour".

#### Section 3.4.3.2, paragraph 4
```
header, the available metrics using an unidirectional or bidirectional obse
                                    ^^
```
Use "a" instead of "an" if the following word doesn't start with a vowel sound,
e.g. "a sentence", "a university".

#### Section 3.4.3.4, paragraph 6
```
4 bits, i.e. QL or QR + SD, allow to have more complete and resilient measur
                                  ^^^^^^^
```
Did you mean "having"? Or maybe you should add a pronoun? In active voice,
"allow" + "to" takes an object, usually a pronoun.

#### Section 3.5, paragraph 1
```
various situations. The choice of the the methods also depends on the specif
                                  ^^^^^^^
```
Possible typo: you repeated a word.

## Notes

This review is in the ["IETF Comments" Markdown format][ICMF], You can use the
[`ietf-comments` tool][ICT] to automatically convert this review into
individual GitHub issues. Review generated by the [`ietf-reviewtool`][IRT].

[ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md
[ICT]: https://github.com/mnot/ietf-comments
[IRT]: https://github.com/larseggert/ietf-reviewtool

[Ballot comment]
# John Scudder, RTG AD, comments for draft-ietf-ippm-explicit-flow-measurements-03
CC @jgscudder

Thanks for this document. While I am not sufficiently expert in the subject area to give it a deep review, I trust this has been done by others. I do have a few questions and comments that I hope may help, below.

I support Jim Guichard's DISCUSS position.

## COMMENT

### Section 1

  the encrypted transport-layer headers (e.g.  QUIC, TCP).

Since when are TCP headers encrypted?

### Section 3.1

There is a list of bullet items that talk about "by setting the T bit", e.g.

  *  the client selects, generates and consequently transmits a first
      train of packets, by setting the T bit to 1;

The way the sentence is constructed, it literally means "the client causes the train of packets to be generated and transmitted by means of setting the T bit to 1". I doubt that's what you mean. Possibly you mean something like "it sets the T bit to 1 to identify packets in this train"?

### Section 3.1.2

The reflection counter is first introduced by mentioning that

                                                            the
      reflection counter is unlocked to start counting incoming marked
      packets that will be reflected later;
     
This makes for a rather bumpy experience for the first-time reader. I suggest introducing this counter somehow before referencing it. You might also mention that as part of initialization, the reflection counter is locked.

I also have a hard time being sure I've understood this correctly:

  The generation token counter should be capped to limit the effects of
  a subsequent sudden reduction in the other endpoint's packet rate
  that could prevent that endpoint from reflecting collected packets.
  The most conservative cap value is 1.

I presume "capped" is used in the normal sense of not being allowed to exceed a certain value. I guess yes, 1 is a conservative value in a way, after all the only lower value is zero and that wouldn't make a lot of sense. As written the text has the odor of recommending the value of 1 without actually doing so, it's just that "conservative" sounds like approval in this context. Is all of that intentional?

### Section 3.3

Surely "IP/IPv6" is the wrong terminology -- should be either "IP" (meaning, IPv4 or IPv6) or "IPv4/IPv6"?

### Section 3.3

Concerning the Unreported Loss counter you have,

                                    If the protocol is able to rescind
  the loss determination later, a positive Unreported Loss counter may
  be decremented due to the rescission, but it should not become
  negative due to the rescission.
 
and later,

                                                        (so Unreported
  Loss counter may become negative when a packet with L=1 is sent after
  a partial packet has been lost)
 
Is my conclusion correct, that negative values are permitted and should be supported, but only in the latter case and not the former?

## NITS

### Section 3.5, Section 4

s/this draft/this document/

### Section 7

s/gleamed/gleaned/

## Notes

This review is in the ["IETF Comments" Markdown format][ICMF], You can use the
[`ietf-comments` tool][ICT] to automatically convert this review into
individual GitHub issues.

[ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md
[ICT]: https://github.com/mnot/ietf-comments

[Ballot discuss]
Thanks for working on this specification. I hope it will be helpful for the valid network observer who does the flow measurement (given that the end-points actually implements the markings).

Thanks to Colin Perkins for the TSVART review (https://mailarchive.ietf.org/arch/msg/ippm/OMrRG_0CG8uRHVz0o6ivqRD8T6g/ ) and also Lucas Pardue (https://mailarchive.ietf.org/arch/msg/ippm/RgtxAHmJfANjlfPkn1Jb4Hbbcs8/ ) for his review of the document. Both reviews had led to changes in the document which should improve and clarify the specification even more.

As I agree with both the reviewers , even though I have already see some resolutions, I am holding this discuss to make sure agreed resolutions are landed in the document we approve.

[Ballot comment]
Besides, I have following comments that I believe will improve the document if addressed -

  - Please define the collaborative endpoints in the context of this document. At least set the exceptions on the client and server to comply with this specification.

  - It would great if we can add a separate section or amend the introduction to explain why the loss and delay is important to measure in the network to provide QoS. A list of use cases would be very motivating .

  - It would also be very useful to describe why "the accurate measurement of packet loss and delay experienced by encrypted transport-layer protocols is highly desired". I believe the information about the need is available and would help the reader and implementer of this specification to understand the need.

  - It says -
   
      Accurate loss and delay information is not critical to the operation of any protocol, though its presence for a sufficient number of flows is important for the operation of networks.
 
    How critical is to have the flow visibility to the observers? I guess the scenario here is that you have a node in the network which act as a gateway to the Internet hence would see sufficient number of flows to decide what might be the RTT, hence can measure loss or delay on a flow. However, I think this specification should also mention the effectiveness of these passive measurements if that is not the case.

  - It says in section 3.3.2.1 -

      If the observer is unable to estimate RTT of the flow, it should accumulate loss measurements over time periods of at least 4 times the typical RTT for the observed flows.

    I am not sure I completely understand what this "the observed flows" refers to. As described in my previous comment, I can relate this to a certain network setup where the is one node via which all the traffic traverses however, not sure how this works for cases where the observer have no other flow visibility between the same endpoints.

  - Question on section 2.2.5. will there be substantial measurement error depending on whether an observer is unidirectional or can see both direction of the flow that the observers should be aware of? As high accuracy of the measurement is desired, it might be worth mentioning the potential issues.

[Ballot comment]
# Internet AD comments for draft-ietf-ippm-explicit-flow-measurements-03
CC @ekline

* comment syntax:
  - https://github.com/mnot/ietf-comments/blob/main/format.md

* "Handling Ballot Positions":
  - https://ietf.org/about/groups/iesg/statements/handling-ballot-positions/

## Nits

### S2.2

* "is comprised of"

  "is composed of" or "comprises"

[Ballot comment]
Thank you to Steve Hanna for the SECDIR review.

** Certain documents previously published out of IPPM were only intended for closed deployments (sometimes called “limited domains”).  Are the approaches described in this document intended for the Internet?  It would be helpful to state the applicability.

** Section 6.  Given that Section 7 reminds the reader that “[a]uthentication techniques may be used where appropriate to guard against these traffic attacks”, what would that mean in the context of QUIC and TCP?

** Section 7.  It appears that these measurement fields introduced in the packet are intended only for the sender and recipient with little information to any intermediaries.  This seems like a recipe for a covert channel not inspected by typical security devices.  Consider noting this possibility.

[Ballot discuss]
I am wondering why this is an informational document when it uses reserved bits from both QUIC and TCP headers (?). If those reserved bits are used by the mechanisms described in this document but there is no "official" allocation of the bits then future documents that wish to use these bits will be limited and/or clash with an Informational RFC. Adding a DISCUSS as although this is not a technical area of expertise for me, it seems unusual and I would like to better understand the document track selection.

I also do not see a transport area directorate review and in fact the document shepherd highlights that the document could benefit from such a review. Given that the bits introduced in the document are suggested to be carried in the QUIC and TCP headers using their reserved bits, then a review by the area responsible for those transport protocols seems mandatory.

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-ippm-explicit-flow-measurements-03, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

For definitions of IANA review states, please see:

https://datatracker.ietf.org/help/state/draft/iana-review

Thank you,

David Dong
IANA Services Specialist

The following Last Call announcement was sent out (ends 2023-04-06):

From: The IESG
To: IETF-Announce
CC: draft-ietf-ippm-explicit-flow-measurements@ietf.org, ippm-chairs@ietf.org, ippm@ietf.org, marcus.ihlar@ericsson.com, martin.h.duke@gmail.com
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Explicit Host-to-Network Flow Measurements Techniques) to Informational RFC


The IESG has received a request from the IP Performance Measurement WG (ippm)
to consider the following document: - 'Explicit Host-to-Network Flow
Measurements Techniques'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2023-04-06. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  This document describes protocol independent methods called Explicit
  Host-to-Network Flow Measurement Techniques that can be applicable to
  transport-layer protocols between client and server.  These methods
  employ just a few marking bits inside the header of each packet for
  performance measurements and require collaborative client and server.
  Both endpoints cooperate by marking and, possibly, mirroring
  information back and forward on the round-trip connection.  The
  techniques are especially valuable when applied to protocols that
  encrypt transport headers, since they enable loss and delay
  measurements by passive on-path network devices.  Different
  techniques are considered within this document.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-ippm-explicit-flow-measurements/


The following IPR Declarations may be related to this I-D:

  https://datatracker.ietf.org/ipr/3460/
  https://datatracker.ietf.org/ipr/5028/
  https://datatracker.ietf.org/ipr/4103/
  https://datatracker.ietf.org/ipr/3624/
  https://datatracker.ietf.org/ipr/4746/
  https://datatracker.ietf.org/ipr/3467/
  https://datatracker.ietf.org/ipr/3630/

# Document Shepherd Write-Up for Group Documents

*This version is dated 4 July 2022.*

Thank you for your service as a document shepherd. Among the responsibilities is
answering the questions in this write-up to give helpful context to Last Call
and Internet Engineering Steering Group ([IESG][1]) reviewers, and your
diligence in completing it is appreciated. The full role of the shepherd is
further described in [RFC 4858][2]. You will need the cooperation of the authors
and editors to complete these checks.

Note that some numbered items contain multiple related questions; please be sure
to answer all of them.

## Document History

1. Does the working group (WG) consensus represent the strong concurrence of a
  few individuals, with others being silent, or did it reach broad agreement?

  As usual in IPPM the consensus represents a subset of the working group, but
  there is strong concurrence within this group and no pushback from other
  IPPM participants.

2. Was there controversy about particular points, or were there decisions where
  the consensus was particularly rough?

  No real controversy that I have seen. The document is a merge of two
  different documents with separate proposals for loss measurements. All
  proposals are described and the various combinations of them are discussed. 

3. Has anyone threatened an appeal or otherwise indicated extreme discontent? If
  so, please summarize the areas of conflict in separate email messages to the
  responsible Area Director. (It should be in a separate email because this
  questionnaire is publicly available.)

  Not to my knowledge.

4. For protocol documents, are there existing implementations of the contents of
  the document? Have a significant number of potential implementers indicated
  plans to implement? Are any existing implementations reported somewhere,
  either in the document itself (as [RFC 7942][3] recommends) or elsewhere
  (where)?

  While this is not a protocol document per se, there are implementations of
  the mechanisms described in the document.

  Akamai and Orange have implemented setting and reading of L and Q bits for
  loss measurements for QUIC traffic.

  The open source tool spindump by Ericsson research contains logic for reading
  bits of most proposals in this document.

  Telecom Italia have implemented setting of delay, RT-loss etc in a forked QUIC
  implementation. They have also extended spindump and integrated it in
  measurement tools.

  Huawei have expressed interest in implementing mechanisms described in the
  document.

## Additional Reviews

5. Do the contents of this document closely interact with technologies in other
  IETF working groups or external organizations, and would it therefore benefit
  from their review? Have those reviews occurred? If yes, describe which
  reviews took place.

  The mechanisms are designed to be as protocol independent as possible but
  there are examples such as the L bit (section 3.2) which is directly related
  to transport layer loss detection where a review from a TSV expert could
  potentially be beneficial. 

6. Describe how the document meets any required formal expert review criteria,
  such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

  n/a

7. If the document contains a YANG module, has the final version of the module
  been checked with any of the [recommended validation tools][4] for syntax and
  formatting validation? If there are any resulting errors or warnings, what is
  the justification for not fixing them at this time? Does the YANG module
  comply with the Network Management Datastore Architecture (NMDA) as specified
  in [RFC 8342][5]?

  n/a

8. Describe reviews and automated checks performed to validate sections of the
  final version of the document written in a formal language, such as XML code,
  BNF rules, MIB definitions, CBOR's CDDL, etc.

  n/a

## Document Shepherd Checks

9. Based on the shepherd's review of the document, is it their opinion that this
  document is needed, clearly written, complete, correctly designed, and ready
  to be handed off to the responsible Area Director?

  The document in its current form is almost ready to be handed off to the
  responsible AD. The main issue that remains is the number of authors.
  A few nits and reference issues should be addressed as well. 

10. Several IETF Areas have assembled [lists of common issues that their
    reviewers encounter][6]. For which areas have such issues been identified
    and addressed? For which does this still need to happen in subsequent
    reviews?

    This document is not a protocol specification per se so none really applies
    here. Section 2.2.3 describes the use of a timer, but there are no risks
    of creating synchronization effects or changing traffic patterns.

11. What type of RFC publication is being requested on the IETF stream ([Best
    Current Practice][12], [Proposed Standard, Internet Standard][13],
    [Informational, Experimental or Historic][14])? Why is this the proper type
    of RFC? Do all Datatracker state attributes correctly reflect this intent?

    The intended status is Informational which I beleive is the right choice for
    this document. It describes a set of mechanisms in a protocol agnostic way
    with the intention to be used as input for specific protocol implementations
    such as QUIC or CoAP. It does not make sense for this document to use
    normative language. 

12. Have reasonable efforts been made to remind all authors of the intellectual
    property rights (IPR) disclosure obligations described in [BCP 79][7]? To
    the best of your knowledge, have all required disclosures been filed? If
    not, explain why. If yes, summarize any relevant discussion, including links
    to publicly-available messages when applicable.

    Authors and active collaborators have been asked about their knowledge of
    any IPRs beyond what has already been disclosed.

13. Has each author, editor, and contributor shown their willingness to be
    listed as such? If the total number of authors and editors on the front page
    is greater than five, please provide a justification.

    Yes, the authors have shown their willingness to be listed as such. The
    total number of authors is 8, this is because the document is a merge of two
    separate drafts with different sets of authors, all of the authors have made
    substantial contributions to the document and it is therefore difficult to
    reduce the number of authors at this point in the process.

14. Document any remaining I-D nits in this document. Simply running the [idnits
    tool][8] is not enough; please review the ["Content Guidelines" on
    authors.ietf.org][15]. (Also note that the current idnits tool generates
    some incorrect warnings; a rewrite is underway.)

    The TCP reference points to the obsolete RFC 793 and should be replaced with
    RFC 9293.

    Another nit the authors could consider is the naming of references.
    Currently there is a mix of descriptive names and RFC XX, a consistent
    naming scheme would improve readability.

15. Should any informative references be normative or vice-versa? See the [IESG
    Statement on Normative and Informative References][16].

    The use of normative and informative references in this document is a bit
    mixed.

    In section 3.5:

    The ConEx and ConExTCP referenes should be informative rather than normative
    as they describe related mechanisms.

    There is an informative reference to RFC9000 (QUIC-TRANSPORT) in sections 5
    and 6.1.
    Since section 6.1 provides proposals on changing the packet header defined
    in the referred document it should be more appropriate to make it a
    normative reference. 

16. List any normative references that are not freely available to anyone. Did
    the community have sufficient access to review any such normative
    references?

    n/a

17. Are there any normative downward references (see [RFC 3967][9] and [BCP
    97][10]) that are not already listed in the [DOWNREF registry][17]? If so,
    list them.

    no

18. Are there normative references to documents that are not ready to be
    submitted to the IESG for publication or are otherwise in an unclear state?
    If so, what is the plan for their completion?

    no

19. Will publication of this document change the status of any existing RFCs? If
    so, does the Datatracker metadata correctly reflect this and are those RFCs
    listed on the title page, in the abstract, and discussed in the
    introduction? If not, explain why and point to the part of the document
    where the relationship of this document to these other RFCs is discussed.

    no

20. Describe the document shepherd's review of the IANA considerations section,
    especially with regard to its consistency with the body of the document.
    Confirm that all aspects of the document requiring IANA assignments are
    associated with the appropriate reservations in IANA registries. Confirm
    that any referenced IANA registries have been clearly identified. Confirm
    that each newly created IANA registry specifies its initial contents,
    allocations procedures, and a reasonable name (see [RFC 8126][11]).

    This document does not contain any requests to IANA

21. List any new IANA registries that require Designated Expert Review for
    future allocations. Are the instructions to the Designated Expert clear?
    Please include suggestions of designated experts, if appropriate.

    n/a

[1]: https://www.ietf.org/about/groups/iesg/
[2]: https://www.rfc-editor.org/rfc/rfc4858.html
[3]: https://www.rfc-editor.org/rfc/rfc7942.html
[4]: https://trac.ietf.org/trac/ops/wiki/yang-review-tools
[5]: https://www.rfc-editor.org/rfc/rfc8342.html
[6]: https://trac.ietf.org/trac/iesg/wiki/ExpertTopics
[7]: https://www.rfc-editor.org/info/bcp79
[8]: https://www.ietf.org/tools/idnits/
[9]: https://www.rfc-editor.org/rfc/rfc3967.html
[10]: https://www.rfc-editor.org/info/bcp97
[11]: https://www.rfc-editor.org/rfc/rfc8126.html
[12]: https://www.rfc-editor.org/rfc/rfc2026.html#section-5
[13]: https://www.rfc-editor.org/rfc/rfc2026.html#section-4.1
[14]: https://www.rfc-editor.org/rfc/rfc2026.html#section-4.2
[15]: https://authors.ietf.org/en/content-guidelines-overview
[16]: https://www.ietf.org/about/groups/iesg/statements/normative-informative-references/
[17]: https://datatracker.ietf.org/doc/downref/

# Document Shepherd Write-Up for Group Documents

*This version is dated 4 July 2022.*

Thank you for your service as a document shepherd. Among the responsibilities is
answering the questions in this write-up to give helpful context to Last Call
and Internet Engineering Steering Group ([IESG][1]) reviewers, and your
diligence in completing it is appreciated. The full role of the shepherd is
further described in [RFC 4858][2]. You will need the cooperation of the authors
and editors to complete these checks.

Note that some numbered items contain multiple related questions; please be sure
to answer all of them.

## Document History

1. Does the working group (WG) consensus represent the strong concurrence of a
  few individuals, with others being silent, or did it reach broad agreement?

  As usual in IPPM the consensus represents a subset of the working group, but
  there is strong concurrence within this group and no pushback from other
  IPPM participants.

2. Was there controversy about particular points, or were there decisions where
  the consensus was particularly rough?

  No real controversy that I have seen. The document is a merge of two
  different documents with separate proposals for loss measurements. All
  proposals are described and the various combinations of them are discussed. 

3. Has anyone threatened an appeal or otherwise indicated extreme discontent? If
  so, please summarize the areas of conflict in separate email messages to the
  responsible Area Director. (It should be in a separate email because this
  questionnaire is publicly available.)

  Not to my knowledge.

4. For protocol documents, are there existing implementations of the contents of
  the document? Have a significant number of potential implementers indicated
  plans to implement? Are any existing implementations reported somewhere,
  either in the document itself (as [RFC 7942][3] recommends) or elsewhere
  (where)?

  While this is not a protocol document per se, there are implementations of
  the mechanisms described in the document.

  Akamai and Orange have implemented setting and reading of L and Q bits for
  loss measurements for QUIC traffic.

  The open source tool spindump by Ericsson research contains logic for reading
  bits of most proposals in this document.

  Telecom Italia have implemented setting of delay, RT-loss etc in a forked QUIC
  implementation. They have also extended spindump and integrated it in
  measurement tools.

  Huawei have expressed interest in implementing mechanisms described in the
  document.

## Additional Reviews

5. Do the contents of this document closely interact with technologies in other
  IETF working groups or external organizations, and would it therefore benefit
  from their review? Have those reviews occurred? If yes, describe which
  reviews took place.

  The mechanisms are designed to be as protocol independent as possible but
  there are examples such as the L bit (section 3.2) which is directly related
  to transport layer loss detection where a review from a TSV expert could
  potentially be beneficial. 

6. Describe how the document meets any required formal expert review criteria,
  such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

  n/a

7. If the document contains a YANG module, has the final version of the module
  been checked with any of the [recommended validation tools][4] for syntax and
  formatting validation? If there are any resulting errors or warnings, what is
  the justification for not fixing them at this time? Does the YANG module
  comply with the Network Management Datastore Architecture (NMDA) as specified
  in [RFC 8342][5]?

  n/a

8. Describe reviews and automated checks performed to validate sections of the
  final version of the document written in a formal language, such as XML code,
  BNF rules, MIB definitions, CBOR's CDDL, etc.

  n/a

## Document Shepherd Checks

9. Based on the shepherd's review of the document, is it their opinion that this
  document is needed, clearly written, complete, correctly designed, and ready
  to be handed off to the responsible Area Director?

  The document in its current form is almost ready to be handed off to the
  responsible AD. The main issue that remains is the number of authors.
  A few nits and reference issues should be addressed as well. 

10. Several IETF Areas have assembled [lists of common issues that their
    reviewers encounter][6]. For which areas have such issues been identified
    and addressed? For which does this still need to happen in subsequent
    reviews?

    This document is not a protocol specification per se so none really applies
    here. Section 2.2.3 describes the use of a timer, but there are no risks
    of creating synchronization effects or changing traffic patterns.

11. What type of RFC publication is being requested on the IETF stream ([Best
    Current Practice][12], [Proposed Standard, Internet Standard][13],
    [Informational, Experimental or Historic][14])? Why is this the proper type
    of RFC? Do all Datatracker state attributes correctly reflect this intent?

    The intended status is Informational which I beleive is the right choice for
    this document. It describes a set of mechanisms in a protocol agnostic way
    with the intention to be used as input for specific protocol implementations
    such as QUIC or CoAP. It does not make sense for this document to use
    normative language. 

12. Have reasonable efforts been made to remind all authors of the intellectual
    property rights (IPR) disclosure obligations described in [BCP 79][7]? To
    the best of your knowledge, have all required disclosures been filed? If
    not, explain why. If yes, summarize any relevant discussion, including links
    to publicly-available messages when applicable.

    Authors and active collaborators have been asked about their knowledge of
    any IPRs beyond what has already been disclosed.

13. Has each author, editor, and contributor shown their willingness to be
    listed as such? If the total number of authors and editors on the front page
    is greater than five, please provide a justification.

    Yes, the authors have shown their willingness to be listed as such. The
    total number of authors is 8, this is because the document is a merge of two
    separate drafts with different sets of authors, all of the authors have made
    substantial contributions to the document and it is therefore difficult to
    reduce the number of authors at this point in the process.

14. Document any remaining I-D nits in this document. Simply running the [idnits
    tool][8] is not enough; please review the ["Content Guidelines" on
    authors.ietf.org][15]. (Also note that the current idnits tool generates
    some incorrect warnings; a rewrite is underway.)

    The TCP reference points to the obsolete RFC 793 and should be replaced with
    RFC 9293.

    Another nit the authors could consider is the naming of references.
    Currently there is a mix of descriptive names and RFC XX, a consistent
    naming scheme would improve readability.

15. Should any informative references be normative or vice-versa? See the [IESG
    Statement on Normative and Informative References][16].

    The use of normative and informative references in this document is a bit
    mixed.

    In section 3.5:

    The ConEx and ConExTCP referenes should be informative rather than normative
    as they describe related mechanisms.

    There is an informative reference to RFC9000 (QUIC-TRANSPORT) in sections 5
    and 6.1.
    Since section 6.1 provides proposals on changing the packet header defined
    in the referred document it should be more appropriate to make it a
    normative reference. 

16. List any normative references that are not freely available to anyone. Did
    the community have sufficient access to review any such normative
    references?

    n/a

17. Are there any normative downward references (see [RFC 3967][9] and [BCP
    97][10]) that are not already listed in the [DOWNREF registry][17]? If so,
    list them.

    no

18. Are there normative references to documents that are not ready to be
    submitted to the IESG for publication or are otherwise in an unclear state?
    If so, what is the plan for their completion?

    no

19. Will publication of this document change the status of any existing RFCs? If
    so, does the Datatracker metadata correctly reflect this and are those RFCs
    listed on the title page, in the abstract, and discussed in the
    introduction? If not, explain why and point to the part of the document
    where the relationship of this document to these other RFCs is discussed.

    no

20. Describe the document shepherd's review of the IANA considerations section,
    especially with regard to its consistency with the body of the document.
    Confirm that all aspects of the document requiring IANA assignments are
    associated with the appropriate reservations in IANA registries. Confirm
    that any referenced IANA registries have been clearly identified. Confirm
    that each newly created IANA registry specifies its initial contents,
    allocations procedures, and a reasonable name (see [RFC 8126][11]).

    This document does not contain any requests to IANA

21. List any new IANA registries that require Designated Expert Review for
    future allocations. Are the instructions to the Designated Expert clear?
    Please include suggestions of designated experts, if appropriate.

    n/a

[1]: https://www.ietf.org/about/groups/iesg/
[2]: https://www.rfc-editor.org/rfc/rfc4858.html
[3]: https://www.rfc-editor.org/rfc/rfc7942.html
[4]: https://trac.ietf.org/trac/ops/wiki/yang-review-tools
[5]: https://www.rfc-editor.org/rfc/rfc8342.html
[6]: https://trac.ietf.org/trac/iesg/wiki/ExpertTopics
[7]: https://www.rfc-editor.org/info/bcp79
[8]: https://www.ietf.org/tools/idnits/
[9]: https://www.rfc-editor.org/rfc/rfc3967.html
[10]: https://www.rfc-editor.org/info/bcp97
[11]: https://www.rfc-editor.org/rfc/rfc8126.html
[12]: https://www.rfc-editor.org/rfc/rfc2026.html#section-5
[13]: https://www.rfc-editor.org/rfc/rfc2026.html#section-4.1
[14]: https://www.rfc-editor.org/rfc/rfc2026.html#section-4.2
[15]: https://authors.ietf.org/en/content-guidelines-overview
[16]: https://www.ietf.org/about/groups/iesg/statements/normative-informative-references/
[17]: https://datatracker.ietf.org/doc/downref/

# Document Shepherd Write-Up for Group Documents

*This version is dated 4 July 2022.*

Thank you for your service as a document shepherd. Among the responsibilities is
answering the questions in this write-up to give helpful context to Last Call
and Internet Engineering Steering Group ([IESG][1]) reviewers, and your
diligence in completing it is appreciated. The full role of the shepherd is
further described in [RFC 4858][2]. You will need the cooperation of the authors
and editors to complete these checks.

Note that some numbered items contain multiple related questions; please be sure
to answer all of them.

## Document History

1. Does the working group (WG) consensus represent the strong concurrence of a
  few individuals, with others being silent, or did it reach broad agreement?

  As usual in IPPM the consensus represents a subset of the working group, but
  there is strong concurrence within this group and no pushback from other
  IPPM participants.

2. Was there controversy about particular points, or were there decisions where
  the consensus was particularly rough?

  No real controversy that I have seen. The document is a merge of two
  different documents with separate proposals for loss measurements. All
  proposals are described and the various combinations of them are discussed. 

3. Has anyone threatened an appeal or otherwise indicated extreme discontent? If
  so, please summarize the areas of conflict in separate email messages to the
  responsible Area Director. (It should be in a separate email because this
  questionnaire is publicly available.)

  Not to my knowledge.

4. For protocol documents, are there existing implementations of the contents of
  the document? Have a significant number of potential implementers indicated
  plans to implement? Are any existing implementations reported somewhere,
  either in the document itself (as [RFC 7942][3] recommends) or elsewhere
  (where)?

  While this is not a protocol document per se, there are implementations of
  the mechanisms described in the document.

  Akamai and Orange have implemented setting and reading of L and Q bits for
  loss measurements for QUIC traffic.

  The open source tool spindump by Ericsson research contains logic for reading
  bits of most proposals in this document.

  Telecom Italia have implemented setting of delay, RT-loss etc in a forked QUIC
  implementation. They have also extended spindump and integrated it in
  measurement tools.

  Huawei have expressed interest in implementing mechanisms described in the
  document.

## Additional Reviews

5. Do the contents of this document closely interact with technologies in other
  IETF working groups or external organizations, and would it therefore benefit
  from their review? Have those reviews occurred? If yes, describe which
  reviews took place.

  The mechanisms are designed to be as protocol independent as possible but
  there are examples such as the L bit (section 3.2) which is directly related
  to transport layer loss detection where a review from a TSV expert could
  potentially be beneficial. 

6. Describe how the document meets any required formal expert review criteria,
  such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

  n/a

7. If the document contains a YANG module, has the final version of the module
  been checked with any of the [recommended validation tools][4] for syntax and
  formatting validation? If there are any resulting errors or warnings, what is
  the justification for not fixing them at this time? Does the YANG module
  comply with the Network Management Datastore Architecture (NMDA) as specified
  in [RFC 8342][5]?

  n/a

8. Describe reviews and automated checks performed to validate sections of the
  final version of the document written in a formal language, such as XML code,
  BNF rules, MIB definitions, CBOR's CDDL, etc.

  n/a

## Document Shepherd Checks

9. Based on the shepherd's review of the document, is it their opinion that this
  document is needed, clearly written, complete, correctly designed, and ready
  to be handed off to the responsible Area Director?

  The document in its current form is almost ready to be handed off to the
  responsible AD. The main issue that remains is the number of authors.
  A few nits and reference issues should be addressed as well. 

10. Several IETF Areas have assembled [lists of common issues that their
    reviewers encounter][6]. For which areas have such issues been identified
    and addressed? For which does this still need to happen in subsequent
    reviews?

    This document is not a protocol specification per se so none really applies
    here. Section 2.2.3 describes the use of a timer, but there are no risks
    of creating synchronization effects or changing traffic patterns.

11. What type of RFC publication is being requested on the IETF stream ([Best
    Current Practice][12], [Proposed Standard, Internet Standard][13],
    [Informational, Experimental or Historic][14])? Why is this the proper type
    of RFC? Do all Datatracker state attributes correctly reflect this intent?

    The intended status is Informational which I beleive is the right choice for
    this document. It describes a set of mechanisms in a protocol agnostic way
    with the intention to be used as input for specific protocol implementations
    such as QUIC or CoAP. It does not make sense for this document to use
    normative language. 

12. Have reasonable efforts been made to remind all authors of the intellectual
    property rights (IPR) disclosure obligations described in [BCP 79][7]? To
    the best of your knowledge, have all required disclosures been filed? If
    not, explain why. If yes, summarize any relevant discussion, including links
    to publicly-available messages when applicable.

    Authors and active collaborators have been asked about their knowledge of
    any IPRs beyond what has already been disclosed.

13. Has each author, editor, and contributor shown their willingness to be
    listed as such? If the total number of authors and editors on the front page
    is greater than five, please provide a justification.

    There are currently seven authors. This is in my view too large and not
    justified. The reason for the large number is that the document is a merge
    of two separate I-Ds with different sets of authors.

14. Document any remaining I-D nits in this document. Simply running the [idnits
    tool][8] is not enough; please review the ["Content Guidelines" on
    authors.ietf.org][15]. (Also note that the current idnits tool generates
    some incorrect warnings; a rewrite is underway.)

    The TCP reference points to the obsolete RFC 793 and should be replaced with
    RFC 9293.

    Another nit the authors could consider is the naming of references.
    Currently there is a mix of descriptive names and RFC XX, a consistent
    naming scheme would improve readability.

15. Should any informative references be normative or vice-versa? See the [IESG
    Statement on Normative and Informative References][16].

    The use of normative and informative references in this document is a bit
    mixed.

    In section 3.5:

    The ConEx and ConExTCP referenes should be informative rather than normative
    as they describe related mechanisms.

    There is an informative reference to RFC9000 (QUIC-TRANSPORT) in sections 5
    and 6.1.
    Since section 6.1 provides proposals on changing the packet header defined
    in the referred document it should be more appropriate to make it a
    normative reference. 

16. List any normative references that are not freely available to anyone. Did
    the community have sufficient access to review any such normative
    references?

    n/a

17. Are there any normative downward references (see [RFC 3967][9] and [BCP
    97][10]) that are not already listed in the [DOWNREF registry][17]? If so,
    list them.

    no

18. Are there normative references to documents that are not ready to be
    submitted to the IESG for publication or are otherwise in an unclear state?
    If so, what is the plan for their completion?

    no

19. Will publication of this document change the status of any existing RFCs? If
    so, does the Datatracker metadata correctly reflect this and are those RFCs
    listed on the title page, in the abstract, and discussed in the
    introduction? If not, explain why and point to the part of the document
    where the relationship of this document to these other RFCs is discussed.

    no

20. Describe the document shepherd's review of the IANA considerations section,
    especially with regard to its consistency with the body of the document.
    Confirm that all aspects of the document requiring IANA assignments are
    associated with the appropriate reservations in IANA registries. Confirm
    that any referenced IANA registries have been clearly identified. Confirm
    that each newly created IANA registry specifies its initial contents,
    allocations procedures, and a reasonable name (see [RFC 8126][11]).

    This document does not contain any requests to IANA

21. List any new IANA registries that require Designated Expert Review for
    future allocations. Are the instructions to the Designated Expert clear?
    Please include suggestions of designated experts, if appropriate.

    n/a

[1]: https://www.ietf.org/about/groups/iesg/
[2]: https://www.rfc-editor.org/rfc/rfc4858.html
[3]: https://www.rfc-editor.org/rfc/rfc7942.html
[4]: https://trac.ietf.org/trac/ops/wiki/yang-review-tools
[5]: https://www.rfc-editor.org/rfc/rfc8342.html
[6]: https://trac.ietf.org/trac/iesg/wiki/ExpertTopics
[7]: https://www.rfc-editor.org/info/bcp79
[8]: https://www.ietf.org/tools/idnits/
[9]: https://www.rfc-editor.org/rfc/rfc3967.html
[10]: https://www.rfc-editor.org/info/bcp97
[11]: https://www.rfc-editor.org/rfc/rfc8126.html
[12]: https://www.rfc-editor.org/rfc/rfc2026.html#section-5
[13]: https://www.rfc-editor.org/rfc/rfc2026.html#section-4.1
[14]: https://www.rfc-editor.org/rfc/rfc2026.html#section-4.2
[15]: https://authors.ietf.org/en/content-guidelines-overview
[16]: https://www.ietf.org/about/groups/iesg/statements/normative-informative-references/
[17]: https://datatracker.ietf.org/doc/downref/

Request for posting confirmation emailed to previous authors: Alexandre Ferrieux , Dmitri Tikhonov , Fabio Bulgarella , Giuseppe Fioccola , Igor Lubashev , Isabelle Hamchaoui , Massimo Nilo , Mauro Cociglio , Riccardo Sisto , ippm-chairs@ietf.org

Request for posting confirmation emailed to previous authors: Alexandre Ferrieux , Dmitri Tikhonov , Fabio Bulgarella , Giuseppe Fioccola , Igor Lubashev , Isabelle Hamchaoui , Massimo Nilo , Mauro Cociglio , Riccardo Sisto