Metrics and Methods for One-Way IP Capacity
draft-ietf-ippm-capacity-metric-method-12

[Ballot comment]
Thank you for the work put into this document.

Please find below some non-blocking COMMENT points (but replies would be appreciated).

I hope that this helps to improve the document,

Regards,

-éric


== COMMENTS ==

Generic comment about the 1 Mbps granularity, this may of course be well suited for many connection but not really for constrained networks where the measurement techniques of this I-D could be applied.

Should the payload content random content to avoid some compression techniques on the path ?


-- Section 4 --
s/the address of a host/one of the IP addresses of a host/

Should there be a SHOULD rather than a MAY in "The IPv6 flow label MAY be included" ?

-- Section 5.3 --
"  o  n0 is the total number of IP-Layer header and payload bits that
      can be transmitted in standard-formed packets [RFC8468] from the
      Src host and correctly received by the Dst host during one
      contiguous sub-interval, dt in length, during the interval [T,
      T+I],"

If we want to split hairs, in the case of IP fragmentation or NAT64, the received bits will be different than the one sent ;-)
I had no time to read RFC 8468 (day job...), but, does the above remark fall in the same category as "Some compression affects on measurement are discussed in Section 6 of [RFC8468]."

-- Section 8.3 --
"  The number of concurrent, independent tests between
  the same Source and Destination SHALL be limited to one."

Is it between hosts or between network? I.e.n "same source and destination networks?"

-- Section 8.4 --
"supports IPv4 and IPv6 address families." does it support a NAT64 function on the path ?

Closed request for Telechat review by INTDIR with state 'Withdrawn': The document is on the IESG telechat today. No need anymore for a review (still welcome by the authors probably).

[Ballot comment]
I've changed my position from NoObj to Yes. I realize that this doesn't change anything at all, but I wanted to signal that I appreciate the time/process wonkery to address the downref.

I'm stealing Robert's ballot text, because it perfectly explains my position/views:
"Thank you for this document, I found it interesting to read.

I have no comments that haven't already been raised by other AD reviews."

[Ballot comment]
I've changed my position from NoObj to Yes. I realize that this doesn't change anything at all, but I wanted to signal that I appreciate the time/process wonkery to address the downref.


I'm stealing Robert's ballot text, because it perfectly explains my position/views:
"Thank you for this document, I found it interesting to read.

I have no comments that haven't already been raised by other AD reviews.
"

[Ballot comment]
Thanks for the work on the document. I consider this document to be an important one when it comes to IP layer capacity for access networks.

After my review and observing the IPPM working group discussions, I consider the valid discuss points raised by Magnus Westerlund is now addressed in the -10 version of this document.

please find my comments and observations below -

* Major Issue: I didn't find any restriction imposed by the memo on the number of allowed concurrent tests between same Source and Destination. My understanding is this memo should either strictly prohibit the concurrent tests between same Source and Destination or provide a maximum limit of concurrent tests where the algorithm has been tested to provide valid results. I am not going to hold a discuss on this but I would like to be corrected if I am wrong in the understanding.

* Minor Issues , nits:

  ** I found it a bit odd to not find Magnus Westerlund's name in the acknowledgement section as his review and comments has improved the algorithms and the document a lot. I would encourage the author's to include names of the individuals who has provided any sort of review (including the directorate reviews) that improved this document.
 
  ** Section 2:

      *** I am not getting the context of "local goal", what does it really mean?

      *** I kind of feel that the paragraph 2 and 3 should swap their order to focus the goal of "defining efficient test procedure" and then "harmonizing the metric and method across the industry" becomes "another goal". This is based on the current text which makes me feel there are number of goals and the ranks (if there is any) of the goals are not clear. If there is not rank among the goals then simply putting them in a bullet list would help the readability.

  ** Section 8.1:

      *** paragraph 1: s/ agressiveness / aggressiveness .

      *** paragraph 2: It will be helpful if it is mentioned who pre-builts the table and  who (and where) it will be used.
 
  ** Section 10 : I find 4, 5, 6 of the new security considerations introduced in this section not entirely security related rather general considerations on how to run the tests. I think those mentioned considerations are more suitable to put it in section 8 (in section 8.3). I understand the feeling the everyone should read the security considerations but there is no guarantee that it is the case specially while implementing the tests. Hence, putting then in section 8 likely will bring these to attention early.

[Ballot comment]
Thanks for the work on the document. I consider this document to be an important one when it comes to IP layer capacity for access networks.

After my review and observing the IPPM working group discussions, I consider the valid discuss points raised by Magnus Westerlund is now addressed in the -10 version of this document.

please find my comments and observations below -

* Major Issue: I didn't find any restriction imposed by the memo on the number of allowed concurrent tests between same Source and Destination. My understanding is this memo should either strictly prohibit the concurrent tests between same Source and Destination or provide a maximum limit of concurrent tests where the algorithm has been tested to provide valid results. I am not going to hold a discuss on this but I would like to be corrected if I am wrong in the understanding.

* Minor Issues , nits:

  ** I found it a bit odd to not find Magnus Westerlund's name in the acknowledgement section as his review and comments has improved the algorithms and the document a lot. I would encourage the author's to include names of the individuals who has provided any sort of review (including the directorate reviews) that improved this document.
  ** Section 2:

      *** I am not getting the context of "local goal", what does it really mean?

      *** I kind of feel that the paragraph 2 and 3 should swap their order to focus the goal of "defining efficient test procedure" and then "harmonizing the metric and method across the industry" becomes "another goal". This is based on the current text which makes me feel there are number of goals and the ranks (if there is any) of the goals are not clear. If there is not rank among the goals then simply putting them in a bullet list would help the readability.

  ** Section 8.1:

      *** paragraph 1: s/ agressiveness / aggressiveness .

      *** paragraph 2: It will be helpful if it is mentioned who pre-builts the table and  who (and where) it will be used.
 
  ** Section 10 : I find 4, 5, 6 of the new security considerations introduced in this section not entirely security related rather general considerations on how to run the tests. I think those mentioned considerations are more suitable to put it in section 8 (in section 8.3). I understand the feeling the everyone should read the security considerations but there is no guarantee that it is the case specially while implementing the tests. Hence, putting then in section 8 likely will bring these to attention early.

[Ballot comment]
Thanks for the work on the document. I consider this document to be an important one when it comes to IP layer capacity for access networks.

After my review and observing the IPPM working group discussions, I consider the valid discuss points raised by Magnus Westerlund is now addressed in the -10 version of this document.

please find my comments and observations below -

* Major Issue: I didn't find any restriction imposed by the memo on the number of allowed concurrent tests between same Source and Destination. My understanding is this memo should either strictly prohibit the concurrent tests between same Source and Destination or provide a maximum limit of concurrent tests where the algorithm has been tested to provide valid results. I am not going to hold a discuss on this but I would like to be corrected if I am wrong in the understanding.

* Minor Issues , nits:

  ** I found it a bit odd to not find Magnus Westerlund's name in the acknowledgement section as his review and comments has improved the algorithms and the document a lot. I would encourage the author's to include names of the individuals who has provided any sort of review (including the directorate reviews) that improved this document.
  ** Section 2:

      *** I am not getting the context of "local goal", what does it really mean?
      *** I kind of feel that the paragraph 2 and 3 should swap their order to focus the goal of "defining efficient test procedure" and then "harmonizing the metric and method across the industry" becomes "another goal". This is based on the current text which makes me feel there are number of goals and the ranks (if there is any) of the goals are not clear. If there is not rank among the goals then simply putting them in a bullet list would help the readability.

  ** Section 8.1:

      *** paragraph 1: s/ agressiveness / aggressiveness .

      *** paragraph 2: It will be helpful if it is mentioned who pre-builts the table and  who (and where) it will be used.
 
  ** Section 10 : I find 4, 5, 6 of the new security considerations introduced in this section not entirely security related rather general considerations on how to run the tests. I think those mentioned considerations are more suitable to put it in section 8 (in section 8.3). I understand the feeling the everyone should read the security considerations but there is no guarantee that it is the case specially while implementing the tests. Hence, putting then in section 8 likely will bring these to attention early.

[Ballot comment]
Thanks for the work on the document. I consider this document to be an important one when it comes to IP layer capacity for access networks.

After my review and observing the IPPM working group discussions, I consider the valid discuss points raised by Magnus Westerlund is now addressed in the -10 version of this document.

please find my comments and observations below -

* Major Issue: I didn't find any restriction imposed by the memo on the number of allowed concurrent tests between same Source and Destination. My understanding is this memo should either strictly prohibit the concurrent tests between same Source and Destination or provide a maximum limit of concurrent tests where the algorithm has been tested to provide valid results. I am not going to hold a discuss on this but I would like to be corrected if I am wrong in the understanding.

* Minor Issues , nits:

  ** I found it a bit odd to not find Magnus Westerlund's name in the acknowledgement section as his review and comments has improved the algorithms and the document a lot. I would encourage the author's to include names of the individuals who has provided any sort of review (including the directorate reviews) that improved this document.
  ** Section 2:

      *** I am not getting the context of "local goal", what does it really mean?
      *** I kind of feel that the paragraph 2 and 3 should swap their order to focus the goal of "defining efficient test procedure" and then "harmonizing the metric and method across the industry" becomes "another goal". This is based on the current text which makes me feel there are number of goals and the ranks (if there is any) of the goals are not clear. If there is not rank among the goals then simply putting them in a bullet list would help the readability.

  ** Section 8.1:

      *** paragraph 1: s/ agressiveness / aggressiveness .
      *** paragraph 2: It will be helpful if it is mentioned who pre-builts the table and  who (and where) it will be used.
 
  ** Section 10 : I find 4, 5, 6 of the new security considerations introduced in this section not entirely security related rather general considerations on how to run the tests. I think those mentioned considerations are more suitable to put it in section 8 (in section 8.3). I understand the feeling the everyone should read the security considerations but there is no guarantee that it is the case specially while implementing the tests. Hence, putting then in section 8 likely will bring these to attention early.

[Ballot comment]
Thanks for the work on the document. I consider this document to be an important one when it comes to IP layer capacity for access networks.

After my review and observing the IPPM working group discussions, I consider the valid discuss points raised by Magnus Westerlund is now addressed in the -10 version of this document.

please find my comments and observations below -

* Major Issue: I didn't find any restriction imposed by the memo on number of concurrent tests between same Source and Destination. My understanding is this memo should either strictly prohibit the concurrent tests between same Source and Destination or provide a maximum limit of concurrent tests where the algorithm has been tested to provide valid results. I am not going to hold a discuss on this but I would like to corrected if I am wrong in the understanding.

* Minor Issues , nits:

  ** I found it a bit odd to not find Magnus Westerlund's name in the acknowledgement section as his review and comments has improved the algorithms and the document a lot. I would encourage the author's to include names of the individuals who has provided any sort of review (including the directorate reviews) that improved this document.
  ** Section 2:

      *** I am not getting the context of "local goal", what does it really mean?
      *** I kind of feel that the paragraph 2 and 3 should swap their order to focus the goal on "defining efficient test procedure" and the "harmonizing the metric and method across the industry" becomes "another goal". This is based on the current text which makes me feel there are number of goals and the ranks (if there is any) of the goals are not clear.
  ** Section 8.1:

      *** paragraph 1: s/ agressiveness / aggressiveness .
      *** paragraph 2: It will be helpful if it is mentioned who pre-builts the table and  who (and where) it will be used.
  ** Section 10 : I find 4, 5, 6 of the new security considerations introduced in this section not really entirely security related rather general considerations for and more suitable to put it in section 8 (in section 8.3). I understand the feeling the everyone should read the security considerations but there is guarantee that it is the case specially while implementing the tests. Hence, putting then in section 8 likely will bring this to attention early.

Note changed to 'draft-10 added a normative downref to RFC 7497 (Informational), which specifies the architectural context for this draft. On 20 May 2021, the IESG elected to waive the requirement for a second IETF Last Call in accordance with its authority in RFC 8067.'

[Ballot discuss]
DOWNREF from this Standards Track doc to Informational RFC7497.
I didn't see this called out in the Last Call, and it's also not a document we have
in the DOWNREF registry already.

[Ballot comment]
Section 1, paragraph 3, comment:
>    Here, the main use case is to assess the maximum capacity of the
>    access network, with specific performance criteria used in the
>    measurement.

If the main use case is to measure access network paths, that applicability
should be made very explicit, i.e., in the title and abstract.

Section 2, paragraph 2, comment:
>    The scope of this memo is to define a metric and corresponding method
>    to unambiguously perform Active measurements of Maximum IP-Layer
>    Capacity, along with related metrics and methods.

I'm not sure what "unambiguously perform" is meant to express?

Section 2, paragraph 3, comment:
>    A local goal is to aid efficient test procedures where possible, and
>    to recommend reporting with additional interpretation of the results.

What is this goal "local" to - the IETF? Also, I'm unclear what "reporting with
additional interpretation of the results" is supposed to express.

Section 2, paragraph 10, comment:
>    - If a network operator is certain of the access capacity to be
>    validated, then testing MAY start with a fixed rate test at the
>    access capacity and avoid activating the load adjustment algorithm.
>    However, the stimulus for a diagnostic test (such as a subscriber
>    request) strongly implies that there is no certainty and the load
>    adjustment algorithm will be needed.

Since the first sentence of this paragraph uses a RFC2119 MAY, I'd suggest
rephrasing the second sentence to end with "there is no certainty and use of the
load adjustment algorithm is RECOMMENDED."

Section 3, paragraph 4, comment:
>    1.  Internet access is no longer the bottleneck for many users.

What is the bottleneck then, and if it's not the access bandwidth, why is a new
metric for it helpful?

Section 4, paragraph 4, comment:
>    o  Src, the address of a host (such as the globally routable IP
>      address).
>
>    o  Dst, the address of a host (such as the globally routable IP
>      address).

For many hosts, their IP address is not globally routable, and hasn't been for
decades. Or did you mean CPE?

Section 8.1, paragraph 1, comment:
> 8.1.  Load Rate Adjustment Algorithm

I wonder why this section is trying to define a crude new AIMD scheme, when we
have lots of good experience with TCP's AIMD approach, which converges on the
available bandwidth relatively well?

Also, the description of the algorithm itself is far from clear.

Section 8.1, paragraph 7, comment:
>    If the feedback indicates that no sequence number anomalies were
>    detected AND the delay range was below the lower threshold, the
>    offered load rate is increased.  If congestion has not been confirmed
>    up to this point, the offered load rate is increased by more than one
>    rate (e.g., Rx+10).  This allows the offered load to quickly reach a
>    near-maximum rate.  Conversely, if congestion has been previously
>    confirmed, the offered load rate is only increased by one (Rx+1).

The first sentence talks about sequence number anomalies and delay ranges. The
rest of the paragraph then talks about whether congestion and whether it was
confirmed or not. Does this mean that (some amount of) sequence number anomalies
and/or delay indicates congestion? Is that defined somewhere?

Section 8.1, paragraph 7, comment:
>    If the feedback indicates that sequence number anomalies were
>    detected OR the delay range was above the upper threshold, the
>    offered load rate is decreased.  The RECOMMENDED values are 0 for

This doesn't agree with the previous paragraph, which says "Conversely, if
congestion has been previously confirmed, the offered load rate is only
increased by one (Rx+1)." (Or I don't understand the difference between
congestion and sequence number anomalies/delay ranges; see previous comment.)

Section 8.4, paragraph 2, comment:
>    This section is for the benefit of the Document Shepherd's form, and
>    will be deleted prior to final review.

Should this section have been deleted before the IESG review then? Would you
like the IESG to ignore it? You probably also need to instruct the RFC Editor to
remove it before publication?

This document uses RFC2119 keywords, but does not contain the recommended
RFC8174 boilerplate. (It contains some text with a similar beginning.)

-------------------------------------------------------------------------------
All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools, so there will likely be some false positives. There is no need
to let me know what you did with these suggestions.

Section 1, paragraph 5, nit:
-    authors found that a high percentage of paths tested support UDP
+    authors found that a high percentage of paths tested support the UDP
+                                                                ++++

Section 3, paragraph 8, nit:
-        to less traffic crossing ISP gateways in future.
+        to less traffic crossing ISP gateways in the future.
+                                                ++++

Section 8.1, paragraph 2, nit:
-    agressiveness (speed of ramp-up and down to the Maximum IP-Layer
+    aggressiveness (speed of ramp-up and down to the Maximum IP-Layer
+    +

Section 8.2, paragraph 5, nit:
-    Another approach comes from Section 24 of RFC 2544[RFC2544] and its
-                                              --------
+    Another approach comes from Section 24 of [RFC2544] and its

Section 8.3, paragraph 4, nit:
-    The path measured may be state-full based on many factors, and the
-                                  -  -
+    The path measured may be stateful based on many factors, and the

Section 14.2, paragraph 5, nit:
-    measurements with a clear and concise rate reduction when warrented,
-                                                                  ^
+    measurements with a clear and concise rate reduction when warranted,
+                                                                  ^

"I", paragraph 2, nit:
>  to the specifications of other Standards Development Organizations (SDO) (t
>                                ^^^^^^^^^
An apostrophe may be missing.

"S", paragraph 2, nit:
> t Maximum IP-Layer Capacity is unknown (which is sometimes the case; service
>                                        ^
Unpaired symbol: ')' seems to be missing

Section 6.3, paragraph 14, nit:
> urement systems MUST be higher than than the smallest of the links on the pa
>                                ^^^^^^^^^
Possible typo: you repeated a word

Section 6.3, paragraph 15, nit:
> PM list over all dt-length intervals in I. Measurements according to these de
>                                      ^^^^
Did you mean "in me", "in her", "in him", "in us", or "in them"?

Section 8, paragraph 3, nit:
> work, since this is an active test method and it will likely cause congestion
>                                    ^^^^^^^^^^
Use a comma before 'and' if it connects two independent clauses (unless they
are closely connected and short).

Section 8.2, paragraph 4, nit:
> etons (dt = 1 second) has proven to a be of practical value during tests of
>                                    ^^^^
After 'a', do not use a verb. Make sure that the spelling of 'be' is correct.
If 'be' is the first word in a compound adjective, use a hyphen between the two
words. Note: This error message can occur if you use a verb as a noun, and the
word is not a noun in standard English.

Section 8.3, paragraph 4, nit:
> nts based on those packets. Many different traffic shapers and on-demand acce
>                            ^^^^^^^^^^^^^^
Consider using "Many".

Section 8.3, paragraph 6, nit:
> , where packet losses may occur independently from the measurement sending ra
>                                ^^^^^^^^^^^^^^^^^^
The usual collocation for "independently" is "of", not "from". Did you mean
"independently of"?

Section 10, paragraph 10, nit:
> ffic (for example, the range of I duration values SHOULD be limited). The exa
>                                ^^^^^^^^^^
Do not use a noun immediately after the pronoun 'I'. Use a verb or an adverb,
or possibly some other part of speech.

"R", paragraph 1, nit:
> ble) seqErr = 0 # Measured count of any of Loss or Reordering impairments de
>                                  ^^^^^^^^^
Consider simply using "of" instead.

Section 14.1, paragraph 10, nit:
> g a test when connectivity is lost for an longer interval (Feedback message o
>                                        ^^
Use "a" instead of 'an' if the following word doesn't start with a vowel sound,
e.g. 'a sentence', 'a university'.

Section 14.2, paragraph 10, nit:
> easurements. A brief review of some of the other SHOULD-level requirements fo
>                                ^^^^^^^^^^^
If the text is a generality, 'of the' is not necessary.

"Authors' Addresses", paragraph 2, nit:
>  200 Laurel Avenue South Middletown,, NJ 07748 USA Phone: +1 732 420
>                                    ^^
Two consecutive commas

"Authors' Addresses", paragraph 5, nit:
>  200 Laurel Avenue South Middletown,, NJ 07748 USA Email: lencia@att
>                                    ^^
Two consecutive commas

[Ballot discuss]
Zahed is still on leave and has not had a chance to review Magnus' earlier DISCUSS.
I deferred this document last time it was on the agenda, but since Zahed is still
not back, I will put in a placeholder DISCUSS for him, so that he has a chance to
complete his review.

I apologize to the authors and the WG for doing this; these are exceptional
circumstances. I will clear this placeholder DISCUSS as soon as Zahed is able to
resume as AD (or Martin Duke indicates that he has reviewed Magnus' DISCUSS.)

All that said, I do have one DISCUSS item:

DOWNREF from this Standards Track doc to Informational RFC7497.
I didn't see this called out in the Last Call, and it's also not a document we have
in the DOWNREF registry already.

[Ballot comment]
(These are my comments and not related to the placeholder DICUSS above.)

Section 1, paragraph 3, comment:
>    Here, the main use case is to assess the maximum capacity of the
>    access network, with specific performance criteria used in the
>    measurement.

If the main use case is to measure access network paths, that applicability
should be made very explicit, i.e., in the title and abstract.

Section 2, paragraph 2, comment:
>    The scope of this memo is to define a metric and corresponding method
>    to unambiguously perform Active measurements of Maximum IP-Layer
>    Capacity, along with related metrics and methods.

I'm not sure what "unambiguously perform" is meant to express?

Section 2, paragraph 3, comment:
>    A local goal is to aid efficient test procedures where possible, and
>    to recommend reporting with additional interpretation of the results.

What is this goal "local" to - the IETF? Also, I'm unclear what "reporting with
additional interpretation of the results" is supposed to express.

Section 2, paragraph 10, comment:
>    - If a network operator is certain of the access capacity to be
>    validated, then testing MAY start with a fixed rate test at the
>    access capacity and avoid activating the load adjustment algorithm.
>    However, the stimulus for a diagnostic test (such as a subscriber
>    request) strongly implies that there is no certainty and the load
>    adjustment algorithm will be needed.

Since the first sentence of this paragraph uses a RFC2119 MAY, I'd suggest
rephrasing the second sentence to end with "there is no certainty and use of the
load adjustment algorithm is RECOMMENDED."

Section 3, paragraph 4, comment:
>    1.  Internet access is no longer the bottleneck for many users.

What is the bottleneck then, and if it's not the access bandwidth, why is a new
metric for it helpful?

Section 4, paragraph 4, comment:
>    o  Src, the address of a host (such as the globally routable IP
>      address).
>
>    o  Dst, the address of a host (such as the globally routable IP
>      address).

For many hosts, their IP address is not globally routable, and hasn't been for
decades. Or did you mean CPE?

Section 8.1, paragraph 1, comment:
> 8.1.  Load Rate Adjustment Algorithm

I wonder why this section is trying to define a crude new AIMD scheme, when we
have lots of good experience with TCP's AIMD approach, which converges on the
available bandwidth relatively well?

Also, the description of the algorithm itself is far from clear.

Section 8.1, paragraph 7, comment:
>    If the feedback indicates that no sequence number anomalies were
>    detected AND the delay range was below the lower threshold, the
>    offered load rate is increased.  If congestion has not been confirmed
>    up to this point, the offered load rate is increased by more than one
>    rate (e.g., Rx+10).  This allows the offered load to quickly reach a
>    near-maximum rate.  Conversely, if congestion has been previously
>    confirmed, the offered load rate is only increased by one (Rx+1).

The first sentence talks about sequence number anomalies and delay ranges. The
rest of the paragraph then talks about whether congestion and whether it was
confirmed or not. Does this mean that (some amount of) sequence number anomalies
and/or delay indicates congestion? Is that defined somewhere?

Section 8.1, paragraph 7, comment:
>    If the feedback indicates that sequence number anomalies were
>    detected OR the delay range was above the upper threshold, the
>    offered load rate is decreased.  The RECOMMENDED values are 0 for

This doesn't agree with the previous paragraph, which says "Conversely, if
congestion has been previously confirmed, the offered load rate is only
increased by one (Rx+1)." (Or I don't understand the difference between
congestion and sequence number anomalies/delay ranges; see previous comment.)

Section 8.4, paragraph 2, comment:
>    This section is for the benefit of the Document Shepherd's form, and
>    will be deleted prior to final review.

Should this section have been deleted before the IESG review then? Would you
like the IESG to ignore it? You probably also need to instruct the RFC Editor to
remove it before publication?

This document uses RFC2119 keywords, but does not contain the recommended
RFC8174 boilerplate. (It contains some text with a similar beginning.)

-------------------------------------------------------------------------------
All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools, so there will likely be some false positives. There is no need
to let me know what you did with these suggestions.

Section 1, paragraph 5, nit:
-    authors found that a high percentage of paths tested support UDP
+    authors found that a high percentage of paths tested support the UDP
+                                                                ++++

Section 3, paragraph 8, nit:
-        to less traffic crossing ISP gateways in future.
+        to less traffic crossing ISP gateways in the future.
+                                                ++++

Section 8.1, paragraph 2, nit:
-    agressiveness (speed of ramp-up and down to the Maximum IP-Layer
+    aggressiveness (speed of ramp-up and down to the Maximum IP-Layer
+    +

Section 8.2, paragraph 5, nit:
-    Another approach comes from Section 24 of RFC 2544[RFC2544] and its
-                                              --------
+    Another approach comes from Section 24 of [RFC2544] and its

Section 8.3, paragraph 4, nit:
-    The path measured may be state-full based on many factors, and the
-                                  -  -
+    The path measured may be stateful based on many factors, and the

Section 14.2, paragraph 5, nit:
-    measurements with a clear and concise rate reduction when warrented,
-                                                                  ^
+    measurements with a clear and concise rate reduction when warranted,
+                                                                  ^

"I", paragraph 2, nit:
>  to the specifications of other Standards Development Organizations (SDO) (t
>                                ^^^^^^^^^
An apostrophe may be missing.

"S", paragraph 2, nit:
> t Maximum IP-Layer Capacity is unknown (which is sometimes the case; service
>                                        ^
Unpaired symbol: ')' seems to be missing

Section 6.3, paragraph 14, nit:
> urement systems MUST be higher than than the smallest of the links on the pa
>                                ^^^^^^^^^
Possible typo: you repeated a word

Section 6.3, paragraph 15, nit:
> PM list over all dt-length intervals in I. Measurements according to these de
>                                      ^^^^
Did you mean "in me", "in her", "in him", "in us", or "in them"?

Section 6.4, paragraph 2, nit:
>  the capacity Samples, RTD[T,I] and OWL[T,I]. The interval dtn,dtn+1 where M
>                                    ^^^^^
The word 'OWL[T' is not standard English. Did you mean "OWL'T" (curly
apostrophe) or "OWL'T" (straight apostrophe)?

Section 6.4, paragraph 3, nit:
> ting sub-interval within RTD[T,I] and OWL[T,I]. Other metrics MAY be measured
>                                      ^^^^^
The word 'OWL[T' is not standard English. Did you mean "OWL'T" (curly
apostrophe) or "OWL'T" (straight apostrophe)?

Section 8, paragraph 3, nit:
> work, since this is an active test method and it will likely cause congestion
>                                    ^^^^^^^^^^
Use a comma before 'and' if it connects two independent clauses (unless they
are closely connected and short).

"T", paragraph 8, nit:
> oad | | | largest value that | | size, bytes |
>          ^^^^^^^
A determiner is probably missing here: "the largest".

Section 8.2, paragraph 4, nit:
> etons (dt = 1 second) has proven to a be of practical value during tests of
>                                    ^^^^
After 'a', do not use a verb. Make sure that the spelling of 'be' is correct.
If 'be' is the first word in a compound adjective, use a hyphen between the two
words. Note: This error message can occur if you use a verb as a noun, and the
word is not a noun in standard English.

Section 8.3, paragraph 4, nit:
> nts based on those packets. Many different traffic shapers and on-demand acce
>                            ^^^^^^^^^^^^^^
Consider using "Many".

Section 8.3, paragraph 6, nit:
> , where packet losses may occur independently from the measurement sending ra
>                                ^^^^^^^^^^^^^^^^^^
The usual collocation for "independently" is "of", not "from". Did you mean
"independently of"?

Section 8.3, paragraph 18, nit:
> P- Layer Capacity measurements", [Y.Sup60], which is the result of continued
>                                    ^^^^^
Add a space between sentences

Section 10, paragraph 10, nit:
> ffic (for example, the range of I duration values SHOULD be limited). The exa
>                                ^^^^^^^^^^
Do not use a noun immediately after the pronoun 'I'. Use a verb or an adverb,
or possibly some other part of speech.

Section 12, paragraph 1, nit:
> s to Joachim Fabini, Matt Mathis, J.Ignacio Alvarez-Hamelin, Wolfgang Balzer
>                                    ^^^^^^^
Add a space between sentences

"R", paragraph 1, nit:
> ble) seqErr = 0 # Measured count of any of Loss or Reordering impairments de
>                                  ^^^^^^^^^
Consider simply using "of" instead.

Section 14.1, paragraph 5, nit:
> nning code [LS-SG12-A] [LS-SG12-B] [Y.Sup60]. Also, the UDP aspect of these m
>                                      ^^^^^
Add a space between sentences

Section 14.1, paragraph 10, nit:
> g a test when connectivity is lost for an longer interval (Feedback message o
>                                        ^^
Use "a" instead of 'an' if the following word doesn't start with a vowel sound,
e.g. 'a sentence', 'a university'.

Section 14.2, paragraph 10, nit:
> esting [LS-SG12-A] [LS-SG12-B] [Y.Sup60] supported this statement hundreds o
>                                  ^^^^^
Add a space between sentences

Section 14.2, paragraph 10, nit:
> easurements. A brief review of some of the other SHOULD-level requirements fo
>                                ^^^^^^^^^^^
If the text is a generality, 'of the' is not necessary.

Section 15.2, paragraph 16, nit:
> rec/T-REC-Y.1540-201912-I/en>. [Y.Sup60] Morton, A., "Recommendation Y.Sup60
>                                  ^^^^^
Add a space between sentences

Section 15.2, paragraph 16, nit:
> Y.Sup60] Morton, A., "Recommendation Y.Sup60, (09/20) Interpreting ITU
>                                        ^^^^^
Add a space between sentences

Section 15.2, paragraph 17, nit:
>  . Authors' Addresses Al Morton
>                                  ^^^^^
Add a space between sentences

"Authors' Addresses", paragraph 2, nit:
>  200 Laurel Avenue South Middletown,, NJ 07748 USA Phone: +1 732 420
>                                    ^^
Two consecutive commas

"Authors' Addresses", paragraph 5, nit:
>  200 Laurel Avenue South Middletown,, NJ 07748 USA Email: lencia@att
>                                    ^^
Two consecutive commas

[Ballot comment]
I reviewed the diff from -06 to -10, and did not see any security-relevant concerns.

Given that some use of terminology/symbols was updated, a fresh gen-art-like review
of the entire document for internal consistency might be warranted.

Note added 'This document received a DISCUSS from Magnus. After resolving that issue and going back through WGLC, this needs one more ballot to be approved. Zahed is reviewing the changes in light of Magnus's DISCUSS.'

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up. Changes are expected over time.

This version is dated 1 November 2019.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Proposed Standard, which is appropriate for a specification of a standard way to measure and report maximum IP-layer capacity. The status is indicated in the header.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:

  This memo revisits the problem of Network Capacity metrics first
  examined in RFC 5136.  The memo specifies a more practical Maximum
  IP-layer Capacity metric definition catering for measurement
  purposes, and outlines the corresponding methods of measurement.

Working Group Summary:

The working group has consensus to publish this document. There was no particular controversy in the working group regarding this document, but the group did provide useful and detailed feedback and review.

The IESG did provide feedback that led to substantial revisions, and a second working group last call on the revision had WG consensus.

Document Quality:

This document represents a standardized definition of concepts that have existed and been documented informationally previously (RFC 5136). This document covers the background and motivation for providing a clearer definition of the metric and method for IP capacity.

The document received detailed review from several (4+) involved working group members, which improved the quality and clarity of the document.

Personnel:

Tommy Pauly is the Document Shepherd. Martin Duke is the Responsible Area Director.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

The shepherd has reviewed the document since before its adoption, and has tracked the updates. The shepherd believes this version is ready to forward to the IESG. There are a few nits (see below), but those can be addressed with any AD review comments.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No concerns; the document has received a good amount of review, among the more detailed for an IPPM document for the time it was in the group.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No, the IPPM WG has the correct expertise for this topic.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Confirmed.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No IPR disclosures.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

The consensus for this document represented various different communities in IPPM, and was a strong consensus.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

There is one nit around updating the boilerplate for RFC2119 language.

There are also several normative references to Informational RFCs that should be updated to informative.
RFC 2330
RFC 2544
RFC 3148
RFC 5136
RFC 6815
RFC 7312
RFC 7594
RFC 7799
RFC 8337
RFC 8468

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

Not applicable.

(13) Have all references within this document been identified as either normative or informative?

Yes, but some need updating (see above).

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No.

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

Yes (see above). These likely should be updated.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No status changes.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 8126).

No IANA Considerations.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

None.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, YANG modules, etc.

None applicable.

(20) If the document contains a YANG module, has the module been checked with any of the recommended validation tools (https://trac.ietf.org/trac/ops/wiki/yang-review-tools) for syntax and formatting validation? If there are any resulting errors or warnings, what is the justification for not fixing them at this time? Does the YANG module comply with the Network Management Datastore Architecture (NMDA) as specified in RFC8342?

No YANG module.

[Ballot comment]
Since a SHOULD leaves an implementer with a choice, it's preferable to see prose explaining why one might deviate from the SHOULD advice.  Thus, the SHOULDs in Sections 5.3 and 6.3 leave me wondering under what circumstances an implementer might legitimately choose to do something else.  If there are none, should it be a MUST?

[Ballot discuss]
A) Section 8. Method of Measurement

I think the metrics are fine, what makes me quite worried here is the measurement method. My concerns with it are the following.

1. The application of this measurement method is not clearly scoped. Therefore I will assume that across the Internet measurements are possible. However in that context I think the definition and protection against severe congestion has significant short comings. The main reason is that the during a configurable time period (default 1 s) the sender will attempt to send at a specified rate by a table independently on what happens during that second.

2. The algorithm for adjusting rate is table driven but give no guidance on how to construct the table and limitations on value changes in the table. In addition the algorithm discusses larger steps in the table without any reflection of what theses steps sides may represent in offered load.

3. Third the algorithms reaction to any sequence number gaps is dependent on delay and how it is related to unspecified delay thresholds. Also no text discussion how these thresholds should be configured for safe operation.

B) Section 8. Method of Measurement

There are no specification of the measurement protocol here that provides sequence numbers, and the feedback channel as well as the control channel. Is this intended to use TWAMP?

From my perspective this document defines the metrics on standards track level. However, the method for actually running the measurements are not specified on a standards track level. No one can build implementation. And if the section is intended to provide requirements on a protocol that performs these measurements I think several aspects are missing. There appear several ways forward here to resolve this; one is to split out the method of measurement and define it separately to standard tracks level using a particular protocol, another is to write it purely as requirements on a measurement protocols.

[Ballot comment]
I'm stealing Robert's ballot text, because it perfectly explains my position/views:
"Thank you for this document, I found it interesting to read.

I have no comments that haven't already been raised by other AD reviews.
"

[Ballot comment]
Thank you for this document, I found it interesting to read.

I have no comments that haven't already been raised by other AD reviews.

[Ballot discuss]
Several of the SHOULDs in this document are giving me trouble.  There are two main categories, one in particular I'd like to discuss:

As described in RFC 2119, we typically use this sort of language to describe interoperability or security concerns.  How are the SHOULDs in Section 9 related to interoperability or security?  Rather, they seem to be describing issues of presentation.  Or is this another instance of the "operational advice" exception that's come up on other operational advice documents?

[Ballot comment]
Similar to my DISCUSS POINT:

Since a SHOULD leaves an implementer with a choice, it's preferable to see prose explaining why one might deviate from the SHOULD advice.  Thus, the SHOULDs in Sections 5.3 and 6.3 leave me wondering under what circumstances an implementer might legitimately choose to do something else.  If there are none, should it be a MUST?

[Ballot discuss]
Several of the SHOULDs in this document are giving me trouble.  There are two categories in particular:

As described in RFC 2119, we typically use this sort of language to describe interoperability or security concerns.  How are the SHOULDs in Section 9 related to interoperability or security?  Rather, they seem to be describing issues of presentation.

Since a SHOULD leaves an implementer with a choice, it's preferable to see prose explaining why one might deviate from the SHOULD advice.  Thus, the SHOULDs in Sections 5.3 and 6.3 leave me wondering under what circumstances an implementer might legitimately choose to do something else.  If there are none, should it be a MUST?

[Ballot comment]
Section 3

  5.  Less emphasis on ISP gateway measurements, possibly due to less
      traffic crossing ISP gateways in future.

nit: sentence fragment.

Section 5

  This section sets requirements for the following components to
  support the Maximum IP-layer Capacity Metric.

editorial/nit: I don't think I understand what "the following
components" are.  Is this referencing some preexisting template for a
metric definition?  (Same for §6 and §7.)

Section 5.3

  The number of these IP-layer bits is designated n0[dtn,dtn+1] for a
  specific dt.

(editorial) I'm a little confused by this notation for n0.  In Section 4
we say that "dtn" references a specific sub-interval, but the brackets
and two items look like they should be the start an end of an interval,
for which perhaps just the 'n' would be more appropriate (but we'd want
a half-open interval, and would need to worry about 0- vs 1-indexing for
n, etc.).

  Anticipating a Sample of Singletons, the interval dt SHOULD be set to
  a natural number m so that T+I = T + m*dt with dtn+1 - dtn = dt and
  with 1 <= n <= m.

nit: "dt [...] set to m" doesn't make any sense.
nit: this looks like more of a "for 1 <= n <= m" than an "and with".

  Mathematically, this definition can be represented as:

                                      ( n0[dtn,dtn+1] )
                      C(T,dt,PM) = -------------------------
                                            dt

(nit) the "n" appears (in 'dtn') only on the right side but there is no
operation applied over it, so this "equation" seems unbalanced without
also specifying 'n'.  I think the introductory text should mention
something about "for each n" or for a given interval dtn" or similar.

  o  n0 is the total number of IP-layer header and payload bits that
      can be transmitted in Standard Formed packets [RFC8468] from the

(nit) RFC 8468 spells it "standard-formed".

  o  C(T,dt,PM) the IP-Layer Capacity, corresponds to the value of n0
      measured in any sub-interval ending at dtn (meaning T + n*dt),
      divided by the length of sub-interval, dt.

If it's supposed to be "ending at dtn", then why is the "dtn+1" in the
picture at all?

  o  PM represents other performance metrics [see section 5.4 below];
      their measurement results SHALL be collected during measurement of
      IP-layer Capacity and associated with the corresponding dtn for
      further evaluation and reporting.

(nit) this seems to duplicate (be a subset of) the paragraph before
"Mathematically, this definition can be represented".

  o  The bit rate of the physical interface of the measurement device
      must be higher than that of the link whose C(T,I,PM) is to be
      measured.

(nit) I thought we were measuring a path, not a link.

Section 5.4

  RTD[dtn-1,dtn] is defined as a sample of the [RFC2681] Round-trip

Do we really want to be using n0[dtn,dtn+1] but RTD[dtn-1,dtn]?  Can we
pick a consistent sub-interval notation?

Section 6.3

  Define the Maximum IP-layer capacity, Maximum_C(T,I,PM), to be the
  maximum number of IP-layer bits n0[dtn,dtn+1] that can be transmitted
  in packets from the Src host and correctly received by the Dst host,

(nit) the relevant formulae include a dt divisor, but I don't see
anything in this prose that would correspond to such a divisor.

  The interval dt SHOULD be set to a natural number m so that T+I = T +
  m*dt with dtn+1 - dtn = dt and with 1 <= n <= m.

nit: "dt [...] set to m" doesn't make any sense.
nit: this looks like more of a "for 1 <= n <= m" than an "and with".

  Mathematically, this definition can be represented as:

                                      max  ( n0[dtn,dtn+1] )
                                    [T,T+I]
                Maximum_C(T,I,PM) = -------------------------
                                              dt
              where:
                  T                                      T+I
                  _________________________________________
                  |  |  |  |  |  |  |  |  |  |  |
              dtn=1  2  3  4  5  6  7  8  9  10  n+1
                                                    n=m

(nit) as mentioned previously, the definition of "dtn" lists it as being
the sub-interval, not the boundary point/time of a sub-interval.

Section 6.5

  If traffic conditioning (e.g., shaping, policing) applies along a
  path for which Maximum_C(T,I,PM) is to be determined, different
  values for dt SHOULD be picked and measurements be executed during
  multiple intervals [T, T+I].  A single constant interval dt SHOULD be
  chosen so that is an integer multiple of increasing values k times
  serialisation delay of a path MTU at the physical interface speed
  where traffic conditioning is expected.  [...]

nit: "so that is an integer multiple" seems to be missing a word.

Also, this seems to say that different values for dt SHOULD be picked,
but also that a constant dt SHOULD be chosen.  How can those both be
recommended and be consistent with each other?  (I mean, I assume that
the intent is to multiple runs with different (fixed) dt, but that's not
what the text seems to say.)

Section 7.3

  Define the IP-layer Sender Bit Rate, B(S,st), to be the number of IP-
  layer bits (including header and data fields) that are transmitted
  from the Source during one contiguous sub-interval, st, during the
  test interval S (where S SHALL be longer than I), and where the
  fixed-size packet count during that single sub-interval st also
  provides the number of IP-layer bits in any interval: n0[stn-1,stn].

(1) there doesn't seem to be any restriction that the observed packets
list Dst as the destination address, so formally it seems this would
count *all* traffic generated by Sender, not just the traffic relevant
for the (path capacity) test.
(2) It seems a little unfortunate that we reuse the 'n0' symbol here for
a different meaning than in the earlier capacity metrics.

  Measurements according to these definitions SHALL use the UDP
  transport layer.  Any feedback from Dst host to Src host received by
  Src host during an interval [stn-1,stn] MUST NOT result in an
  adaptation of the Src host traffic conditioning during this interval
  (rate adjustment occurs on st boundaries).

Hmm, this "MUST NOT" is interesting, as it seems to imply extremely
tight coordination between the measurement point for this metric and the
Source itself.  (Note that the toplevel §7 admits the possibility that
measurement will occur at a location other than the Src host to network
path interface, via "(or as close as practical)".)

Section 8.1

  At the beginning of a test, the sender begins sending at rate R1 and
  the receiver starts a feedback timer at interval F (while awaiting

It's a little hard to search for, but I didn't find any previous mention
of 'F' or it being defined as a parameter or term.  Should it be a
listed parameter somewhere?

  If the feedback indicates that sequence number anomalies were
  detected OR the delay range was above the upper threshold, the
  offered load rate is decreased.  Also, if congestion is now confirmed
  by the current feedback message being processed, then the offered
  load rate is decreased by more than one rate (e.g., Rx-30).  [...]

Does "congestion is now confirmed" mean that "congestion confirmed" is
like a one-way latch and this transition only occurs at most once over
the course of a test?  Or could the Rx-30 happen multiple times?
(The pseudocode indicates the former.)

  If the feedback indicates that there were no sequence number
  anomalies AND the delay range was above the lower threshold, but
  below the upper threshold, the offered load rate is not changed.

The way this is written suggests that there will always be a lower and
an upper threshold for delay, but the rest of the document so far didn't
give me that impression.  E.g., we talk about PM only as "at least one
fundamental metric and target performance threshold MUST be supplied",
and to me having both upper and lower thresholds would be two
thresholds, not one.

Section 8.2

  Here, as with any Active Capacity test, the test duration must be
  kept short. 10 second tests for each direction of transmission are
  common today.  The default measurement interval specified here is I =
  10 seconds).  In combination with a fast search method and user-
  network coordination, the concerns raised in RFC 6815[RFC6815] are
  alleviated.  [...]

I skimmed RFC 6815 and had a bit of a hard time making the connection
for why combining a 10-second interval, fast search method, and
user-network coordination alleviate the concerns of RFC 6815.  There
doesn't seem to be much in 6815 itself about how testing in production
can be done safely, so my current working assumption is that the
conclusion presented here reflects the results of "new work" being
recorded for the first time (in the RFC series) in this document.  If
that assumption is correct, I'd suggest spending some more words to
support the conclusion, e.g., making analogies to other "normal" traffic
patterns and how the benchmarking setup is not qualitatively different
from them.

Section 8.3

  As testing continues, implementers should expect some evolution in
  the methods.  The ITU-T has published a Supplement (60) to the
  Y-series of Recommendations, "Interpreting ITU-T Y.1540 maximum IP-
  layer capacity measurements", [Y.Sup60], which is the result of
  continued testing with the metric and method described here.

I pulled up the [Y.Sup60] reference, and it does not seem to reference
this draft by name.  On what basis do we conclude that it "is the result
of continued testing with the metric and method described here"?
Skimming/searching, I do see many similar formulae and methods
presented, but how do we conclude they are precisely the same?

Section 10

Should we say something about making sure that I is reasonably bounded?
IIRC we say so elsewhere in the text but not exactly here.

  2.  A REQUIRED user client-initiated setup handshake between
      cooperating hosts and allows firewalls to control inbound
      unsolicited UDP which either go to a control port [expected and
      w/authentication] or to ephemeral ports that are only created as
      needed.  [...]

nit: the grammar is odd in the first part of this sentence; the part
before the "and" doesn't seem like it can join up with anything after
the "and".  Is the intent something like "It is REQUIRED to have a user
client-initiated setup handshake between cooperating hosts that allows
firewalls to [...]"?

  3.  Integrity protection for feedback messages conveying measurements
      is RECOMMENDED.

(In some sense you want authentication as well as integrity protection.)

  5.  Senders MUST be rate-limited.  This can be accomplished using the
      pre-built table defining all the offered load rates that will be
      supported (Section 8.1).  The recommended load-control search
      algorithm results in "ramp up" from the lowest rate in the table.

nit: since (effectively) each implementation will have their own
pre-built table, I think it should be "using a pre-built table".

Appendix 13

If we start at Rx (row) 1, is it going to cause problems when we drop
down to Rx = 0 in the loss/congestion cases?

The mechcanism in the pseudocode to stop taking large increments in
sending rate above the "hSpeedThresh" does not seem to be described in
the prose in §8.1.  (That said, it seems like a good idea, given the
likely table composition.)

(Also, indenting one tab for the outer conditionals and two more for the
inner ones looks a bit unusual.)

Section 14

It's not entirely clear to me why RFC 2330 is classified as normative
but RFC 7312 is informative, just based on the locations where they are
referenced.

[Ballot comment]
[[ comments ]]

[ section 4 ]

* RFC 6438 isn't only about flow label treatment at Tunnel End Points, since
  other devices in the network can do ECMP where the flow label is part of
  the flow.

  Maybe just a full stop after "when routers have complied with RFC6438
  guidelines."?


[[ questions ]]

[ sections 5.6/6.6/7.4 ]

* Should the "megabit = 1,000,000 bits" text from section 6.6 be also used
  in sections 5.6 and 7.4, or even called out separately earlier on?

  (Maybe it's just my own experience, but I've found that, while "mibi" 100%
  of the time means base 2, "mega" only mostly means base 10 and occasionally
  can still be interpreted as base 2.)


[[ nits ]]

[ section 2 ]

* "Also, to foster the development..."

  This appears to be a sentence fragment rather than a grammatically correct
  sentence.  I assume the point is to say that fostering the development ...
  is also goal of the document.

* "The supporting" -> "Supporting the", might read more naturally?

[ section 5.4 ]

* "The statistics used to to summarize" -> "The statistics used to summarize"

[ section 6.5 ]

* "so that is" -> "so that it is"?

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has reviewed draft-ietf-ippm-capacity-metric-method-05, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist

The following Last Call announcement was sent out (ends 2021-02-16):

From: The IESG
To: IETF-Announce
CC: Ian Swett , draft-ietf-ippm-capacity-metric-method@ietf.org, ippm-chairs@ietf.org, ippm@ietf.org, martin.h.duke@gmail.com, tpauly@apple.com
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Metrics and Methods for One-way IP Capacity) to Proposed Standard


The IESG has received a request from the IP Performance Measurement WG (ippm)
to consider the following document: - 'Metrics and Methods for One-way IP
Capacity'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2021-02-16. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  This memo revisits the problem of Network Capacity metrics first
  examined in RFC 5136.  The memo specifies a more practical Maximum
  IP-layer Capacity metric definition catering for measurement
  purposes, and outlines the corresponding methods of measurement.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-ippm-capacity-metric-method/



No IPR declarations have been submitted directly on this I-D.


The document contains these normative downward references.
See RFC 3967 for additional information:
    rfc7799: Active and Passive Metrics and Methods (with Hybrid Types In-Between) (Informational - IETF stream)
    rfc7312: Advanced Stream and Sampling Framework for IP Performance Metrics (IPPM) (Informational - IETF stream)
    rfc6815: Applicability Statement for RFC 2544: Use on Production Networks Considered Harmful (Informational - IETF stream)
    rfc5136: Defining Network Capacity (Informational - IETF stream)
    rfc3148: A Framework for Defining Empirical Bulk Transfer Capacity Metrics (Informational - IETF stream)
    rfc2544: Benchmarking Methodology for Network Interconnect Devices (Informational - Legacy stream)
    rfc8337: Model-Based Metrics for Bulk Transport Capacity (Experimental - IETF stream)
    rfc8468: IPv4, IPv6, and IPv4-IPv6 Coexistence: Updates for the IP Performance Metrics (IPPM) Framework (Informational - IETF stream)

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up. Changes are expected over time.

This version is dated 1 November 2019.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Proposed Standard, which is appropriate for a specification of a standard way to measure and report maximum IP-layer capacity. The status is indicated in the header.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:

  This memo revisits the problem of Network Capacity metrics first
  examined in RFC 5136.  The memo specifies a more practical Maximum
  IP-layer Capacity metric definition catering for measurement
  purposes, and outlines the corresponding methods of measurement.

Working Group Summary:

The working group has consensus to publish this document. There was no particular controversy in the working group regarding this document, but the group did provide useful and detailed feedback and review.

Document Quality:

This document represents a standardized definition of concepts that have existed and been documented informationally previously (RFC 5136). This document covers the background and motivation for providing a clearer definition of the metric and method for IP capacity.

The document received detailed review from several (4+) involved working group members, which improved the quality and clarity of the document.

Personnel:

Tommy Pauly is the Document Shepherd. Martin Duke is the Responsible Area Director.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

The shepherd has reviewed the document since before its adoption, and has tracked the updates. The shepherd believes this version is ready to forward to the IESG. There are a few nits (see below), but those can be addressed with any AD review comments.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No concerns; the document has received a good amount of review, among the more detailed for an IPPM document for the time it was in the group.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No, the IPPM WG has the correct expertise for this topic.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Confirmed.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No IPR disclosures.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

The consensus for this document represented various different communities in IPPM, and was a strong consensus.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

There is one nit around updating the boilerplate for RFC2119 language.

There are also several normative references to Informational RFCs that should be updated to informative.
RFC 2330
RFC 2544
RFC 3148
RFC 5136
RFC 6815
RFC 7312
RFC 7594
RFC 7799
RFC 8337
RFC 8468

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

Not applicable.

(13) Have all references within this document been identified as either normative or informative?

Yes, but some need updating (see above).

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No.

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

Yes (see above). These likely should be updated.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No status changes.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 8126).

No IANA Considerations.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

None.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, YANG modules, etc.

None applicable.

(20) If the document contains a YANG module, has the module been checked with any of the recommended validation tools (https://trac.ietf.org/trac/ops/wiki/yang-review-tools) for syntax and formatting validation? If there are any resulting errors or warnings, what is the justification for not fixing them at this time? Does the YANG module comply with the Network Management Datastore Architecture (NMDA) as specified in RFC8342?

No YANG module.

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up. Changes are expected over time.

This version is dated 1 November 2019.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Proposed Standard, which is appropriate for a specification of a standard way to measure and report maximum IP-layer capacity. The status is indicated in the header.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:

  This memo revisits the problem of Network Capacity metrics first
  examined in RFC 5136.  The memo specifies a more practical Maximum
  IP-layer Capacity metric definition catering for measurement
  purposes, and outlines the corresponding methods of measurement.

Working Group Summary:

The working group has consensus to publish this document. There was no particular controversy in the working group regarding this document, but the group did provide useful and detailed feedback and review.

Document Quality:

This document represents a standardized definition of concepts that have existed and been documented informationally previously (RFC 5136). This document covers the background and motivation for providing a clearer definition of the metric and method for IP capacity.

The document received detailed review from several (4+) involved working group members, which improved the quality and clarity of the document.

Personnel:

Tommy Pauly is the Document Shepherd. Martin Duke is the Responsible Area Director.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

The shepherd has reviewed the document since before its adoption, and has tracked the updates. The shepherd believes this version is ready to forward to the IESG. There are a few nits (see below), but those can be addressed with any AD review comments.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No concerns; the document has received a good amount of review, among the more detailed for an IPPM document for the time it was in the group.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No, the IPPM WG has the correct expertise for this topic.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Confirmed.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No IPR disclosures.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

The consensus for this document represented various different communities in IPPM, and was a strong consensus.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

There is one nit around updating the boilerplate for RFC2119 language.

There are also several normative references to Informational RFCs that should be updated to informative.
RFC 2330
RFC 2544
RFC 3148
RFC 5136
RFC 6815
RFC 7312
RFC 7594
RFC 7799
RFC 8337
RFC 8468

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

Not applicable.

(13) Have all references within this document been identified as either normative or informative?

Yes, but some need updating (see above).

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No.

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

Yes (see above). These likely should be updated.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No status changes.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 8126).

No IANA Considerations.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

None.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, YANG modules, etc.

None applicable.

(20) If the document contains a YANG module, has the module been checked with any of the recommended validation tools (https://trac.ietf.org/trac/ops/wiki/yang-review-tools) for syntax and formatting validation? If there are any resulting errors or warnings, what is the justification for not fixing them at this time? Does the YANG module comply with the Network Management Datastore Architecture (NMDA) as specified in RFC8342?

No YANG module.