Rate Measurement Test Protocol Problem Statement and Requirements
draft-ietf-ippm-rate-problem-10

[Ballot comment]
- Thanks for the "Operational Considerations" new section, as discussed with D. Romascanu

- The document tile says problem statement, but it's a mix of problem statement, use cases, and some protocol requirements with RFC 2119 keywords. I believe the title should be modified accordingly.

- I agree with Stephen regarding the use of RFC 2119.

  Service subscribers and
  authorized users SHOULD obtain their network operator's or service
  provider's permission before conducting tests.

Really, should I have my provider consent before running http://www.speedtest.net/ ? :-)

-

  Current IETF standardized test protocols do not
  possess the asymmetric size generation capability with two-way
  testing.

Do you mean OWAMP/TWAMP? Please be specific

[Ballot comment]
I'll follow along on Stephen's comments and don't have any others to add.  The draft is well-written, thank you for your work on it.

[Ballot comment]

- I'm not clear why it's a good plan for this document to
state a bunch of MUST level requirements on protocol
specifications, but I assume if that were problematic then
this informational document would be ignored and if it's not
a problem that's fine:-). Including that kind of thing can
lead to unnecessary arguments though so I wondered. For
example, on page 8 it (I think) says that protocols MUST
allow for control (by whom?) of payload lengths, which is not
something I see in typical speed test tools that'd be used by
massive numbers of users, or at least that's not exposed via
a UI that I can see.

- section 6: surely this needs to note the problem that there
can be privacy issues with (esp. multiple uses) of
measurement - e.g. if I use my phone in different places to
do similar measurements I may be open to some new forms of
tracking. Why not note this? Why is there no "MUST NOT"
requirement from this?  (Assuming you stick with including
such.) RFC2330 does recognise privacy for example, so why not
here too? BTW I don't think I see privacy issues considered
in 4656 or 5357, but I only had a very quick look. (This
would be a DISCUSS from me if this were a protocol document,
but I'm ok to just leave it as a comment on one like this.)

[Ballot comment]
I have no objection to the publication of this document, but I did
have one observation:


Although section 2 has discussion of the variation of a number of
fields and specifically includes...

  o  Transport protocol (e.g. where TCP packets may be routed
      differently from UDP)

...I was surprised to find no mention of ECMP and the consequences that
it may introduce for the reliability of rate measurement figures.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet
Standard, Informational, Experimental, or Historic)? Why is this the proper
type of RFC? Is this type of RFC indicated in the title page header?

Informational

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved documents.
The approval announcement contains the following sections:

Technical Summary:

This memo presents an access rate-measurement problem statement for test
protocols to measure IP Performance Metrics.  The rate measurement scenario
has wide-spread attention of Internet access subscribers and seemingly all
industry players, including regulators. Key test protocol aspects require
the ability to control packet size on the tested path and enable
asymmetrical packet size testing in a controller-responder architecture.

Working Group Summary:

A controversial topic regards the level of requirement for the capability
to control asymmetric packet sizes in two-way testing architectures. A few
participants felt that the document should not require asymmetric packet
sizes, even though there are a number of cases where it would appear
asymmetric packets sizes would be essential to measure the path capacity
accurately.

As a compromise, the author updated the document to require asymmetric
rate generation, and both symmetric and asymmetric packet sizes are
recommended. The many circumstances where asymmetric packet size testing is
needed are documented, and many of these circumstances would be unknown
prior to comparative tests using symmetric and asymmetric packet sizes.

Document Quality:

The draft was reviewed by many members of WG. There are already standards
track protocol extension proposals that respond to the requirements in
this draft. At least one protocol solution has been implemented and
deployed.

Personnel:

The document shepherd is Bill Cerveny. The responsible area director is
Spencer Dawkins.

(3) Briefly describe the review of this document that was performed by the
Document Shepherd. If this version of the document is not ready for
publication, please explain why the document is being forwarded to the IESG.

It is felt that this document is sufficiently ready for publication. The
document has been a working group document for over two years. There have
been two working group last calls on this document, and there has been
considerable discussion and revision over the life of this working group
Internet-Draft.

(4) Does the document Shepherd have any concerns about the depth or breadth
of the reviews that have been performed?

No. There have been numerous reviews by the working group participants.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that took
place.

No

(6) Describe any specific concerns or issues that the Document Shepherd has
with this document that the Responsible Area Director and/or the IESG
should be aware of? For example, perhaps he or she is uncomfortable with
certain parts of the document, or has concerns whether there really is a
need for it. In any event, if the WG has discussed those issues and has
indicated that it still wishes to advance the document, detail those
concerns here.

See "Working Group Summary" above.

(7) Has each author confirmed that any and all appropriate IPR disclosures
required for full conformance with the provisions of BCP 78 and BCP 79 have
already been filed. If not, explain why?

There are no IPR disclosures.

(8) Has an IPR disclosure been filed that references this document? If so,
summarize any WG discussion and conclusion regarding the IPR disclosures.

There are no IPR disclosures

(9) How solid is the WG consensus behind this document? Does it represent
the strong concurrence of a few individuals, with others being silent, or
does the WG as a whole understand and agree with it?

There exists a rough consensus for moving this document forward. Over the
life of the working group document, there has been significant discussion. 
There were 3 commenters in the second and last working group last call.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate email
messages to the Responsible Area Director. (It should be in a separate
email because this questionnaire is publicly available.)

There haven't been any threats of appeals, but the chairs did declare
rough consensus based on 1-1/2 years of fairly intense working group
discussion on the point described in the working group summary.

(11) Identify any ID nits the Document Shepherd has found in this document.
(See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist).
Boilerplate checks are not enough; this check needs to be thorough.

idnits 2.13.01

tmp/draft-ietf-ippm-rate-problem-07.txt:

  Checking boilerplate required by RFC 5378 and the IETF Trust (see
  http://trustee.ietf.org/license-info):
  ----------------------------------------------------------------------------

    No issues found here.

  Checking nits according to http://www.ietf.org/id-info/1id-guidelines.txt:
  ----------------------------------------------------------------------------

    No issues found here.

  Checking nits according to http://www.ietf.org/id-info/checklist :
  ----------------------------------------------------------------------------

    No issues found here.

  Miscellaneous warnings:
  ----------------------------------------------------------------------------

  -- The document date (October 7, 2014) is 34 days in the past.  Is this
    intentional?


  Checking references for intended status: Informational
  ----------------------------------------------------------------------------

  == Outdated reference: A later version (-07) exists of
    draft-ietf-ippm-lmap-path-06


    Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--).

    Run idnits with the --verbose option for more detailed information about
    the items above.

(12) Describe how the document meets any required formal review criteria,
such as the MIB Doctor, media type, and URI type reviews.

None necessary.

(13) Have all references within this document been identified as either
normative or informative?

Yes

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

No

(15) Are there downward normative references references (see RFC 3967)? If
so, list these downward references to support the Area Director in the Last
Call procedure.

No

(16) Will publication of this document change the status of any existing
RFCs? Are those RFCs listed on the title page header, listed in the
abstract, and discussed in the introduction? If the RFCs are not listed in
the Abstract and Introduction, explain why, and point to the part of the
document where the relationship of this document to the other RFCs is
discussed. If this information is not in the document, explain why the WG
considers it unnecessary.

No

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes are
associated with the appropriate reservations in IANA registries. Confirm
that any referenced IANA registries have been clearly identified. Confirm
that newly created IANA registries include a detailed specification of the
initial contents for the registry, that allocations procedures for future
registrations are defined, and a reasonable name for the new registry has
been suggested (see RFC 5226).

This memo makes no requests of IANA.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find useful in
selecting the IANA Experts for these new registries.

This memo makes no requests of IANA.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal language,
such as XML code, BNF rules, MIB definitions, etc.

None necessary.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Rate Measurement Test Protocol Problem Statement) to Informational RFC


The IESG has received a request from the IP Performance Metrics WG (ippm)
to consider the following document:
- 'Rate Measurement Test Protocol Problem Statement'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2014-12-22. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  This memo presents an access rate-measurement problem statement for
  test protocols to measure IP Performance Metrics.  The rate
  measurement scenario has wide-spread attention of Internet access
  subscribers and seemingly all industry players, including regulators.
  Key test protocol aspects require the ability to control packet size
  on the tested path and enable asymmetrical packet size testing in a
  controller-responder architecture.





The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-ippm-rate-problem/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-ippm-rate-problem/ballot/


No IPR declarations have been submitted directly on this I-D.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet
Standard, Informational, Experimental, or Historic)? Why is this the proper
type of RFC? Is this type of RFC indicated in the title page header?

Informational

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved documents.
The approval announcement contains the following sections:

Technical Summary:

This memo presents an access rate-measurement problem statement for test
protocols to measure IP Performance Metrics.  The rate measurement scenario
has wide-spread attention of Internet access subscribers and seemingly all
industry players, including regulators. Key test protocol aspects require
the ability to control packet size on the tested path and enable
asymmetrical packet size testing in a controller-responder architecture.

Working Group Summary:

A controversial topic regards the level of requirement for the capability
to control asymmetric packet sizes in two-way testing architectures. A few
participants felt that the document should not require asymmetric packet
sizes, even though there are a number of cases where it would appear
asymmetric packets sizes would be essential to measure the path capacity
accurately.

As a compromise, the author updated the document to require asymmetric
packet rate generation, and both symmetric and asymmetric packet sizes are
recommended. The many circumstances where asymmetric packet size testing is
needed are documented, and many of these circumstances would be unknown
prior to comparative tests using symmetric and asymmetric packet sizes.

Document Quality:

The draft was reviewed by many members of WG. There are already standards
track protocol extension proposals that respond to the requirements in
this draft. At least one protocol solution has been implemented and
deployed.

Personnel:

The document shepherd is Bill Cerveny. The responsible area director is
Spencer Dawkins.

(3) Briefly describe the review of this document that was performed by the
Document Shepherd. If this version of the document is not ready for
publication, please explain why the document is being forwarded to the IESG.

It is felt that this document is sufficiently ready for publication. The
document has been a working group document for over two years. There have
been two working group last calls on this document, and there has been
considerable discussion and revision over the life of this working group
Internet-Draft.

(4) Does the document Shepherd have any concerns about the depth or breadth
of the reviews that have been performed?

No. There have been numerous reviews by the working group participants.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that took
place.

No

(6) Describe any specific concerns or issues that the Document Shepherd has
with this document that the Responsible Area Director and/or the IESG
should be aware of? For example, perhaps he or she is uncomfortable with
certain parts of the document, or has concerns whether there really is a
need for it. In any event, if the WG has discussed those issues and has
indicated that it still wishes to advance the document, detail those
concerns here.

See "Working Group Summary" above.

(7) Has each author confirmed that any and all appropriate IPR disclosures
required for full conformance with the provisions of BCP 78 and BCP 79 have
already been filed. If not, explain why?

There are no IPR disclosures.

(8) Has an IPR disclosure been filed that references this document? If so,
summarize any WG discussion and conclusion regarding the IPR disclosures.

There are no IPR disclosures

(9) How solid is the WG consensus behind this document? Does it represent
the strong concurrence of a few individuals, with others being silent, or
does the WG as a whole understand and agree with it?

There exists a rough consensus for moving this document forward. Over the
life of the working group document, there has been significant discussion. 
There were 3 commenters in the second and last working group last call.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate email
messages to the Responsible Area Director. (It should be in a separate
email because this questionnaire is publicly available.)

There haven't been any threats of appeals, but the chairs did declare
rough consensus based on 1-1/2 years of fairly intense working group
discussion on the point described in the working group summary.

(11) Identify any ID nits the Document Shepherd has found in this document.
(See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist).
Boilerplate checks are not enough; this check needs to be thorough.

idnits 2.13.01

tmp/draft-ietf-ippm-rate-problem-07.txt:

  Checking boilerplate required by RFC 5378 and the IETF Trust (see
  http://trustee.ietf.org/license-info):
  ----------------------------------------------------------------------------

    No issues found here.

  Checking nits according to http://www.ietf.org/id-info/1id-guidelines.txt:
  ----------------------------------------------------------------------------

    No issues found here.

  Checking nits according to http://www.ietf.org/id-info/checklist :
  ----------------------------------------------------------------------------

    No issues found here.

  Miscellaneous warnings:
  ----------------------------------------------------------------------------

  -- The document date (October 7, 2014) is 34 days in the past.  Is this
    intentional?


  Checking references for intended status: Informational
  ----------------------------------------------------------------------------

  == Outdated reference: A later version (-07) exists of
    draft-ietf-ippm-lmap-path-06


    Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--).

    Run idnits with the --verbose option for more detailed information about
    the items above.

(12) Describe how the document meets any required formal review criteria,
such as the MIB Doctor, media type, and URI type reviews.

None necessary.

(13) Have all references within this document been identified as either
normative or informative?

Yes

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

No

(15) Are there downward normative references references (see RFC 3967)? If
so, list these downward references to support the Area Director in the Last
Call procedure.

No

(16) Will publication of this document change the status of any existing
RFCs? Are those RFCs listed on the title page header, listed in the
abstract, and discussed in the introduction? If the RFCs are not listed in
the Abstract and Introduction, explain why, and point to the part of the
document where the relationship of this document to the other RFCs is
discussed. If this information is not in the document, explain why the WG
considers it unnecessary.

No

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes are
associated with the appropriate reservations in IANA registries. Confirm
that any referenced IANA registries have been clearly identified. Confirm
that newly created IANA registries include a detailed specification of the
initial contents for the registry, that allocations procedures for future
registrations are defined, and a reasonable name for the new registry has
been suggested (see RFC 5226).

This memo makes no requests of IANA.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find useful in
selecting the IANA Experts for these new registries.

This memo makes no requests of IANA.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal language,
such as XML code, BNF rules, MIB definitions, etc.

None necessary.