Model-Based Metrics for Bulk Transport Capacity
draft-ietf-ippm-model-based-metrics-13

[Ballot comment]
The abstract is overly long (long enough to run onto the second page.) That seems to detract from the purpose of an abstract. Can it be shortened to a paragraph or two?

[Ballot comment]
High level comment:
I still (compared to an earlier review I did during the first wg last call) think that the document could benefit from a mayor editorial pass that aims to remove redundancy (especially in section 8) and hopefully would lead to a much shorter document in the end. Especially the split between section 6, 7 and 8 is not always clear. It would actually be nice if the general model aspects and its applicability to one traffic class with the assumption that self-clocked, loss/CE-based congestion control is used would be separated more clearly. This would also help to separate generally reasoning from the test and evaluation instructions and make those more clear and understandable (in the sense that concrete implementation guidelines for both the tests and the evaluation seem quite hidden between all the other text at the moment). However, I understand that this is a lot of work and probably not appropriate at this state of processing.

Processing comment:
The references are not split into normative and informative. I guess most references including draft-ietf-tcpm-rack are informative. Indicating this clearly in two separate reference sections would also address the editor comment in 1.1 regarding draft-ietf-tcpm-rack.

Various, mostly editorial comments:
- The terminology mentions open loop. I would recommend to also note in the intro that these are open loop tests (compared to using TCP traffic directly).
- ECN should be spelled out somewhere and RFC3168 should be referred (informatively).
- It seems that the currently specified metrics and tests assume ack-clocked and loss/ECN-based congestion control. This might also worth stating in the intro given quic is experimenting with rate-based schemes.
- Regarding the list at the end of section 4.1 on bursts: Shouldn't the initial window (which is usually today more than 4 packets) be mentioned additionally?
- Network power could be defined in the terminology section.
- It might also be useful to spell out more clearly in the intro that for these tests both endpoints (of the subpath under test) need to be under control of the tester.
- As mentioned several times in the document, some of the tests are not intended for frequent monitoring tests as the high load can impact other traffic negatively. This should be re-stated clearly  in the security considerations.

[Ballot comment]
I saw one editorial nit in new text added to -11. If this gets fixed before the telechat, fine, and if not, it should be fixed afterwards.

s/deigned/designed/

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has reviewed draft-ietf-ippm-model-based-metrics-10.txt, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
IANA Services Specialist
PTI

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC: ietf@trammell.ch, draft-ietf-ippm-model-based-metrics@ietf.org, ippm-chairs@ietf.org, spencerdawkins.ietf@gmail.com, ippm@ietf.org
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Model Based Metrics for Bulk Transport Capacity) to Experimental RFC


The IESG has received a request from the IP Performance Metrics WG (ippm)
to consider the following document:
- 'Model Based Metrics for Bulk Transport Capacity'
  as Experimental RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-03-14. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  We introduce a new class of Model Based Metrics designed to assess if
  a complete Internet path can be expected to meet a predefined Target
  Transport Performance by applying a suite of IP diagnostic tests to
  successive subpaths.  The subpath-at-a-time tests can be robustly
  applied to critical infrastructure, such as network interconnections
  or even individual devices, to accurately detect if any part of the
  infrastructure will prevent paths traversing it from meeting the
  Target Transport Performance.

  Model Based Metrics rely on peer-reviewed mathematical models to
  specify a Targeted Suite of IP Diagnostic tests, designed to assess
  whether common transport protocols can be expected to meet a
  predetermined Target Transport Performance over an Internet path.

  For Bulk Transport Capacity, the IP diagnostics are built on test
  streams that mimic TCP over the complete path and statistical
  criteria for evaluating the packet transfer statistics of those
  streams.  The temporal structure of the test stream (bursts, etc)
  mimic TCP or other transport protocol carrying bulk data over a long
  path.  However they are constructed to be independent of the details
  of the subpath under test, end systems or applications.  Likewise the
  success criteria evaluates the packet transfer statistics of the
  subpath against criteria determined by protocol performance models
  applied to the Target Transport Performance of the complete path.
  The success criteria also does not depend on the details of the
  subpath, end systems or application.

  Model Based Metrics exhibit several important new properties not
  present in other Bulk Transport Capacity Metrics, including the
  ability to reason about concatenated or overlapping subpaths.  The
  results are vantage independent which is critical for supporting
  independent validation of tests by comparing results from multiple
  measurement points.

  This document does not define the IP diagnostic tests, but provides a
  framework for designing suites of IP diagnostic tests that are
  tailored to confirming that infrastructure can meet the predetermined
  Target Transport Performance.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-ippm-model-based-metrics/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-ippm-model-based-metrics/ballot/


No IPR declarations have been submitted directly on this I-D.

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

Experimental; appropriate and correctly indicated. Sections 9.1 and 10 discuss
aspects of the validation and evaluation of experiments with the approach.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

    The document introduces a new class of Model Based Metrics, designed to
    assess if a complete Internet path can be expected to meet a predefined
    Target Transport Performance by applying a suite of IP diagnostic tests to
    successive subpaths.  The subpath-at-a-time tests can be robustly applied
    to key infrastructure, such as interconnects or even individual devices,
    to accurately detect if any part of the infrastructure will prevent paths
    traversing it from meeting the Target Transport Performance. Model Based
    Metrics exhibit several important new properties not present in other Bulk
    Transport Capacity Metrics, including the ability to reason about
    concatenated or overlapping subpaths.  The results are vantage independent
    which is critical for supporting independent validation of tests by
    comparing results from multiple measurement points.

Working Group Summary

    The document was extensively discussed in the IPPM working group over a
    period of three years; comments received during a first WGLC (particularly
    with respect to readability) led to extensive changes and a second last
    call. There was no particular controversy in the working group.

Document Quality

    The document has seen thorough review in the working group. The
    experimental methodology it describes has been under continuous
    development by the authors during the document's lifetime in the WG.

Personnel

    Brian Trammell is the document shepherd.
    Spencer Dawkins is the responsible AD.


(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

    The document was extensively discussed within the IPPM working group over a
    period of three years; comments received during a first WGLC (particularly
    with respect to readability) led to extensive changes and a second last
    call.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

    No.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

    No.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

    No concerns.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

  Yes.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

    No IPR disclosures.

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

    There is strong consensus among those who have read and reviewed it,
    though the subject is quite dense, so the entire group of active
    contributors has not read it. I had no concerns with the breadth of review
    and consensus.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

    No.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

    Reference to 2861 should be replaced with a reference to 7661.

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

    No formal review criteria.

(13) Have all references within this document been identified as
either normative or informative?

    Yes.

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

    No.

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in
the Last Call procedure.

    No.

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

    No to all. The document seeks to solve the same problems as those
    addresses in RFC 3148, but due to its experimental nature neither updates
    nor supercedes it. After experimentation with further model-based metrics,
    it may make sense to move 3148 to Historic.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).

    No actions for IANA.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

    No actions for IANA.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

    No automated checks apply.