Initial Performance Metrics Registry Entries
draft-ietf-ippm-initial-registry-16

=== This writeup is in progress, and is not yet ready for submission to the IESG. ===

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

Proposed Standard; this is indicated correctly. This type was selected by the group
as metrics specified by IPPM are usually published as PS and this document
provides even tighter specifications of the original metrics for assignment in the
registry.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

  This memo defines the set of Initial Entries for the IANA Performance
  Metrics Registry (defined in draft-ietf-ippm-metric-registry).
  The set includes, UDP Round-trip Latency and Loss,
  Packet Delay Variation, DNS Response Latency and Loss, UDP Poisson
  One-way Delay and Loss, UDP Periodic One-way Delay and Loss, ICMP
  Round-trip Latency and Loss, and TCP round-trip Latency and Loss.

  As the document provides an initial registry, the content is determined
  both by initial requirements from the LMAP working group (which has
  since concluded, taking a direction that no longer requires the registry)
  as well as an analysis within the WG of commonly used metrics.

  Mock-ups of the implementation of the base registry itself
  have been prepared with IANA's help. A recent version
  is available here: http://encrypted.net/IETFMetricsRegistry-106.html


Working Group Summary

  The document has been discussed extensively in the WG, and has had
  significant input from the community.

Document Quality

  The document does not specify a protocol; the state of implementation
  is unknown to the shepherd. The design of the registry
  (in draft-ietf-ippm-metric-registry) was "tested" by the entries in the initial
  registry, and both of these have received early IANA review together.

Personnel

  Brian Trammell is the Document Shepherd. Mirja Kuehlewind is the Responsible Area
  Director.


(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

I have reviewed the document and it is IMO ready for publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

No.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

No reviews beyond the performance measurement community
represented in IPPM seem particularly necessary.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

Yes.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

No IPR disclosures.

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

The document has strong consensus in the WG.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

Fixed.

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

No formal review necessary.

(13) Have all references within this document been identified as
either normative or informative?

Yes

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

Normative reference to draft-ietf-ippm-metric-registry,
which this document fills out and which will be submitted at the same time.

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in
the Last Call procedure.

There are normative references to RFC 2330, as well as to two academic publications (in CACM and TMA).

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 8126).

The whole document is essentially an IANA considerations section
populate the registry in draft-ietf-ippm-metric-registry. It does not
interact with any other registries.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

No new registries; these are defined in draft-ietf-ippm-metric-registry.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

No automated checks.

[Ballot discuss]
Quite minor points, really, but they do need to be resolved before
publication.  (Slightly more details/locations in the COMMENT section.)

We don't actually provide a definition (whether directly or by
reference) for the "classic calculation for standard deviation of a
population".

We don't actually provide a definition (whether directly or by
reference) for what the "time_offset" calibration value records.

[Ballot comment]
Section 2

  This document defines the initial set of Performance Metrics Registry
  entries, for which IETF approval (following development in the IP
  Performance Metrics (IPPM) Working Group) will satisfy the
  requirement for Expert Review.  Most are Active Performance Metrics,

That's not really how Expert Review works, though as is being discussed
in other ballot threads, we have other options for getting these registered.

Section 4

  Note: Each Registry entry only produces a "raw" output or a
  statistical summary.  To describe both "raw" and one or more
  statistics efficiently, the Identifier, Name, and Output Categories
  can be split and a single section can specify two or more closely-
  related metrics.  This section specifies two Registry entries with
  many common columns.  See Section 7 for an example specifying
  multiple Registry entries with many common columns.

I'm not sure I understand the intended scope/audience of this remark.
It is discussing something that we are doing for these registrations or
something that might be done in the future?

Section 4.2.2

  o  UDP Payload

      *  total of 100 bytes

In the vein of "clear and reproducible metric definition", should we say
something about the content of the payload (even if it's just "the
content is arbitrary")?  Per Section 4.3.1 we could even note that a
sequence number could be used as the payload.
(I won't mention a similar consideration for the later metrics.)

  o  Tmax: a loss threshold waiting time

      *  3.0, expressed in units of seconds, as a positive value of type
        decimal64 with fraction digits = 4 (see section 9.3 of
        [RFC6020]) and with resolution of 0.0001 seconds (0.1 ms), with
        lossless conversion to/from the 32-bit NTP timestamp as per
        section 6 of [RFC5905].

Am I reading this right that the Tmax is a fixed parameter with value
3.0 seconds?  If so, I'm confused why the extra specification on its
representation is needed (and why we write only "3.0" and not "3.0000"
to get the 4 fraction digits, though per RFC 6020 the fraction digits is
just a maximum).
(I won't mention a similar consideration for the later metrics.)

Section 4.3.1

  The calculations on the delay (RTT) SHALL be performed on the
  conditional distribution, conditioned on successful packet arrival
  within Tmax.  Also, when all packet delays are stored, the process
  which calculates the RTT value MAY enforce the Tmax threshold on
  stored values before calculations.  See section 4.1 of [RFC3393] for

Why is this a MAY and not a MUST?  Wouldn't it make the results not
necessarily reproducible/transferrable?
(I won't mention a similar consideration for the later metrics.)

Section 4.3.2

  dT the duration of the interval for allowed sample start times, with
      value 1.0, expressed in units of seconds, as a positive value of
      type decimal64 with fraction digits = 4 (see section 9.3 of
      [RFC6020]) and with resolution of 0.0001 seconds (0.1 ms).

[1.0 does not have 4 fraction digits; though we did use 0.0200 for incT]

Section 4.3.5

There are potentially privacy considerations (albeit likely to be
minimal) with requiring the reporting of Src and Dst addresses as part
of the measurement results.

Section 4.4.2

  95Percentile  The time value of the result is expressed in units of
      seconds, as a positive value of type decimal64 with fraction
      digits = 9 (see section 9.3 of [RFC6020]) with resolution of
      0.000000001 seconds (1.0 ns), and with lossless conversion to/from
      the 64-bit NTP timestamp as

Is there a missing end of the sentence?

Section 4.4.4

  When a measurement controller requests a calibration measurement, the
  loopback is applied and the result is output in the same format as a
  normal measurement with additional indication that it is a
  calibration result.

If the goal of the calibration is intended in part to quantify the
random errors of a measurement, shouldn't we also report something akin
to a variance or standard distribution, as opposed to just the same
summary results that we would get from a non-calibration measurement?
My understanding of what this says to do will only give you the
"average offset" due to the random+systematic error (for the appropriate
definition of "average") but not give you a sense for whether the
effective precision is reduced by those random errors.
(I won't mention a similar consideration for the later metrics.)

Section 5.4.4

Does this really give me a clear definition of what the time_offset is
measuring (as opposed to just how to represent it)?
(I won't mention a similar consideration for the later metrics.)

Section 6.3.1

  DNS Messages bearing Queries provide for random ID Numbers in the
  Identification header field, so more than one query may be launched
  while a previous request is outstanding when the ID Number is used.
  Therefore, the ID Number MUST be retained at the Src or included with
  each response packet to disambiguate packet reordering if it occurs.

Is this an "or" or an "and"?  Also, the DNS protocol is pretty well
specified about this already, right, so we're just noting the properties
that is has?

  In addition to operations described in [RFC2681], the Src MUST parse
  the DNS headers of the reply and prepare the information for
  subsequent reporting as a measured result, along with the Round-Trip
  Delay.

Since we only use the Rcode, maybe we could be more specific than "the
information"?

Section 6.3.2

  Section 11.1.3 of RFC 2681 [RFC2330] provides three methods to
  generate Poisson sampling intervals.  The reciprocal of lambda is the
  average packet rate, thus the Run-time Parameter is Reciprocal_lambda
  = 1/lambda, in seconds.

Wouldn't something measured in seconds be an inter-packet interval, not
a "packet rate"?  (Hmm, § 7.3.2 calls it the "average packet spacing".)

Section 6.3.5

  Trunc  Upper limit on Poisson distribution expressed in units of
      seconds, as a positive value of type decimal64 with fraction
      digits = 4 (see section 9.3 of [RFC6020]) with resolution of
      0.0001 seconds (0.1 ms), and with lossless conversion to/from the
      32-bit NTP timestamp as per section 6 of [RFC5905] (values above
      this limit will be clipped and set to the limit value). (if fixed,
      Trunc = 30.0000 seconds.)

For a registered matric, a parameter is either Fixed or Run-time; there
is no option.  This last parenthetical needs to be removed.

  ID The 16-bit identifier assigned by the program that generates the
      query, and which must vary in successive queries, see
      Section 4.1.1 of [RFC1035].  This identifier is copied into the
      corresponding reply and can be used by the requester (Src) to
      match-up replies to outstanding queries.

If it must vary in successive queries (which are still part of the same
Stream), that's not a single Run-time parameter anymore, is it?

  QTYPE  The Query Type, which will correspond to the IP address family
      of the query (decimal 1 for IPv4 or 28 for IPv6, formatted as a
      uint16, as per section 9.2 of [RFC6020].

Does this implicitly exclude non-IP-address queries?

Section 7.2.2

Do we need to report or specify which TWAMP security features are in
use, in case that has an effect on processing times and the
corresponding measurements?
(I won't mention a similar consideration for the later metrics.)

Section 7.3.1

  The reference method requires some way to distinguish between
  different packets in a stream to establish correspondence between
  sending times and receiving times for each successfully-arriving
  packet.  Sequence numbers or other send-order identification MUST be
  retained at the Src or included with each packet to disambiguate
  packet reordering if it occurs.

  Since a standard measurement protocol is employed [RFC5357], then the
  measurement process will determine the sequence numbers or timestamps
  applied to test packets after the Fixed and Runtime parameters are
  passed to that process.  The measurement protocol dictates the format
  of sequence numbers and time-stamps conveyed in the TWAMP-Test packet
  payload.

I'm not sure whether it makes sense to keep the end of the first
paragraph given that the second paragraph covers the specifics of how
the requirement is satisfied.
(I won't mention a similar consideration for the later metrics.)

Section 7.4.2.5

  See section 4.3.2 of [RFC6049] for a closely related method for
  calculating this statistic, and 4.3.3 of [RFC6049].  The formula is
  the classic calculation for standard deviation of a population.

Please provide the actual formula, whether directly or by reference.  A
"closely related method" implies that it is not the actual method to
use, leaving the actual method to use underspecified here.

Section 9

Interesting to see that we don't include 95thpercentile or StdDev for
ICMP, though this is of course not actually problematic.

  All column entries beside the ID, Name, Description, and Output
  Reference Method categories are the same, thus this section proposes
  two closely-related registry entries.  As a result, IANA is also
  asked to assign four corresponding URLs to each Named Metric.

nit: s/four//

Section 9.2.2

  o  ICMP Payload

      *  total of 32 bytes of random info

Randomly selected when -- per-test?  per-packet?
I note that this is the "fixed parameters" section, and the above seems
to be anything but *fixed*.

Section 10

  All column entries beside the ID, Name, Description, and Output
  Reference Method categories are the same, thus this section proposes
  four closely-related registry entries.  As a result, IANA is also
  asked to assign four corresponding URLs to each Named Metric.

nit: s/four//

Section 10.1.2

nit: s/opportuinty/opportunity/

Section 10.2.1


  Traffic filters reduce the packet stream at the OP to a Qualified
  bidirectional flow packets.

nit: singular/plural mistmatch "a"/"packets"

  For a real number, RTD_fwd, >> the Round-trip Delay in the Forward
  direction from OP to host B at time T' is RTD_fwd << REQUIRES that OP
  observed a Qualified Packet to host B at wire-time T', that host B
  received that packet and sent a Corresponding Packet back to host A,
  and OP observed the Corresponding Packet at wire-time T' + RTD_fwd.

I'm either failing to parse this or confused by the notation (or both).
Are the doubled angle brackets intended to enclose a parenthetical
remark defining RTD_fwd?  (If so, why would traditional parentheses not
work?)

Section 11

Please mention the potential privacy considerations for observed
traffic, particularly for passive metrics.

An attacker that knows that its TCP connection is being measured can
modify its behavior to skew the measurement results.

  Security.  Each referenced Metric contains a Security Considerations
  section.

I do not see any such sections.

Section 14.2

Shouldn't RFC 5481 be normative, given that in Section 5.3.1 we see that
"this metric entry requires implementation of the PDV form defined in
section 4.2 of [RFC5481]"?

[Ballot discuss]
The shepherd writeup (which has a note that it is still in progress and
not ready for submission) indicates that the "IPR questions are pending".
Please confirm that they were affirmatively resolved.

[Ballot discuss]
I think these entries shows well how the registry is intended to extended and the benefit in being clear on what information is needed. I do have a question that I think needs an answer:

Regarding Section 4.2.2:

  o  IPv6 header values:

      *  DSCP: set to 0

      *  Hop Count: set to 255

      *  Protocol: Set to 17 (UDP)

Does anything about the IPv6 flow ID need to be stated here? As this is a path delay measurement, the value of the flow ID field has the potential to change the result. If one would use a new random value for each individual measurement in a sequence one may see different results than from using the same ID for all the measurements. Or is this specified in any of the references? In most case I would expect one use a single value, but likely randomly selected. However, it does depend on what purpose of ones measurement one have, thus I think this do matter.

I think this question applies to all measurements that are multi-packet ones so section 5, 7, 8 and 9 most definitely.

I also wonder if IPv6 Flow ID is an output parameter that needs to be kept?

[Ballot comment]
Section 4.2.2:

  o  UDP header values:

      *  Checksum: the checksum MUST be calculated and included in the
        header

I guess this implies that a non-zero UDP checksum is to be used. Maybe one could make this water tight by changing it to:

      *  Checksum: the checksum MUST be calculated and the non-zero checksum
          included in the header

[Ballot comment]
Trivial editorial nits:
- Section 4.4.4.  s/*available accuracy*/available accuracy/
- Section 4.5.2. s/This RFC numner/This RFC number/
- Section 7.5.2. s/This REFC number/This RFC number/
- Section 10.5.2. s/This RFC/This RFC number/
- Section 10.3.1.  s/with[Strowes]/with [Strowes/]

[Ballot comment]
I have a significant issue related to the approval of this initial registry, and then a minor point.

(1) This is the significant issue:

§2:

  This document defines the initial set of Performance Metrics Registry
  entries, for which IETF approval (following development in the IP
  Performance Metrics (IPPM) Working Group) will satisfy the
  requirement for Expert Review.  Most are Active Performance Metrics,
  which are based on RFCs prepared in the IPPM working group of the
  IETF, according to their framework [RFC2330] and its updates.

s/IETF approval/IESG Approval   
There is no "IETF approval" policy in rfc8126.

Even though it is self-serving for this document to provide guidance on the approval for the new registry entries, I think that it should be possible to do so using IESG Approval given that these initial entries into the new registry have been reviewed by the WG.  However, draft-ietf-ippm-metric-registry only explicitly talks about using Specification Required (including Expert Review), and it makes no exceptions.

This document should not make statements about criteria used for this initial registration, so §2 should be deleted.

draft-ietf-ippm-metric-registry should be explicit about the initial population of the registry.  I am them balloting DISCUSS on that other document to address the approval of the initial registry entries in this document.


(2) Minor point:

§1: "The process in [I-D.ietf-ippm-metric-registry] also requires that new entries are administered by IANA through Expert Review or IETF Standards action, which will ensure that the metrics are tightly defined."  The registration policy defined in I-D.ietf-ippm-metric-registry is "Specification Required", which requires the review of an expert.

[Ballot comment]
Thank you for the work put into this document.

Thank you for quickly addressing my previous DISCUSS (kept here for history)

Regards,

-éric

== DISCUSS ==

-- Section 4.2.2 --
Easy to fix: there is no 'protocol' field in the IPv6 header but a 'next header' one that has the same semantic.

-- Section 9.2.2. --
Also easy to fix, 'next header' for ICMPv6 is not 01 but 58 (decimal) and 'ICMPv6 echo request' is 128 (decimal).

== COMMENTS ==

-- Section 4.2.2 -- and others

Should some additional parameters be specified for IPv6? Namely flow label and the absence of extension header.

[Ballot comment]
Please be aware of the following note in the IANA section of draft-ietf-ippm-metric-registry-22 (which might simply reviewing):

"Mock-ups of the implementation of this set of requests
  have been prepared with IANA's help during development of this memo,
  and have been captured in the Proceedings of IPPM working group
  sessions.  IANA is currently preparing a mock-up.  A recent version
  is available here: http://encrypted.net/IETFMetricsRegistry-106.html"

Also as draft-ietf-ippm-metric-registry defines the registry that this draft specifies initial values for, it makes sense to review draft-ietf-ippm-metric-registry first!

[Ballot comment]
Meta comment: Next time a document like this is produced, I think it would be very helpful to have something like a "tag" (e.g: '{IANA}') to help the IANA find all of the sections where they need to take action / make an edit -- this document (unsurprisingly!) has many places where edits are needed (e.g: 'YYYY-MM-DD') it is seems possible that one will be missed.

I found the example registry helpful, thanks! Also, props to Eric for catching the Ipv6 codepoint issues :-)

Nits:
Abstract - 'The set includes, UDP Round-trip Latency and Loss, ...' -> 'The set includes: UDP Round-trip Latency and Loss, ...'

[Ballot comment]
Thank you for this document.

I only have one question:

Sections 4.3.3, 6.3.3., 9.3.3., and 10.3.3 seem to have kept the text from the template:
  The measured results based on a filtered version of the packets
  observed, and this section provides the filter details (when
  present).
Is that on purpose?

[Ballot comment]
Thank you for this document.

I only have one question:

Section 4.3.3, 6.3.3., 9.3.3., and 10.3.3 seems to have kept the text from the template:
  The measured results based on a filtered version of the packets
  observed, and this section provides the filter details (when
  present).
Is that on purpose?

[Ballot comment]
Thank you for this document.

I only have a couple of comment/questions, and found few nits while reading.

  Performance Metrics Registry MAY use the same
I'm not sure 2119/8174 language is needed here.

I'm not sure to understand the Version (of Registry Format) column. All entries will have the same number there, right?
If this RFC-to-be is updated I guess we'll change to version 2. What shall happen to existing entries, will they keep Version=1 or adopt Version=2?


s/in order to included/in order to be included/

  As any IETF registry, the primary use for a registry is to manage a
  registry for its use within one or more protocols.
This sentence seems a bit hard to parse

s/other form of Performance Metric/other form of Performance Measurement/ ?

=== This writeup is in progress, and is not yet ready for submission to the IESG. ===

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

Proposed Standard; this is indicated correctly. This type was selected by the group
as metrics specified by IPPM are usually published as PS and this document
provides even tighter specifications of the original metrics for assignment in the
registry.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

  This memo defines the set of Initial Entries for the IANA Performance
  Metrics Registry (defined in draft-ietf-ippm-metric-registry).
  The set includes, UDP Round-trip Latency and Loss,
  Packet Delay Variation, DNS Response Latency and Loss, UDP Poisson
  One-way Delay and Loss, UDP Periodic One-way Delay and Loss, ICMP
  Round-trip Latency and Loss, and TCP round-trip Latency and Loss.

  As the document provides an initial registry, the content is determined
  both by initial requirements from the LMAP working group (which has
  since concluded, taking a direction that no longer requires the registry)
  as well as an analysis within the WG of commonly used metrics.

  Mock-ups of the implementation of the base registry itself
  have been prepared with IANA's help. A recent version
  is available here: http://encrypted.net/IETFMetricsRegistry-106.html


Working Group Summary

  The document has been discussed extensively in the WG, and has had
  significant input from the community.

Document Quality

  The document does not specify a protocol; the state of implementation
  is unknown to the shepherd. The design of the registry
  (in draft-ietf-ippm-metric-registry) was "tested" by the entries in the initial
  registry, and both of these have received early IANA review together.

Personnel

  Brian Trammell is the Document Shepherd. Mirja Kuehlewind is the Responsible Area
  Director.


(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

I have reviewed the document and it is IMO ready for publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

No.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

No reviews beyond the performance measurement community
represented in IPPM seem particularly necessary.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

IPR questions are pending.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

No IPR disclosures.

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

The document has strong consensus in the WG.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

=== NITS NEED FIXING ===

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

No formal review necessary.

(13) Have all references within this document been identified as
either normative or informative?

Yes

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

Normative reference to draft-ietf-ippm-metric-registry,
which this document fills out and which will be submitted at the same time.

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in
the Last Call procedure.

=== CHECK AFTER NITS FIXED ===

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 8126).

The whole document is essentially an IANA considerations section
populate the registry in draft-ietf-ippm-metric-registry. It does not
interact with any other registries.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

No new registries; these are defined in draft-ietf-ippm-metric-registry.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

No automated checks.

[Ballot comment]
Please be aware of the following note in the IANA section of draft-ietf-ippm-metric-registry-22 (which might simply reviewing):

"Mock-ups of the implementation of this set of requests
  have been prepared with IANA's help during development of this memo,
  and have been captured in the Proceedings of IPPM working group
  sessions.  IANA is currently preparing a mock-up.  A recent version
  is available here: http://encrypted.net/IETFMetricsRegistry-106.html"

[Ballot discuss]
Thank you for the work put into this document.

I have a couple of easy and trivial to fix DISCUSS. And a couple of COMMENTs, feel free to ignore the COMMENTs but I would appreciate it if you replied to them.

Regards,

-éric

== DISCUSS ==

-- Section 4.2.2 --
Easy to fix: there is no 'protocol' field in the IPv6 header but a 'next header' one that has the same semantic.

-- Section 9.2.2. --
Also easy to fix, 'next header' for ICMPv6 is not 01 but 58 (decimal) and 'ICMPv6 echo request' is 128 (decimal).

[Ballot comment]
== COMMENTS ==

-- Section 4.2.2 -- and others

Should some additional parameters be specified for IPv6? Namely flow label and the absence of extension header.

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has completed its review of draft-ietf-ippm-initial-registry-12. If any part of this review is inaccurate, please let us know.

IANA has noted the detailed instructions in this draft for the creation of initial content for ippm metric registries and the establishment of those initial registries created in the companion document [draft-ietf-ippm-metric-registry]. Upon publication of these two documents, IANA will work with the authors to establish and populate the registries.

Note:  The actions requested in this document will not be completed until the document has been approved for publication as an RFC. This message is meant only to confirm the list of actions that will be performed.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist

The following Last Call announcement was sent out (ends 2019-11-06):

From: The IESG
To: IETF-Announce
CC: ippm-chairs@ietf.org, ippm@ietf.org, Brian Trammell , ietf@kuehlewind.net, ietf@trammell.ch, draft-ietf-ippm-initial-registry@ietf.org
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Initial Performance Metrics Registry Entries) to Proposed Standard


The IESG has received a request from the IP Performance Measurement WG (ippm)
to consider the following document: - 'Initial Performance Metrics Registry
Entries'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2019-11-06. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  This memo defines the set of Initial Entries for the IANA Performance
  Metrics Registry.  The set includes, UDP Round-trip Latency and Loss,
  Packet Delay Variation, DNS Response Latency and Loss, UDP Poisson
  One-way Delay and Loss, UDP Periodic One-way Delay and Loss, ICMP
  Round-trip Latency and Loss, and TCP round-trip Latency and Loss.





The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-ippm-initial-registry/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-ippm-initial-registry/ballot/


No IPR declarations have been submitted directly on this I-D.

=== This writeup is in progress, and is not yet ready for submission to the IESG. ===

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

Proposed Standard; this is indicated correctly.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

  This memo defines the set of Initial Entries for the IANA Performance
  Metrics Registry (defined in draft-ietf-ippm-metric-registry).
  The set includes, UDP Round-trip Latency and Loss,
  Packet Delay Variation, DNS Response Latency and Loss, UDP Poisson
  One-way Delay and Loss, UDP Periodic One-way Delay and Loss, ICMP
  Round-trip Latency and Loss, and TCP round-trip Latency and Loss.

  As the document provides an initial registry, the content is determined
  both by initial requirements from the LMAP working group (which has
  since concluded, taking a direction that no longer requires the registry)
  as well as an analysis within the WG of commonly used metrics.

Working Group Summary

  The document has been discussed extensively in the WG, and has had
  significant input from the community.

Document Quality

  The document does not specify a protocol; the state of implementation
  is unknown to the shepherd. The design of the registry
  (in draft-ietf-ippm-metric-registry) was "tested" by the entries in the initial
  registry, and both of these have received early IANA review together.

Personnel

  Brian Trammell is the Document Shepherd. Mirja Kuehlewind is the Responsible Area
  Director.


(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

I have reviewed the document and it is IMO ready for publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

No.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

No reviews beyond the performance measurement community
represented in IPPM seem particularly necessary.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

IPR questions are pending.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

No IPR disclosures.

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

The document has strong consensus in the WG.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

=== NITS NEED FIXING ===

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

No formal review necessary.

(13) Have all references within this document been identified as
either normative or informative?

Yes

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

Normative reference to draft-ietf-ippm-metric-registry,
which this document fills out and which will be submitted at the same time.

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in
the Last Call procedure.

=== CHECK AFTER NITS FIXED ===

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 8126).

The whole document is essentially an IANA considerations section
populate the registry in draft-ietf-ippm-metric-registry. It does not
interact with any other registries.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

No new registries; these are defined in draft-ietf-ippm-metric-registry.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

No automated checks.

=== This writeup is in progress, and is not yet ready for submission to the IESG. ===

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

Proposed Standard; this is indicated correctly.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

  This memo defines the set of Initial Entries for the IANA Performance
  Metrics Registry (defined in draft-ietf-ippm-metric-registry).
  The set includes, UDP Round-trip Latency and Loss,
  Packet Delay Variation, DNS Response Latency and Loss, UDP Poisson
  One-way Delay and Loss, UDP Periodic One-way Delay and Loss, ICMP
  Round-trip Latency and Loss, and TCP round-trip Latency and Loss.

  As the document provides an initial registry, the content is determined
  both by initial requirements from the LMAP working group (which has
  since concluded, taking a direction that no longer requires the registry)
  as well as an analysis within the WG of commonly used metrics.

Working Group Summary

  The document has been discussed extensively in the WG, and has had
  significant input from the community.

Document Quality

  The document does not specify a protocol; the state of implementation
  is unknown to the shepherd. The design of the registry
  (in draft-ietf-ippm-metric-registry) was "tested" by the entries in the initial
  registry, and both of these have received early IANA review together.

Personnel

  Brian Trammell is the Document Shepherd. Mirja Kuehlewind is the Responsible Area
  Director.


(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

I have reviewed the document and it is IMO ready for publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

No.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

No reviews beyond the performance measurement community
represented in IPPM seem particularly necessary.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

No concerns.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

IPR questions are pending.

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

No IPR disclosures.

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

The document has strong consensus in the WG.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

=== NITS NEED FIXING ===

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

No formal review necessary.

(13) Have all references within this document been identified as
either normative or informative?

Yes

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

Normative reference to draft-ietf-ippm-metric-registry,
which this document fills out and which will be submitted at the same time.

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in
the Last Call procedure.

=== CHECK AFTER NITS FIXED ===

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 8126).

The whole document is essentially an IANA considerations section
populate the registry in draft-ietf-ippm-metric-registry. It does not
interact with any other registries.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

No new registries; these are defined in draft-ietf-ippm-metric-registry.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

No automated checks.