IP Performance Metrics (IPPM): Spatial and Multicast
draft-ietf-ippm-multimetrics-12

[Ballot comment]
In Section 10:

>  This document defines the reporting of all the metrics
  introduced in a single section to provide consistent information, to
  avoid repetitions and to conform to IESG recommendation of gathering
  manageability considerations in a dedicated section.

While it is true that some of the IESG members hold the opinion that gathering manageability considerations in a dedicated section is a good thing, there is no IESG recommendation on this respect.

[Ballot comment]
The term "ipdv" was introduced here for the first time, please add
a reference or a term definition:

  o  Type-P-Spatial-One-way-ipdv-Vector divides an end-to-end Type-P-
      One-way-ipdv into a spatial vector of ipdv singletons.

[Ballot discuss]
This specification is a solid and useful piece of work. However,
before recommending the publication of this specification as an RFC,
I would like to talk about the following.

The document uses the term 'host' to refer to measurement points, despite
their role in the communications. For instance:

  A metric is said to be spatial if one of the hosts (measurement
  collection points) involved is neither the source nor a destination
  of the measured packet(s).

The traditional terminology is of course hosts, routers, and nodes.
My read of RFC 2330 is that it tries to be accurate about referring to
entities as routers when they are indeed routers. It is true that routers
are hosts too, but saying 'host' when a packet passes through a router is
somewhat misleading, in my opinion. I wonder if the term node would have
been more appropriate for some of this, or following the model in
RFC 2330.

[Ballot comment]
The term "ipdv" was introduced here for the first time, please add
a reference or a term definition:

  o  Type-P-Spatial-One-way-ipdv-Vector divides an end-to-end Type-P-
      One-way-ipdv into a spatial vector of ipdv singletons.

[Ballot comment]
Notation nits:

Figure 4's right-most column has repeated R3's where it meant R1, R2, R3

The paragraph below that figure talks about "observed at M points of
interest" where I think it meant "n points".

As discussed in email, there is a mix of RnMD and RnDM in section 8.3 that should be the same.

As discussed in email, Ln(k) in figure 10 and L(k,n) in figure 11 could use additional explanation.

[Ballot comment]
Figure 2

This would benefit from some explanation.

I presume 'x' does not have the same quality as 'X' although 'X' is not referenced.

It is not clear whether this is an example such that all nodes are candidate points of interest, but those 'x' just happen to not be points of interest.

Is there any significance in Figure 2 using 1,2,3,J where Figure 1 used 1,2,3,I?

Is the figure supposed to imply labeling of the 'X' hosts as 1,2,3,J?

---

Nice to expand "ipdv" on first use.

---

Section 4.1

  o  Monitoring the decomposed performance of a multicast tree based on
      of MPLS point-to-multipoint communications.

s/on of/on/

---

Figure 6

I suppose that this only applies for J[n] > 0

Obviously, it would be pointless to compute if no packets were received. Need to say so?

---

Section 9.1

OLD
  However, it may result in a lost of information.  As all measured
  singletons are not available for building up the group matrix, the
  real performance over time can be hidden from the result.
NEW
  However, it may result in a loss of information.  As not all
                                  ^                    ^^^
  measured singletons are available for building up the group matrix,
  the real performance over time can be hidden from the result.

---

Section 9.2

  To prevent any bias in the result, the configuration of a one-to-many
  measure must take in consideration that intrically more packets will
  to be routed than sent (copies of a packet sent are expected to
  arrive at many destination points) and selects a test packets rate
  that will not impact the network performance.

"intrically"?

Do you mean "intrinsically" or "intricately"? Maybe just delete the word and let "more" stand on its own.

---

Section 10

s/documents defines/documents define/

But actually...
  Usually IPPM WG documents defines each metric reporting within its
  definition.
...is either circuitous or has no meaning!

---

Section 10.1.2

  It is highly suggested to use the TTL in IPv4,
  the Hop Limit in IPv6 or the corresponding information in MPLS.

Is "highly suggested" language for inclusion in draft-ietf-rfc2119bis-00.txt?

---

Section 13

  Metrics defined in this memo Metrics defined in this memo are

Duplicate words.

---

Section 13

You might help IANA by making it clear that each "nn" is a different
number, possibly by using aa, bb, cc, etc.

[Ballot discuss]
Section 10

Shouldn't the must/should language here all be in RFC 2119 form?
It seems to be mixed.

---

Section 11

This section needs to highlight that path reporting mechanisms (such as indicated here) can be used to determine where in a network to attack a traffic flow.

Spatial reporting may indicate which nodes on a path are most vulnerable to attack.

Both of these issues can be determined by inspection without any need to attack the measurement packets themselves.

IANA comments:

Upon approval of this document, IANA will make the following
assignments in the "IANA-IPPM-METRICS-REGISTRY-MIB" registry at
http://www.iana.org/assignments/ianaippmmetricsregistry-mib

ietfSpatialOneWayDelayVector OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-Spatial-One-way-Delay-Vector"
REFERENCE
"Reference [RFC-ietf-ippm-multimetrics-11], section 5.1."
:= { ianaIppmMetrics TBD }

ietfSpatialPacketLossVector OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-Spatial-Packet-Loss-Vector"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 5.2."
:= { ianaIppmMetrics TBD }

ietfSpatialOneWayIpdvVector OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-Spatial-One-way-ipdv-Vector"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 5.3."
:= { ianaIppmMetrics TBD }

ietfSegmentOneWayDelayStream OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-Segment-One-way-Delay-Stream"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 6.1."
:= { ianaIppmMetrics TBD }

ietfSegmentPacketLossStream OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-Segment-Packet-Loss-Stream"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 6.2."
:= { ianaIppmMetrics TBD }

ietfSegmentIpdvPrevStream OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-Segment-ipdv-prev-Stream"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 6.3."
:= { ianaIppmMetrics TBD }

ietfSegmentIpdvMinStream OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-Segment-ipdv-min-Stream"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 6.4."
:= { ianaIppmMetrics TBD }

-- One-to-group metrics

ietfOneToGroupDelayVector OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Delay-Vector"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 7.1."
:= { ianaIppmMetrics TBD }

ietfOneToGroupPacketLossVector OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Packet-Loss-Vector"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 7.2."
:= { ianaIppmMetrics TBD }

ietfOneToGroupIpdvVector OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-ipdv-Vector"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 7.3."
:= { ianaIppmMetrics TBD }

-- One to group statistics

--

ietfOnetoGroupReceiverNMeanDelay OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Receiver-n-Mean-Delay"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 8.3.1."
:= { ianaIppmMetrics TBD }

ietfOneToGroupMeanDelay OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Mean-Delay"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 8.3.2."
:= { ianaIppmMetrics TBD }

ietfOneToGroupRangeMeanDelay OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Range-Mean-Delay"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 8.3.3."
:= { ianaIppmMetrics TBD }

ietfOneToGroupMaxMeanDelay OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Max-Mean-Delay"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 8.3.4."
:= { ianaIppmMetrics TBD }

ietfOneToGroupReceiverNLossRatio OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Receiver-n-Loss-Ratio"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 8.4.1."
:= { ianaIppmMetrics TBD }
--

ietfOneToGroupReceiverNCompLossRatio OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Receiver-n-Comp-Loss-Ratio"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 8.4.2."
:= { ianaIppmMetrics TBD }

ietfOneToGroupLossRatio OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Loss-Ratio"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 8.4.3."
:= { ianaIppmMetrics TBD }
--

ietfOneToGroupRangeLossRatio OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Range-Loss-Ratio"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 8.4.4."
:= { ianaIppmMetrics TBD }

ietfOneToGroupRangeDelayVariation OBJECT-IDENTITY
STATUS current
DESCRIPTION
"Type-P-One-to-group-Range-Delay-Variation"
REFERENCE
"Reference RFC-ietf-ippm-multimetrics-11, section 8.5.1."
:= { ianaIppmMetrics TBD }


We understand the above to be the only IANA Action for this document.

    (1.a) Who is the Document Shepherd for this document? Has the
          Document Shepherd personally reviewed this version of the
          document and, in particular, does he or she believe this
          version is ready for forwarding to the IESG for publication?

The document shepherd is Matt Zekauskas
The document shepherd has personally reviewed this document and
believes this version is read for forwarding to the IESG for
publication.

    (1.b) Has the document had adequate review both from key WG members
          and from key non-WG members? Does the Document Shepherd have
          any concerns about the depth or breadth of the reviews that
          have been performed?

I believe the document has received sufficient review from key WG
members, especially in the rev before and the two revs after WGLC.  I
don't believe there are key non-WG members that need to review the
document.  I have no concerns about the depth or breadth of reviews
for this document.

    (1.c) Does the Document Shepherd have concerns that the document
          needs more review from a particular or broader perspective,
          e.g., security, operational complexity, someone familiar with
          AAA, internationalization or XML?

no.

    (1.d) Does the Document Shepherd have any specific concerns or
          issues with this document that the Responsible Area Director
          and/or the IESG should be aware of? For example, perhaps he
          or she is uncomfortable with certain parts of the document, or
          has concerns whether there really is a need for it. In any
          event, if the WG has discussed those issues and has indicated
          that it still wishes to advance the document, detail those
          concerns here. Has an IPR disclosure related to this document
          been filed? If so, please include a reference to the
          disclosure and summarize the WG discussion and conclusion on
          this issue.

I think the AD should be aware of one potential issue.  In some of the
summary statistics, this document uses means of finite delays.
This has been of some past controversy, and I believe it has been
settled for this document; the document itself has a discussion on
packet loss in section 8.1.

(My view of) the issue is this: RFC 2679 (and the framework RFC 2330)
treat lost packets as having infinite delay.  RFC 2679 specifically
avoided using means of received packet delays, and only used order
statistics.  If you only want to report one value, knowing that the
50th percentile of delay is infinite tells you the path is bad.
Knowing only that the mean is 5mS is misleading if packet loss is 80%.
(Furthermore, means imply a normal distribution, and Internet
distributions tend to be far from normal.)  However, the industry
still uses means of finite delay, and using means can be useful in
some situations, and if the mean is always reported (or at least
considered) with loss and perhaps delay histograms, you can tell when
it is meaningless.  [Al Morton has an exposition in Section 4 of his
(so far personal) draft draft-morton-ippm-reporting-metrics-06.txt.]

This document was developed with an understanding of the limitations
of the mean, and it was chosen for composition.  There are a couple other
documents in the pipeline that also have this issue.  However, I can
believe that it might get raised again in IETF last call, although it
was not in WGLC.

I know of no IPR disclosures related to this document.

    (1.e) How solid is the WG consensus behind this document? Does it
          represent the strong concurrence of a few individuals, with
          others being silent, or does the WG as a whole understand and
          agree with it?

I think WG consensus behind this document is fairly strong at this point,
although I think there has been more silence on this document than
some of the other recent ones.

    (1.f) Has anyone threatened an appeal or otherwise indicated extreme
          discontent? If so, please summarise the areas of conflict in
          separate email messages to the Responsible Area Director. (It
          should be in a separate email because this questionnaire is
          entered into the ID Tracker.)

no.

    (1.g) Has the Document Shepherd personally verified that the
          document satisfies all ID nits? (See
          http://www.ietf.org/ID-Checklist.html and
          http://tools.ietf.org/tools/idnits/). Boilerplate checks are
          not enough; this check needs to be thorough. Has the document
          met all formal review criteria it needs to, such as the MIB
          Doctor, media type and URI type reviews?

Yes.  There are no other area formal reviews needed.

    (1.h) Has the document split its references into normative and
          informative? Are there normative references to documents that
          are not ready for advancement or are otherwise in an unclear
          state? If such normative references exist, what is the
          strategy for their completion? Are there normative references
          that are downward references, as described in [RFC3967]? If
          so, list these downward references to support the Area
          Director in the Last Call procedure for them [RFC3967].

References are split and appropriate.  There are no downward refs, nor
normative refs to wait for.  There is one non-normative work-in-progress
reference (to Spatial Composition of Metrics) that has a new revision number,
but the reference is still appropriate and would be referenced as a work
in progress rather than specific version number.


    (1.i) Has the Document Shepherd verified that the document IANA
          consideration section exists and is consistent with the body
          of the document? If the document specifies protocol
          extensions, are reservations requested in appropriate IANA
          registries? Are the IANA registries clearly identified? If
          the document creates a new registry, does it define the
          proposed initial contents of the registry and an allocation
          procedure for future registrations? Does it suggest a
          reasonable name for the new registry? See [RFC5226]. If the
          document describes an Expert Review process has Shepherd
          conferred with the Responsible Area Director so that the IESG
          can appoint the needed Expert during the IESG Evaluation?

The IANA consideration section exists, and is consistent with the
other documents from IPPM and RFC 4148, which describes the IPPM
Metrics Registry.

    (1.j) Has the Document Shepherd verified that sections of the
          document that are written in a formal language, such as XML
          code, BNF rules, MIB definitions, etc., validate correctly in
          an automated checker?

The only section where this could apply is the IANA Considerations
section, where IANA is asked to register the metrics in the IPPM
Metrics Registry.  The entries given are proper form with respect
to RFC 4148, which defined the registry.

    (1.k) The IESG approval announcement includes a Document
          Announcement Write-Up. Please provide such a Document
          Announcement Write-Up? Recent examples can be found in the
          "Action" announcements for approved documents. The approval
          announcement contains the following sections:

          Technical Summary

The IETF has standardized IP Performance Metrics (IPPM) for measuring
end-to-end performance between two points.  This memo defines two new
categories of metrics that extend the coverage to multiple
measurement points.  It defines spatial metrics for measuring the
performance of segments of a source to destination path, and metrics
for measuring the performance between a source and many destinations
in multiparty communications (e.g., a multicast tree).

The scope of this memo is limited to metrics using a single source
packet or stream, and observations of corresponding packets along the
path (spatial), at one or more destinations (one-to-group), or both.
Note that all the metrics defined herein are based on observations of
packets dedicated to testing, a process that is called active
measurement.  Passive measurement (for example, a spatial metric
based on the observation of user traffic) is beyond the scope of this
memo.


          Working Group Summary

The working group input has improved this document through its
revisions, and the document itself has been uncontroversial.


          Document Quality

I know of no current implementations that claim to implement this metric.
However, other implementers in the group have read the draft.