Last Call Review of draft-ietf-alto-performance-metrics-17
review-ietf-alto-performance-metrics-17-artart-lc-amsuess-2021-10-18-00

## Summary for the IoT Directorate

This document extends the Application-Layer Traffic Optimization (ALTO)
protocol, by which applications that need to a peer from a selection can ask
their uplink for preferences. It adds more fine-grained vocabulary to pick the
peer with best expected bandwidth, or best latency.

As for conventions around the Internet of Things, this makes no choices -- if
follows the path set out by ALTO, and adds terms and considerations for metrics
established outside of ALTO.

The mechanisms (more of ALTO than this particular document) can be applicable
to IoT applications when unconstrained devices relay data into cloud systems --
then they might make better choices in presence of decentralized backends. If
this document gains traction in deployments, these systems will need deeper
application knowledge pertaining to the application's requirements (selecting
the low-latency vs. the high-bandwidth option).

## High-level

The document can be followed well after brief familiarization with ALTO; it is
in a good over-all shape.

2.2 Performance Metric Statistics:

* The metric-identifier definition seems disconnected with the rest of the
  terminology. I assume from context that this is a way for building a
  CostMetric value. If this assumption is wrong, some of the later points are
  moot.

  (By the way, what language is the metric-identifier definition?  It's not
  ABNF, and I don't find any other formal language in thenormative references.)

* p0 and p100 are aliases to min and max IIUC. Is there a preferred form, and
  why is the other form allowed? (Same for p50 and median.)

* Allowing decimals into the cost metric identifier introduces a dot which is
  reserved as per RFC7285 Section 10.6.

* comparing to 7 IANA Considerations: The registered identifeirs are only the
  metric-base-identifiers.

  Registering all combined values from metric-base-identifier x stat is not
  practical (especially with the numeric percentiles); however, the 'priv:'
  prefix indicates that some structure was intended in RFC7285.

  A way out could be to formalize this structure and register the
  metric-base-identifiers for use with and without a stat parameter following
  the colon (instead of the dash).

* Section 2.2 (Performance Metric Statistics) allows adding -stdvar to
  metric-base-identifiers; delay-variation-mean is probably similar to
  delay-ow-stdvar. It's only similar (and not identical) due to the offset from
  minimum detailed in 3.3.3; anyhow, pointing out that out in the "Note that in
  statistics" paragraph, e.g. as in "... networking convention. Due to this, it
  is expressed as a dedicated metric and not just as delay-ow-stddev".

  A few words on which statistic can be used with which metric could also help
  with bw-maxres. (What does bw-maxres-p50 mean, is it meaningful at all?)

3 Packet Performance Metrics:

* Cost-Context Specification Considerations: There is a lot of duplication
  here, especially around SLA and the topic of "link" descriptions. (For the
  technically interesting parts, the later sections already refer back to
  3.1.4).

6 Security Considerations:

* On the topic of confidentiality, can these metrics be used with the "ordinal"
  cost mode? A client might just want to pick the peer that has the "best"
  round-trip time, and could thus avoid the need to obtain the confidential
  metrics.

## Nits

* Table 1: The current XML RFC format can do tables that render to HTML, this
  would also make it easier to follow the links.

  For tables and figures, also more accessible labeling is available there.

* Section 3 Packet Performance Metrics: The pkt.* notation appears very ad-hoc,
  and is not used anywhere outside the paragraph. If these are referring to
  external definitions, please consider referencing that definition; otherwise:
  do these labels add value?

* "smaller than milliseconds": smaller than one millisecond?

* "Content-Length: TBA": Does this add value to the examples?

* Example in 3.1.3: Does the example make sense in terms of addresses? Querying
  metrics between an IPv4 and an IPv6 address seems strange. (And is this
  primarily used in a field where IPv4 is still dominant?)

  In particular, the hop count example has an unreasonably high number of hops
  between 192.0.2.2 and 192.168.2.89...

* "the -<percentile> component": "any 'stat' component"?

## Slightly off topic

(ie. this is out my expertise and would be checked later by someone else, but
it may be helpful to revisit it right away)

* IANA considerations: Creating a new regisry usually comes with ore than
  "reqeusts the creation of" and some values;
  https://datatracker.ietf.org/doc/html/rfc8126#section-2.2 in particular asks
  for a registration policy.

  (Given the document says "MUST be one of three", there may not be a need to
  set up a registry in the first place).