IETF Last Call Review of draft-ietf-bmwg-mlrsearch-14
review-ietf-bmwg-mlrsearch-14-perfmetrdir-lc-min-2025-10-27-00

I have been selected as the Performance Metrics Directorate (perfmetrdir)
reviewer for this Internet-Draft.

The Performance Metrics Directorate assists the OPS Area Directors to review
performance-related documents.

Summary: I've reviewed this document and I think this document is READY with
Nits.

The Performance Metrics Directorate will be applying the Guidelines for
Considering New Performance Metric Development, RFC 6390.

The review will cover the following items:

* Are the performance metrics unambiguously defined?

[Reviewer]>>> Yes, I think so. In parallel to the existing strict definition of
throughput that requires zero-loss, this document introduces a composite metric
supporting both zero-loss throughput and non-zero-loss throughput, consisting
of Relevant Upper Bound, Relevant Lower Bound, and Conditional Throughput.
Among them the Conditional Throughput is a value computed at the Relevant Lower
Bound according to an algorithm defined in Appendix B of this document.

* Are the units of measure specified?

[Reviewer]>>> Yes, the units for all of Relevant Upper Bound, Relevant Lower
Bound, and Conditional Throughput are frames per second.

* Does the metric clearly define the measurement interval where applicable?

[Reviewer]>>> Yes, the operator can select the measurement interval in seconds.
There are Full-Length Trials and Short Trials to be selected.

* Are significant sources of measurement errors identified and discussed?

[Reviewer]>>> Yes, I think so. This document provides enough details on how to
eliminate the interference caused by inaccurate trials.

* Does the method of measurement ensure that results are repeatable?

[Reviewer]>>> Yes. Actually one major aim of MLRsearch is to improve result
repeatability and comparability.

* Does the metric or method of measurement appear to be implementable (or offer
evidence of a working implementation)?

[Reviewer]>>> Yes, exactly.

* Are there any undocumented assumptions concerning the underlying process that
would affect an implementation or interpretation of the metric?

[Reviewer]>>> No, I don't find any such assumptions.

* Can the metric results be related to application performance or user
experience, when such a relationship is of value?

[Reviewer]>>> No, I don't think the metric results have direct relationship
with application performance or user experience. As mentioned in this document,
"For users, performance of higher protocol layers is important, for example,
goodput of TCP connection (TCP throughput, [RFC6349])".

* Is there an existing relationship to metrics defined elsewhere within the
IETF or within other SDOs?

[Reviewer]>>> Yes. RFC 1242 provides the definition of throughput. RFC 2544
provides the definition of throughput test methodology. Following this
question, I suggest some text changes as below.

Section 2.1, The throughput definition per [RFC2544] -> The definition of
throughput test methodology per [RFC2544]

Section 2.3, the strict definition of throughput in [RFC2544] -> the strict
definition of throughput test methodology in [RFC2544]

Section 2.5, Definitions of throughput in [RFC1242] and [RFC2544] ->
Definitions of throughput and its test methodology in [RFC1242] and [RFC2544]

* Do the security considerations adequately address denial-of-service attacks,
unwanted interference with the metric/measurement, and user data
confidentiality (when measuring live traffic)?

[Reviewer]>>> It seems to me this document is irrelevant to this item.

* Is the method of measurement described or referenced? If described, does it
describe how the metrics are being measured and wherever other protocols are
applicable as well? If other protocols are applicable as well, does it answer
how described measurement method differs?

[Reviewer]>>> Yes, the method of measurement is described in detail. As to the
relationship with other protocols, Section 4.10 of this document discusses
compliance relations between MLRsearch and other test procedures including
[RFC2544] and [TST009].

One more nit as below.

Section 5.6, s/Section Section 3.6.1/Section 3.6.1.