Last Call Review of draft-ietf-aqm-eval-guidelines-11
review-ietf-aqm-eval-guidelines-11-genart-lc-droms-2016-04-28-00

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair.  Please treat these comments just
like any other last call comments.

For more information, please see the FAQ at

<

http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.

Document: draft-ietf-aqm-eval-guidelines-11
Reviewer: Ralph Droms
Review Date: 2016-04-28
IETF LC End Date: 2016-05-04
IESG Telechat date: 2016-05-19

Summary: This draft is on the right track but has open issues, described in the
review.

In general, I think the document could be read, implemented and used to
generate useful characterizations of AQM schemes.  However, the motivations for
some of the measurements and scenarios seems weak to me, which might compromise
the weight given to the conclusions drawn from the guidelines.

Major issues:

None.  However, the list of minor issues and nits, taken together, could be
considered a major issue to be resolved before publication.

Minor issues:

I often react to the use of RFC 2119 language in an Informational document by
asking is that language really necessary?  I'll ask the question here: in the
context of this Informational document, which appears to be entirely advisory
in providing guidelines, what does the use of RFC 2119 "requirements language"
add to the meaning of the document.

Figure 1 is not clear to me.  Where are the physical links and interfaces?  Are
there multiple physical senders and receivers or are "senders A" instantiated
on a single host (does it make a difference)?  Are there static-sized buffers
for each interface or do all the buffers share one memory space?

In section 3.1, is there a need to say something about the relative capacities
of the various links and the rates at which the various flows generate traffic?

I would have trouble following the guidelines set out in section 4.3.1.  I can
understand the need for consideration of the tunable control parameters when
comparing different AQM schemes.  However, I don't know what "comparable" means
for control parameters that are likely quite different between AQM schemes.  I
also think one would want to compare optimal control settings for the different
schemes, to compare best-case performance.  Or, for AQM schemes whose
performance is highly dependent on operational conditions, one might want to
compare settings that are sub-optimal for any particular test condition but
that give better performance over a wide range of conditions.

Section 4.4 seems to give advice to the AQM designer rather than describe
guidelines for characterization.  Section 4.4 should either be rewritten to
give guidelines for structuring measurements to account for varying packet
sizes or the section should be elided.

In section 4.5, what is the motivation for giving the advice about ECN to AQM
designers?  I can understand that ECN will have affect the impact of AQM, but
for this document I think the section should focus on measurement guidlines
that account for that impact.

The specific topology in section 10 does not seem well-motivated to me.  Why is
router R with no AQM included in the topology?  The choice of measurements is
similarly not well-motivated.  Why would it not be of interest to run all the
tests described earlier in the document?

Nits/editorial comments:

There are several instances of the word "advice" which should be replaced with
"advise"; e.g., in section 2.3.

Last sentence of the abstract: I don't get the meaning of "precautionary
characterizations of AQM schemes".  I recommend that the phrase be reworded

Section 1, first paragraph: The last sentence doesn't follow the rest of the
paragraph and I recommend that it be elided.

Section 1, third paragraph: This text is redundant with the text in the
Glossary section:

   When speaking of a specific queue in this
   document, "buffer occupancy" refers to the amount of data (measured
   in bytes or packets) that are in the queue, and the "maximum buffer
   size" refers to the maximum buffer occupancy.

Section 1, third paragraph:

OLD:

   In real
   implementations of switches, a global memory is often shared between
   the available devices, and thus, the maximum buffer size may vary
   over the time.

NEW:

   In switches and routers, a global memory space is often shared
   between the available interfaces, and thus, the maximum buffer size
   for any given interface may vary over the time.

Section 1, fifth paragraph, last sentence: Is this document just concerned with
"deployability" or more generally with "applicability, performance and
deployability"?

Section 1.1, first paragraph: Would it be helpful to qualify "goodput" as
"goodput in individual flows", to contrast with "goodput at a router"?  If
"goodput" is well-known in this community to be "flow goodput", no change is
needed.

Section 1.1, second paragraph: What is "BDP", as in "BDP-sized buffer"?

Section 1.1, third paragraph, first sentence: "environment" is unclear; perhaps
"deployment scenario" or "use case"?  I'm not sure what was meant.

Section 1.3: for completeness, a definition of "latency" would be useful.

Section 2, first paragraph: I'm going to sound pedantic here, but it seems to
me this section is not just intended "to better quantify (1) the reduction of
latency, (2) maximization of goodput and (3) the trade-off between these two"
but also to define other performance metrics that get to the goal from the
abstract of describing "various criteria for performing precautionary
characterizations of AQM schemes"

Section 2, last paragraph: What is "application-limited traffic"?  Later, what
is "non application-limited" traffic?

Section 2.2: s/of/and/

Section 2.5, first paragraph: s/induces/requires/  ???

Section 2.6, second paragraph is just not clear to me.  In particular, what is
the antecedent of "they":

   The introduction of an AQM scheme would impact these
   metrics (end-to-end latency and jitter) and therefore they should be
   considered in the end-to-end evaluation of performance."

Section 5.1: is this new text correct?

OLD:

   The transmission of the non application-limited flow must start
   before the transmission of the application-limited flow and only
   after the steady state has been reached by non application-limited
   flow.

NEW:

   The transmission of the non application-limited flow must start
   first and the transmission of the application-limited flow starts
   after the non application-limited flow has reached steady state.

Throughout section 5, the text like "the graph described in Section 2.7 could
be generated" seems redundant.

Section 6.1, second paragraph: I don't understand the last sentence.  What is
the "type of fairness" that the AQM scheme might affect?

Section 6.2, second paragraph: The words up to the first period don't
constitute a sentence.

In section 7.2, what is "IW10"?  Why are the particular traffic flow chosen in
figure 2?  s/could/should/ in the last paragraph?

Attachment:

signature.asc

Description:

 Message signed with OpenPGP using GPGMail