Last Call Review of draft-ietf-bmwg-sdn-controller-benchmark-term-07
review-ietf-bmwg-sdn-controller-benchmark-term-07-opsdir-lc-bonica-2018-01-31-00

Folks,

I have reviewed this document as part of the Operational directorate's ongoing
effort to review all IETF documents being processed by the IESG.  These
comments were written with the intent of improving the operational aspects of
the IETF drafts. Comments that are not addressed in last call may be included
in AD reviews during the IESG review.  Document editors and WG chairs should
treat these comments just like any other last call comments.

Summary: Not ready for publication

(Maybe I am missing something very fundamental. A conversation with the authors
might be appropriate.

Major issues:

2.3.1.1 - If the discovery time begins with the transmission of the first
message and ends with some event after transmission of the last message,
doesn't this metric rely on the speed at which the network device transmits as
well as the speed at which the controller processes messages. Is the controller
really the DUT? Or is it a combination of the controller and the network device.

2.3.1.1 - How does the tester know when the discovery process is complete? Ask
the controller? Observe some behavior at the network device?

2.3.1.1 - Does discovery time depend on number of nodes and richness of
topology? Do we have to standardize these so that we can compare apples to
apples?

2.3.1.2 - Do some messages take longer to process than others? Do we need to
standardize the messages?

2.3.1.3 - Same questions

2.3.1.4 - Does Reactive Path Provisioning time depend on the number of nodes in
the path and the flow control mechanisms implemented (or not implemented) by
network devices? How do we compare apples to apples?

These same issues hit other sections. Maybe I am just being dense. I am open to
a call with the authors.

None

Minor issues:

None

Nits:

None