Telechat Review of draft-ietf-bmwg-sdn-controller-benchmark-term-09
review-ietf-bmwg-sdn-controller-benchmark-term-09-genart-telechat-bryant-2018-04-16-00

Request Review of draft-ietf-bmwg-sdn-controller-benchmark-term
Requested rev. no specific revision
Type Telechat Review
Team General Area Review Team (Gen-ART) (genart)
Deadline 2018-04-17
Requested 2018-02-27
Other Reviews Rtgdir Last Call review of -07 by Ines Robles (diff)
Opsdir Last Call review of -07 by Ron Bonica (diff)
Genart Last Call review of -07 by Stewart Bryant (diff)
Secdir Last Call review of -07 by Paul Hoffman (diff)
Opsdir Telechat review of -09 by Ron Bonica
Review State Completed
Reviewer Stewart Bryant
Review review-ietf-bmwg-sdn-controller-benchmark-term-09-genart-telechat-bryant-2018-04-16
Posted at https://mailarchive.ietf.org/arch/msg/gen-art/mjwGYssuYVKh1uDEe9DsZYHiPbE
Reviewed rev. 09
Review result Ready with Nits
Draft last updated 2018-04-16
Review completed: 2018-04-16

Review
review-ietf-bmwg-sdn-controller-benchmark-term-09-genart-telechat-bryant-2018-04-16

I am the assigned Gen-ART reviewer for this draft. The General Area
Review Team (Gen-ART) reviews all IETF documents being processed
by the IESG for the IETF Chair. Please wait for direction from your
document shepherd or AD before posting a new version of the draft.

For more information, please see the FAQ at

<https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.

Document: draft-ietf-bmwg-sdn-controller-benchmark-term-09
Reviewer: Stewart Bryant
Review Date: 2018-04-16
IETF LC End Date: 2018-02-02
IESG Telechat date: 2018-04-19

Summary: Generally a well written document. The various comments I make are mostly editorial with some on the fringe of being technical.
I do not seem to have received a response to a number of issues raised, and as far as I can see there is no change to the text in the issues noted below. Apologies if I have missed your email. 

Major issues: None

Minor issues:

I do not seem to have received a response to this issue, and as far as I can see there is no change to the text. Apologies if I have missed your email. 

*rate* implies actions per unit time, but this does not seem to feature in your definition.

2.3.1.3. Asynchronous Message Processing Rate

Definition:
   The number responses to asynchronous messages (such as new flow
SB> That should be the number of responses per second.


Discussion:
   As SDN assures flexible network and agile provisioning, it is
   important to measure how many network events the controller can
   handle at a time. This benchmark is obtained by sending asynchronous
   messages from every connected Network Device at the rate that the
   controller processes (without dropping them). 

SB> So what you are testing here is the control network and the 
SB> controller. This is perhaps the only practical way to run the 
SB> test, but it seems a pity that you do not deconvolve these
SB> two aspects of the test.
SB>
SB> I suppose this is really network Async Msg Proc rate rather than
SB> controller Async proc rate.
SB>
SB> We may get to this in the companion document, but doesn't there
SB> need to be some standardization of the event message to compare
SB> apple with apples over time?


Nits/editorial comments: 


 

2.2.4. Number of Cluster nodes


Discussion:
   This parameter is relevant when testing the controller performance
   in clustering/teaming mode. The number of nodes in the cluster MUST
   be greater than 1.

SB> I see what you are saying, but you may wish to clarify that this 
SB> constraint does not apply all the time. For example one of two nodes
SB> may start later than another, or fail, or maybe I worry over nothing here.



2.3.2.1. Control Sessions Capacity

Measurement Units:
SB> Surely this should be in units of sessions?

2.3.2.2. Network Discovery Size

Measurement Units:

   N/A

SB> How can this be N/A surely it is a number of network units of various type.

2.3.2.3. Forwarding Table Capacity