Skip to main content

Last Call Review of draft-ietf-bmwg-sdn-controller-benchmark-meth-07
review-ietf-bmwg-sdn-controller-benchmark-meth-07-opsdir-lc-bradner-2018-01-29-00

Request Review of draft-ietf-bmwg-sdn-controller-benchmark-meth
Requested revision No specific revision (document currently at 09)
Type Last Call Review
Team Ops Directorate (opsdir)
Deadline 2018-02-02
Requested 2018-01-19
Authors Bhuvaneswaran Vengainathan , Anton Basil , Mark Tassinari , Vishwas Manral , Sarah Banks
Draft last updated 2018-01-29
Completed reviews Rtgdir Last Call review of -07 by Henning Rogge (diff)
Opsdir Last Call review of -07 by Scott O. Bradner (diff)
Genart Last Call review of -07 by Stewart Bryant (diff)
Secdir Last Call review of -07 by Russ Housley (diff)
Genart Telechat review of -08 by Stewart Bryant (diff)
Assignment Reviewer Scott O. Bradner
State Completed
Review review-ietf-bmwg-sdn-controller-benchmark-meth-07-opsdir-lc-bradner-2018-01-29
Reviewed revision 07 (document currently at 09)
Result Has Issues
Completed 2018-01-29
review-ietf-bmwg-sdn-controller-benchmark-meth-07-opsdir-lc-bradner-2018-01-29-00
This ID specifies the methodology to be used in testing the performance of SDN
controllers. It is a standard BMWG methodology document and thus, cannot have
any impact, operational or otherwise, on operational networks - see RFC 6815

follow are some comments on the document itself

section 4.1 - Leaf-Spine used but not defined
section 4.4 - using old software versions seems to add complexity with little
benefit section 4.7 - would be useful to say that the variance over the
repeated tests should be reported in the "test reporting" section - e.g., in
section 5.1.1 I would add a row that reports the variance - same for all tests
section 5.1.1 - is the topology discovery process slow enough that a 3 second
granularity of the measurement will show differences between systems? section
5.1.2 - procedure step 2 - what inserts the timestamp in the response message
(R1)? section 5.1.2 - the ID says "successful messages exchanged" - might it be
useful to report the % of unsuccessful exchanges? section 5.1.6 - the title &
objective description do not match - the title says "rate" but the objective is
a count - the test seems to try to get the rate section 5.1.7 - same issue as
section 5.1.6 section 5.2.3 - procedure step 1 - this reads as if the addresses
are unique but unchanging - if that is not what is meant then it should
specifically say that the addresses are changing section 5.3.1 - it might be
useful to establish specific "invalid messages" since I could imagine that the
devices could handle different types in different ways that could have an
impact on speed of handling section 5.3.2 - "a huge number" is somewhat
undefined
        do you mean to say TCP SYN messages rather than TCP SYNC messages?
section 6 - RFC 2026 is referenced in the introduction but not listed in the
references section 9 - I have now retired so no affiliation should be listed