Last Call Review of draft-ietf-bmwg-sdn-controller-benchmark-meth-07
review-ietf-bmwg-sdn-controller-benchmark-meth-07-opsdir-lc-bradner-2018-01-29-00
| Request | Review of | draft-ietf-bmwg-sdn-controller-benchmark-meth |
|---|---|---|
| Requested revision | No specific revision (document currently at 09) | |
| Type | Last Call Review | |
| Team | Ops Directorate (opsdir) | |
| Deadline | 2018-02-02 | |
| Requested | 2018-01-19 | |
| Authors | Bhuvaneswaran Vengainathan , Anton Basil , Mark Tassinari , Vishwas Manral , Sarah Banks | |
| Draft last updated | 2018-01-29 | |
| Completed reviews |
Rtgdir Last Call review of -07
by
Henning Rogge
(diff)
Opsdir Last Call review of -07 by Scott O. Bradner (diff) Genart Last Call review of -07 by Stewart Bryant (diff) Secdir Last Call review of -07 by Russ Housley (diff) Genart Telechat review of -08 by Stewart Bryant (diff) |
|
| Assignment | Reviewer | Scott O. Bradner |
| State | Completed | |
| Review |
review-ietf-bmwg-sdn-controller-benchmark-meth-07-opsdir-lc-bradner-2018-01-29
|
|
| Reviewed revision | 07 (document currently at 09) | |
| Result | Has Issues | |
| Completed | 2018-01-29 |
review-ietf-bmwg-sdn-controller-benchmark-meth-07-opsdir-lc-bradner-2018-01-29-00
This ID specifies the methodology to be used in testing the performance of SDN controllers. It is a standard BMWG methodology document and thus, cannot have any impact, operational or otherwise, on operational networks - see RFC 6815 follow are some comments on the document itself section 4.1 - Leaf-Spine used but not defined section 4.4 - using old software versions seems to add complexity with little benefit section 4.7 - would be useful to say that the variance over the repeated tests should be reported in the "test reporting" section - e.g., in section 5.1.1 I would add a row that reports the variance - same for all tests section 5.1.1 - is the topology discovery process slow enough that a 3 second granularity of the measurement will show differences between systems? section 5.1.2 - procedure step 2 - what inserts the timestamp in the response message (R1)? section 5.1.2 - the ID says "successful messages exchanged" - might it be useful to report the % of unsuccessful exchanges? section 5.1.6 - the title & objective description do not match - the title says "rate" but the objective is a count - the test seems to try to get the rate section 5.1.7 - same issue as section 5.1.6 section 5.2.3 - procedure step 1 - this reads as if the addresses are unique but unchanging - if that is not what is meant then it should specifically say that the addresses are changing section 5.3.1 - it might be useful to establish specific "invalid messages" since I could imagine that the devices could handle different types in different ways that could have an impact on speed of handling section 5.3.2 - "a huge number" is somewhat undefined do you mean to say TCP SYN messages rather than TCP SYNC messages? section 6 - RFC 2026 is referenced in the introduction but not listed in the references section 9 - I have now retired so no affiliation should be listed