Data Center Benchmarking Terminology

Note: This ballot was opened for revision 10 and is now closed.

Warren Kumari Yes

Alia Atlas No Objection

Deborah Brungard No Objection

Ben Campbell No Objection

Comment (2017-06-21 for -17)
I find the naming of the draft fairly confusing. It goes way beyond "terminology"; it makes a number of normative (using 2119 language) statements about benchmarking procedures. I wonder why the sections about procedure did not go into the methodology draft instead. In general, I don't think putting normative language in an informational terminology draft is a good idea. (This would have been a DISCUSS, except that I am aware the bmwg has decided to make all its drafts informational and to still use 2119 language. For the record, I think that policy falls down with this draft.)

I agree with the comment from others that this does not seem to be specific to datacenters.

- 2.2: Definitions of "store-and-forward" and "cut-through" when used in this context would be helpful. The first may be obvious, but the best I can do with "cut-through" is assume it means the opposite of "store-and-forward".

- 6.2: After reading the definition of "Incast" several times, I'm still not sure what it means or what is being measured.

Benoit Claise No Objection

Alissa Cooper No Objection

Spencer Dawkins No Objection

Comment (2017-06-20 for -14)
(I'm making the same comment on draft-ietf-bmwg-dcbench-terminology and draft-ietf-bmwg-dcbench-methodology)

I'm looking at the ballot positions on draft-ietf-bmwg-dcbench-terminology and draft-ietf-bmwg-dcbench-methodology that assert these documents aren't specific to "data centers". That wouldn't surprise me, but I'm not seeing a definition of "data center" in either document - did I miss it?

I suspect that the authors have specific technical characteristics in mind, that happen to map onto what data centers look like today, but may not in the future ("RFCs last forever"). Is it possible to tease those characteristics out?

(Full disclosure: my first working group in the IETF could have been called "TCP over cellular links", but it turned out that when we said "cellular links", we meant "low-speed links with high loss rates and asymmetric bandwidth". "Cellular links" in the late 1990s didn't have the same characteristics that "cellular links" have in 2017, but there are other link types with those characteristics, so the documents ended up being useful in places like CORE. I'm not suggesting anything like a restructure of the document(s), only that they be clear about how future readers would know whether they should be reading them in 2027)

Suresh Krishnan No Objection

Mirja Kühlewind No Objection

Comment (2017-06-14 for -10)
I didn't put in a discuss because it’s not worth holding up the document as the content seems fine, BUT I really don't see why this terminology is specified FOR data centers ONLY. All terminology seem to be applicable in general and as such this seems to be an extension of RFC1242. Even incast, which is actually a data center term, is so generally defined that one could use it also in a different context as well. Further latency is defined in both documents, where this drafts mentions RFC1242 and mainly extends the description there. That's in general fine, but I probably would recommend to simply change the title, remove the work data center there, and make this document update RFC 1242. (+ remove the intro text on data centers because the same text is anyway in draft-ietf-bmwg-dcbench-methodology).

Terry Manderson No Objection

Alexey Melnikov No Objection

Kathleen Moriarty No Objection

Alvaro Retana No Objection

Comment (2017-06-19 for -13)
I agree with Mirja.  Given that the text (in the Abstract) already recognize the broader applicability, I would like to see that reflected in a clear way on the title and the contents.

Adam Roach No Objection

Comment (2017-06-21 for -17)
I am surprised to find normative statements in a terminology document. It might be appropriate -- if awkward -- to say things like "this term MUST mean x," but this document includes statements pertaining specifically to what "MUST be measured," which seems well beyond its purported scope. I would suggest either removing all such statements, or clearly expanding the scope of the document (including, and quite importantly, revising its title).

- The first paragraph of Section 6.1 uses plural forms for B, kB, and MB, but singular for GB. Please make these consistent. 
- Typically, data units are capitalized per SI-system prefix rules, which would make "kB" the correct abbreviation for kilobytes, rather than "KB."
- Something has gone well and truly bonkers with the references section formatting.
- Please fix reference [1] so that it correctly points to draft-ietf-bmwg-dcbench-methodology. This will ensure that it is updated to the correct RFC value at publication.

Please expand the following acronyms upon first use;
see for guidance.

 - FPGA - Field-Programmable Gate Array
 - LLDP - Link Layer Discovery Protocol


  ** The abstract seems to contain references ([1]), which it shouldn't. 
     Please replace those with straight textual mentions of the documents in

  == Missing Reference: 'RFC5481' is mentioned on line 285, but not defined

  == Unused Reference: 'RFC5841' is defined on line 732, but no explicit
     reference was found in the text

  ** Obsolete normative reference: RFC 2554 (ref. 'RFC2544') (Obsoleted by
     RFC 4954)