Skip to main content

Data Center Benchmarking Methodology
RFC 8239

Revision differences

Document history

Date Rev. By Action
2020-01-21
18 (System) Received changes through RFC Editor sync (added Verified Errata tag)
2019-03-13
18 (System) Received changes through RFC Editor sync (added Errata tag)
2017-08-30
18 (System)
Received changes through RFC Editor sync (created alias RFC 8239, changed abstract to 'The purpose of this informational document is to establish test and …
Received changes through RFC Editor sync (created alias RFC 8239, changed abstract to 'The purpose of this informational document is to establish test and evaluation methodology and measurement techniques for physical network equipment in the data center.  RFC 8238 is a prerequisite for this document, as it contains terminology that is considered normative.  Many of these terms and methods may be applicable beyond the scope of this document as the technologies originally applied in the data center are deployed elsewhere.', changed pages to 19, changed standardization level to Informational, changed state to RFC, added RFC published event at 2017-08-30, changed IESG state to RFC Published)
2017-08-30
18 (System) RFC published
2017-08-24
18 (System) RFC Editor state changed to AUTH48-DONE from AUTH48
2017-08-22
18 Wesley Eddy Closed request for Last Call review by TSVART with state 'Overtaken by Events'
2017-08-09
18 (System) RFC Editor state changed to AUTH48 from RFC-EDITOR
2017-08-01
18 (System) RFC Editor state changed to RFC-EDITOR from AUTH
2017-08-01
18 Gunter Van de Velde Request for Last Call review by OPSDIR Completed: Ready. Reviewer: Scott Bradner.
2017-07-27
18 (System) RFC Editor state changed to AUTH from EDIT
2017-07-06
18 (System) RFC Editor state changed to EDIT
2017-07-06
18 (System) IESG state changed to RFC Ed Queue from Approved-announcement sent
2017-07-06
18 (System) Announcement was received by RFC Editor
2017-07-06
18 (System) IANA Action state changed to No IC from In Progress
2017-07-06
18 (System) IANA Action state changed to In Progress
2017-07-06
18 Cindy Morgan IESG state changed to Approved-announcement sent from IESG Evaluation::AD Followup
2017-07-06
18 Cindy Morgan IESG has approved the document
2017-07-06
18 Cindy Morgan Closed "Approve" ballot
2017-07-06
18 Cindy Morgan Ballot approval text was generated
2017-07-06
18 Cindy Morgan Ballot writeup was changed
2017-07-06
18 Mirja Kühlewind
[Ballot comment]
Clarifying the differences to the tests in RFC2889 addresses my discuss. Thanks and sorry for the delay! I still think describing these tests …
[Ballot comment]
Clarifying the differences to the tests in RFC2889 addresses my discuss. Thanks and sorry for the delay! I still think describing these tests in a more general form and then (just) discuss if there are specific things to consider when applying these tests for dc testing would have been the better approach but I do understand well that this work only focussed on dc so far and it would be quite some more work to make it more generic.

-----

Old comments:

Please provide the appropriate references for DSCP and COS!

Also I find the name of section 6 confusing ("Incast Stateful and Stateless Traffic ") because your microburst test (section 4) is basically also incast testing but without TCP cross-traffic. Further the terms stateful and stateless are also confusing to me; I'd usually use adaptive and constant bit rate (CBR)/non-adaptive; or is stateful/stateless the commonly-used term in benchmarking?
2017-07-06
18 Mirja Kühlewind [Ballot Position Update] Position for Mirja Kühlewind has been changed to No Objection from Discuss
2017-06-24
18 Tero Kivinen Closed request for Last Call review by SECDIR with state 'No Response'
2017-06-22
18 Cindy Morgan IESG state changed to IESG Evaluation::AD Followup from IESG Evaluation
2017-06-22
18 Michelle Cotton IANA Review state changed to IANA OK - No Actions Needed from Version Changed - Review Needed
2017-06-22
18 Alexey Melnikov [Ballot Position Update] New position, No Objection, has been recorded for Alexey Melnikov
2017-06-21
18 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-18.txt
2017-06-21
18 (System) New version approved
2017-06-21
18 (System) Request for posting confirmation emailed to previous authors: Lucien Avramov , " jhrapp@gmail.com"
2017-06-21
18 Lucien Avramov Uploaded new revision
2017-06-21
17 Alvaro Retana [Ballot comment]
[Thanks for addressing my DISCUSS.]
2017-06-21
17 Alvaro Retana [Ballot Position Update] Position for Alvaro Retana has been changed to No Objection from Discuss
2017-06-21
17 (System) IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed
2017-06-21
17 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-17.txt
2017-06-21
17 (System) New version approved
2017-06-21
17 (System) Request for posting confirmation emailed to previous authors: Lucien Avramov , " jhrapp@gmail.com"
2017-06-21
17 Lucien Avramov Uploaded new revision
2017-06-21
16 Adam Roach
[Ballot comment]
I agree with Alvaro's discuss -- you can't cite 2119 and then override it. It may well be that what 2119 does not …
[Ballot comment]
I agree with Alvaro's discuss -- you can't cite 2119 and then override it. It may well be that what 2119 does not make sense for this document; if so, don't cite it, and be clear up front that you're using a *different* set of meanings for these specific terms.

I *think* I can suss out the nature of "east-west" versus "north-south" in the introduction, but I'm really not sure. Can you please define these terms or point to a document that does so?

Editorial:

- Something has gone well and truly bonkers with the references section formatting.

- Please fix reference [1] so that it correctly points to draft-ietf-bmwg-dcbench-terminology. This will ensure that it is updated to the correct RFC value at publication.

Nits:

Please expand "SUT" on first use.

  ** The abstract seems to contain references ([1]), which it shouldn't.
    Please replace those with straight textual mentions of the documents in
    question.
2017-06-21
16 Adam Roach [Ballot Position Update] Position for Adam Roach has been changed to No Objection from No Record
2017-06-21
16 Alia Atlas [Ballot Position Update] New position, No Objection, has been recorded for Alia Atlas
2017-06-21
16 Kathleen Moriarty [Ballot Position Update] New position, No Objection, has been recorded for Kathleen Moriarty
2017-06-21
16 Adam Roach
[Ballot comment]
Nits:

Please expand "SUT" on first use.

  ** The abstract seems to contain references ([1]), which it shouldn't.
    Please replace …
[Ballot comment]
Nits:

Please expand "SUT" on first use.

  ** The abstract seems to contain references ([1]), which it shouldn't.
    Please replace those with straight textual mentions of the documents in
    question.
2017-06-21
16 Adam Roach Ballot comment text updated for Adam Roach
2017-06-21
16 Amanda Baber IANA Review state changed to IANA OK - No Actions Needed from Version Changed - Review Needed
2017-06-21
16 Adam Roach [Ballot comment]
Please expand "SUT" on first use.
2017-06-21
16 Adam Roach Ballot comment text updated for Adam Roach
2017-06-21
16 Ben Campbell
[Ballot comment]
- I support Alvaro's DISCUSS

- I agree with the questions about why this is specific to datacenters.

- Please expand DUT on …
[Ballot comment]
- I support Alvaro's DISCUSS

- I agree with the questions about why this is specific to datacenters.

- Please expand DUT on first use.
2017-06-21
16 Ben Campbell [Ballot Position Update] New position, No Objection, has been recorded for Ben Campbell
2017-06-21
16 Suresh Krishnan
[Ballot comment]
Not sure if the WG discussed this, but it would have been interesting to have some energy efficiency related metrics as the draft …
[Ballot comment]
Not sure if the WG discussed this, but it would have been interesting to have some energy efficiency related metrics as the draft is datacenter focused and energy efficiency is an important consideration in datacenters.
2017-06-21
16 Suresh Krishnan Ballot comment text updated for Suresh Krishnan
2017-06-21
16 Suresh Krishnan
[Ballot comment]
Not sure if the WG discussed this, but it would have been interesting to have something energy efficiency related metrics (in both the …
[Ballot comment]
Not sure if the WG discussed this, but it would have been interesting to have something energy efficiency related metrics (in both the terminology and the methodology documents).
2017-06-21
16 Suresh Krishnan [Ballot Position Update] New position, No Objection, has been recorded for Suresh Krishnan
2017-06-21
16 Alissa Cooper [Ballot Position Update] New position, No Objection, has been recorded for Alissa Cooper
2017-06-21
16 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-16.txt
2017-06-21
16 (System) New version approved
2017-06-21
16 (System) Request for posting confirmation emailed to previous authors: Lucien Avramov , " jhrapp@gmail.com"
2017-06-21
16 Lucien Avramov Uploaded new revision
2017-06-21
15 Benoît Claise
[Ballot comment]
At some point in time, I was wondering if this document was about (benchmarking) terminology or about (benchmarking) performance metric definition?
And I …
[Ballot comment]
At some point in time, I was wondering if this document was about (benchmarking) terminology or about (benchmarking) performance metric definition?
And I was wondering what the connection was with the RFC 6390 PMOL template and with https://tools.ietf.org/html/draft-ietf-ippm-metric-registry-11?
When I look at the section 3 "jitter", there are obvious elements from the PMOL template.
Thinking about it, whether a performance metric is used for benchmark or not, we need a formal definition.
Is it time for:
1. IPPM to finish up draft-ietf-ippm-metric-registry
2. start populating the registry
3. BMWG to start using/specifying those performance metrics

Thoughts?

Below is Tim Chown's OPS DIR review:
verall, the document is well written, and I believe it to be Ready with minor
nits.

General comment:

It would be interesting to see an Appendix with an example of a recorded test
using the language defined here.

Specific comments:

Section 1:

“- Low amount of buffer (in the Mb range)”
Change to MB?  (given later you refer to KB/MB/GB as the measurement unit for
buffers)

Section 2.1

Expand DUT on first use.

Section 3.1

Perhaps clarify relationship of Delay and Latency, since you focus on Latency
in the document and not Delay?

Last para, you say “If” here, but for Latency the FILO timestamp was a MUST in
Section 2?  This doesn’t seem consistent?

Expand PDV on first use.

Section 6.1.1

“1518 bytes frames” -> “1518 byte frames”

Section 7.1,

Why is ‘and’ in quotes here?  Not sure you can say the balance is defined by
goodput?  Do you mean that goodput is an indication of the balance?  For
standard TCP, a very small loss can have a dramatic effect on application
throughput.

The second para should follow the first, change “[RFC2647].  Goodput…” to
“[RFC2647], i.e., goodput…”

Section 7.3

I don’t understand how the example given correlates to G = S/Ft ?

There’s a few typos in this section; please re-check.
2017-06-21
15 Benoît Claise Ballot comment text updated for Benoit Claise
2017-06-21
15 Benoît Claise [Ballot Position Update] New position, No Objection, has been recorded for Benoit Claise
2017-06-20
15 Terry Manderson [Ballot comment]
There are two discusses in place that perfectly capture my thoughts.
2017-06-20
15 Terry Manderson [Ballot Position Update] New position, No Objection, has been recorded for Terry Manderson
2017-06-20
15 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-15.txt
2017-06-20
15 (System) New version approved
2017-06-20
15 (System) Request for posting confirmation emailed to previous authors: Lucien Avramov , " jhrapp@gmail.com"
2017-06-20
15 Lucien Avramov Uploaded new revision
2017-06-20
14 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-14.txt
2017-06-20
14 (System) New version approved
2017-06-20
14 (System) Request for posting confirmation emailed to previous authors: Lucien Avramov , " jhrapp@gmail.com"
2017-06-20
14 Lucien Avramov Uploaded new revision
2017-06-20
13 Spencer Dawkins
[Ballot comment]
(I'm making the same comment on draft-ietf-bmwg-dcbench-terminology and draft-ietf-bmwg-dcbench-methodology)

I'm looking at the ballot positions on draft-ietf-bmwg-dcbench-terminology and draft-ietf-bmwg-dcbench-methodology that assert these …
[Ballot comment]
(I'm making the same comment on draft-ietf-bmwg-dcbench-terminology and draft-ietf-bmwg-dcbench-methodology)

I'm looking at the ballot positions on draft-ietf-bmwg-dcbench-terminology and draft-ietf-bmwg-dcbench-methodology that assert these documents aren't specific to "data centers". That wouldn't surprise me, but I'm not seeing a definition of "data center" in either document - did I miss it?

I suspect that the authors have specific technical characteristics in mind, that happen to map onto what data centers look like today, but may not in the future ("RFCs last forever"). Is it possible to tease those characteristics out?

(Full disclosure: my first working group in the IETF could have been called "TCP over cellular links", but it turned out that when we said "cellular links", we meant "low-speed links with high loss rates and asymmetric bandwidth". "Cellular links" in the late 1990s didn't have the same characteristics that "cellular links" have in 2017, but there are other link types with those characteristics, so the documents ended up being useful in places like CORE. I'm not suggesting anything like a restructure of the document(s), only that they be clear about how future readers would know whether they should be reading them in 2027)
2017-06-20
13 Spencer Dawkins [Ballot Position Update] New position, No Objection, has been recorded for Spencer Dawkins
2017-06-20
13 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-13.txt
2017-06-20
13 (System) New version approved
2017-06-20
13 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , Lucien Avramov , bmwg-chairs@ietf.org
2017-06-20
13 Lucien Avramov Uploaded new revision
2017-06-20
12 Deborah Brungard [Ballot Position Update] New position, No Objection, has been recorded for Deborah Brungard
2017-06-19
12 Alvaro Retana
[Ballot discuss]
This document contains both the RFC2119 boilerplate and specific definitions for MUST, SHOULD/RECOMMENDED and MAY. Even though the additional definitions are close enough …
[Ballot discuss]
This document contains both the RFC2119 boilerplate and specific definitions for MUST, SHOULD/RECOMMENDED and MAY. Even though the additional definitions are close enough to RFC2119, I think that one of them should be taken off to avoid any type of confusion.

Note that the (new) definitions in this document are focused around "metrics", which would not apply to text such as (from 2.2) "A traffic generator SHOULD be connected to all ports on the DUT. Two tests MUST be conducted...".
2017-06-19
12 Alvaro Retana
[Ballot comment]
I support Mirja's DISCUSS.  The authors recognize that the applicability of the tests described may go beyond the DC, so it should be …
[Ballot comment]
I support Mirja's DISCUSS.  The authors recognize that the applicability of the tests described may go beyond the DC, so it should be reflected appropriately (starting with the title of the document).  From the Abstract:

  The purpose of this informational document is to establish test and
  evaluation methodology and measurement techniques for physical
  network equipment in the data center. Many of these terms and methods
  may be applicable beyond this publication's scope as the technologies
  originally applied in the data center are deployed elsewhere.
2017-06-19
12 Alvaro Retana [Ballot Position Update] New position, Discuss, has been recorded for Alvaro Retana
2017-06-19
12 Dan Romascanu Request for Telechat review by GENART Completed: Ready. Reviewer: Dan Romascanu. Sent review to list.
2017-06-18
12 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-12.txt
2017-06-18
12 (System) New version approved
2017-06-18
12 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , Lucien Avramov
2017-06-18
12 Lucien Avramov Uploaded new revision
2017-06-18
11 (System) IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed
2017-06-18
11 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-11.txt
2017-06-18
11 (System) New version approved
2017-06-18
11 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , Lucien Avramov
2017-06-18
11 Lucien Avramov Uploaded new revision
2017-06-16
10 (System) IANA Review state changed to IANA OK - No Actions Needed from Version Changed - Review Needed
2017-06-15
10 (System) IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed
2017-06-15
10 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-10.txt
2017-06-15
10 (System) New version approved
2017-06-15
10 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , Lucien Avramov
2017-06-15
10 Lucien Avramov Uploaded new revision
2017-06-15
09 Jean Mahoney Request for Telechat review by GENART is assigned to Dan Romascanu
2017-06-15
09 Jean Mahoney Request for Telechat review by GENART is assigned to Dan Romascanu
2017-06-14
09 Mirja Kühlewind
[Ballot discuss]
Again, I don't really understand why any of these tests are specific for data center equipment. These tests can be applied to any …
[Ballot discuss]
Again, I don't really understand why any of these tests are specific for data center equipment. These tests can be applied to any equipment. I think the only point is that these tests are especially important to run for data center equipment but that doesn't change the fact that the test itself are not specific for data center equipment. The reason why it's a discuss this time, is that I further really don't understand why existing tests have to be specified again is this document even though a test for that case already exists that specifies exactly the same thing in another RFC; I'm talking about Section 5 on Head of Line Blocking that specifies the same test as in section 5.5 of RFC2889 (there called Congestion Control which a confusing term because this doesn't address end-to-end congestion control, anyway...). I didn't check if that's also the case for any of the other tests.
2017-06-14
09 Mirja Kühlewind
[Ballot comment]
Please provide the appropriate references for DSCP and COS!

Also I find the name of section 6 confusing ("Incast Stateful and Stateless Traffic …
[Ballot comment]
Please provide the appropriate references for DSCP and COS!

Also I find the name of section 6 confusing ("Incast Stateful and Stateless Traffic ") because your microburst test (section 4) is basically also incast testing but without TCP cross-traffic. Further the terms stateful and stateless are also confusing to me; I'd usually use adaptive and constant bit rate (CBR)/non-adaptive; or is stateful/stateless the commonly-used term in benchmarking?
2017-06-14
09 Mirja Kühlewind [Ballot Position Update] New position, Discuss, has been recorded for Mirja Kühlewind
2017-06-13
09 Warren Kumari Placed on agenda for telechat - 2017-06-22
2017-06-13
09 Warren Kumari IESG state changed to IESG Evaluation from Waiting for Writeup
2017-06-13
09 Warren Kumari Ballot has been issued
2017-06-13
09 Warren Kumari [Ballot Position Update] New position, Yes, has been recorded for Warren Kumari
2017-06-13
09 Warren Kumari Created "Approve" ballot
2017-06-13
09 Warren Kumari Ballot writeup was changed
2017-06-13
09 (System) IESG state changed to Waiting for Writeup from In Last Call
2017-06-09
09 (System) IANA Review state changed to IANA OK - No Actions Needed from IANA - Review Needed
2017-06-09
09 Sabrina Tanamal
(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has reviewed draft-ietf-bmwg-dcbench-methodology-07.txt, which is currently in Last Call, and has the following comments:

We …
(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has reviewed draft-ietf-bmwg-dcbench-methodology-07.txt, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
IANA Services Specialist
PTI
2017-06-08
09 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-09.txt
2017-06-08
09 (System) New version approved
2017-06-08
09 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , Lucien Avramov
2017-06-08
09 Lucien Avramov Uploaded new revision
2017-06-08
08 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-08.txt
2017-06-08
08 (System) New version approved
2017-06-08
08 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , Lucien Avramov
2017-06-08
08 Lucien Avramov Uploaded new revision
2017-06-08
07 Dan Romascanu Request for Last Call review by GENART Completed: Ready with Issues. Reviewer: Dan Romascanu. Sent review to list.
2017-06-06
07 Martin Stiemerling Request for Last Call review by TSVART is assigned to Lixia Zhang
2017-06-06
07 Martin Stiemerling Request for Last Call review by TSVART is assigned to Lixia Zhang
2017-06-02
07 Tero Kivinen Request for Last Call review by SECDIR is assigned to Russ Mundy
2017-06-02
07 Tero Kivinen Request for Last Call review by SECDIR is assigned to Russ Mundy
2017-06-01
07 Jean Mahoney Request for Last Call review by GENART is assigned to Dan Romascanu
2017-06-01
07 Jean Mahoney Request for Last Call review by GENART is assigned to Dan Romascanu
2017-05-31
07 Gunter Van de Velde Request for Last Call review by OPSDIR is assigned to Scott Bradner
2017-05-31
07 Gunter Van de Velde Request for Last Call review by OPSDIR is assigned to Scott Bradner
2017-05-30
07 Cindy Morgan IANA Review state changed to IANA - Review Needed
2017-05-30
07 Cindy Morgan
The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC: Sarah Banks , bmwg-chairs@ietf.org, bmwg@ietf.org, draft-ietf-bmwg-dcbench-methodology@ietf.org, sbanks@encrypted.net, …
The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC: Sarah Banks , bmwg-chairs@ietf.org, bmwg@ietf.org, draft-ietf-bmwg-dcbench-methodology@ietf.org, sbanks@encrypted.net, warren@kumari.net
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Data Center Benchmarking Methodology) to Informational RFC


The IESG has received a request from the Benchmarking Methodology WG
(bmwg) to consider the following document:
- 'Data Center Benchmarking Methodology'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-06-13. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  The purpose of this informational document is to establish test and
  evaluation methodology and measurement techniques for physical
  network equipment in the data center.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-dcbench-methodology/ballot/


No IPR declarations have been submitted directly on this I-D.




2017-05-30
07 Cindy Morgan IESG state changed to In Last Call from Last Call Requested
2017-05-30
07 Warren Kumari Changed consensus to Yes from Unknown
2017-05-30
07 Warren Kumari Last call was requested
2017-05-30
07 Warren Kumari Last call announcement was generated
2017-05-30
07 Warren Kumari Ballot approval text was generated
2017-05-30
07 Warren Kumari Ballot writeup was generated
2017-05-30
07 Warren Kumari IESG state changed to Last Call Requested from AD Evaluation
2017-05-30
07 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-07.txt
2017-05-30
07 (System) New version approved
2017-05-30
07 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , Lucien Avramov , bmwg-chairs@ietf.org
2017-05-30
07 Lucien Avramov Uploaded new revision
2017-05-28
06 Warren Kumari IESG state changed to AD Evaluation from Publication Requested
2017-05-26
06 Sarah Banks
This is the Publication Request and document shepherd write-up for   

Data Center Benchmarking Methodology and Terminology
  https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-08
  https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-06


This version is dated …
This is the Publication Request and document shepherd write-up for   

Data Center Benchmarking Methodology and Terminology
  https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-08
  https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-methodology-06


This version is dated 11 May 2017.

Sarah Banks is the Document Shepherd, and prepared this form: May 2017.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Informational, as indicated on the title page. All BMWG RFCs are traditionally Informational, in part because they do not define protocols and the traditional conditions for standards track advancement did not apply.  However, they are specifications and the RFC 2119 terms are applicable to identify the level of requirements.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:
This draft establishes definitions and techniques for benchmarking within the data center. The draft defines a set of definitions, metrics, methodologies and terminologies for benchmarking in a wide variety of traffic conditions.


Working Group Summary:
There has been an extensive amount of work done on this draft, and progress made on revisions, feedback, and comments. There have been extensive conversations on, for example, "goodput". This is one of the most reviewed drafts we've had in BMWG in quite some time, and the document has undergone 2 WGLCs.


Document Quality:
This document is ready for publication.

Personnel:

Who is the Document Shepherd? Who is the Responsible Area Director?
Sarah Banks is the Shepherd, Warren Kumari is the AD.


(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

I've reviewed this draft at WGLC. Nits check is clean. All comments and feedback by the WG have been addressed by the Author.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No. This document has been thoroughly reviewed.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No. There was extensive discussion and review and revision of this draft, over a long period of time, and the draft underwent 2 WGLCs.


(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

None.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

No IPR has been filed.


(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No IPR disclosed.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

A second WGLC was called on September 13, 2016, and closed on September 27, 2016. There has been a significant amount of feedback on this draft, reviewed in meetings and on the list, and the latest revisions of the draft include and account for feedback from IETF98. There is solid consensus that this draft is ready for publication.


(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

No nits.

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews.

NA

(13) Have all references within this document been identified as either normative or informative?

All references are either Normative or Informative, and marked as such.


(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

None

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226).

There are no IANA actions required.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

None.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc.

NA.
2017-05-26
06 Sarah Banks Responsible AD changed to Warren Kumari
2017-05-26
06 Sarah Banks IETF WG state changed to Submitted to IESG for Publication from WG Document
2017-05-26
06 Sarah Banks IESG state changed to Publication Requested
2017-05-26
06 Sarah Banks IESG process started in state Publication Requested
2017-05-16
06 Sarah Banks Intended Status changed to Informational from None
2017-05-16
06 Sarah Banks Changed document writeup
2017-05-11
06 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-06.txt
2017-05-11
06 (System) New version approved
2017-05-11
06 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , Lucien Avramov , bmwg-chairs@ietf.org
2017-05-11
06 Lucien Avramov Uploaded new revision
2017-05-11
05 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-05.txt
2017-05-11
05 (System) New version approved
2017-05-11
05 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , Lucien Avramov , bmwg-chairs@ietf.org
2017-05-11
05 Lucien Avramov Uploaded new revision
2017-04-26
04 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-04.txt
2017-04-26
04 (System) New version approved
2017-04-26
04 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , Lucien Avramov , bmwg-chairs@ietf.org
2017-04-26
04 Lucien Avramov Uploaded new revision
2017-03-11
03 Al Morton Notification list changed to Sarah Banks <sbanks@encrypted.net>
2017-03-11
03 Al Morton Document shepherd changed to Sarah Banks
2016-12-30
03 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-03.txt
2016-12-30
03 (System) New version approved
2016-12-30
03 (System) Request for posting confirmation emailed to previous authors: " jhrapp@gmail.com" , bmwg-chairs@ietf.org, "Lucien Avramov"
2016-12-30
03 Lucien Avramov Uploaded new revision
2016-10-29
02 (System) Document has expired
2016-07-19
02 Al Morton Added to session: IETF-96: bmwg  Wed-1000
2016-04-27
02 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-02.txt
2016-04-04
01 Al Morton Added to session: IETF-95: bmwg  Thu-1000
2015-10-19
01 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-01.txt
2015-06-18
00 Al Morton This document now replaces draft-bmwg-dcbench-methodology instead of None
2015-06-18
00 Lucien Avramov New version available: draft-ietf-bmwg-dcbench-methodology-00.txt