Considerations for Benchmarking Virtual Network Functions and Their Infrastructure
draft-ietf-bmwg-virtual-net-05

[Ballot comment]
I have a few mostly editorial comments:

- Abstract and Introduction: Missing "the" before "Benchmarking..."

-Abstract: Will the paragraph about new version history stay in the RFC?

-1: Much of this section, especially the 2nd paragraph, reads like a commercial, or a marketing white paper. I'm not going to put this in the way of publication, but an IETF RFC should generally take a more neutral tone. It's enough to acknowledge that people are doing (or plan to do) NFV.

-2: Language of the form of "BMG will consider" will quickly become dated. Consider something to the effect of "At the time of this writing, BMG is considering/plans to consider..."

Can you offer a definition or citation for "bare metal"?

"Also,  benchmarking combinations of physical and virtual devices and functions in a System Under Test.": Sentence fragment.

- 4.2, first paragraph, last sentence: Can you offer a citation for the 4x3 matrix?
:

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has reviewed draft-ietf-bmwg-virtual-net-04.txt, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
IANA Services Specialist
PTI

The following Last Call announcement was sent out:

From: The IESG
To: "IETF-Announce"
CC: bmwg-chairs@ietf.org, joelja@gmail.com, draft-ietf-bmwg-virtual-net@ietf.org, "Sarah Banks" , bmwg@ietf.org, sbanks@encrypted.net
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Considerations for Benchmarking Virtual Network Functions and Their Infrastructure) to Informational RFC


The IESG has received a request from the Benchmarking Methodology WG
(bmwg) to consider the following document:
- 'Considerations for Benchmarking Virtual Network Functions and Their
  Infrastructure'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-02-15. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  Benchmarking Methodology Working Group has traditionally conducted
  laboratory characterization of dedicated physical implementations of
  internetworking functions.  This memo investigates additional
  considerations when network functions are virtualized and performed
  in general purpose hardware.

  See the new version history section for updates.





The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-virtual-net/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-virtual-net/ballot/


No IPR declarations have been submitted directly on this I-D.

This is the Publication Request and document shepherd write-up for   

Considerations for Benchmarking Virtual Network Functions and Their Infrastructure
  https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-04

This version is dated 9 January 2017.

Sarah Banks is the Document Shepherd, and prepared this form: January 2017.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Informational, as indicated on the title page. All BMWG RFCs are traditionally Informational, in part because they do not define protocols and the traditional conditions for standards track advancement did not apply.  However, they are specifications and the RFC 2119 terms are applicable to identify the level of requirements.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:
This draft examines considerations for benchmarking virtualized network functions (VNFs) and their supported infrastructure, with consideration for general-purpose hardware, and considers benchmarking both physical and virtual network functions in the same fashion, allowing for comparison between the two, where possible.

Working Group Summary:
There has been a fair amount of work done on this draft, and progress made on revisions, feedback, and comments. Several presentations have been made in
the room during IETF meetings, and followup and discussion taken to the BMWG list. This draft is particularly useful, given the popularity of VNF's within the industry.


Document Quality:
This document is ready for publication.

Personnel:

Who is the Document Shepherd? Who is the Responsible Area Director?
Sarah Banks is Shepherd, Joel Jaeggli is AD.


(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

I've reviewed this draft at WGLC. Nits check is clean, with 2 warnings that have no impact on publication.
All comments and feedback by the WG have been addressed by the Author.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No specific reason to call out these reviews here, but the document itself was reviewed in ETSI NFV, and used as a reference by at least one of the TST WG and IFA WF specifications (2 references in total). The document has also been reviewed by several individuals from the OPNFV community, and NFVRG (IRTF) is aware of the draft.


(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

None.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

No IPR has been filed.


(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No IPR disclosed.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

A WGLC was called on March 23, 2016, and closed June 20, 2016. There was significant support for this work at IETF 95, and consensus was achieved amongst the WG.


(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

No nits

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews.

NA

(13) Have all references within this document been identified as either normative or informative?

All references are either Normative or Informative, and marked as such.


(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226).

There are no IANA actions required.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

None.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc.

NA.