Skip to main content

Considerations for Benchmarking Virtual Network Functions and Their Infrastructure
RFC 8172

Revision differences

Document history

Date Rev. By Action
2017-07-31
05 (System)
Received changes through RFC Editor sync (created alias RFC 8172, changed abstract to 'The Benchmarking Methodology Working Group has traditionally conducted laboratory characterization of …
Received changes through RFC Editor sync (created alias RFC 8172, changed abstract to 'The Benchmarking Methodology Working Group has traditionally conducted laboratory characterization of dedicated physical implementations of internetworking functions.  This memo investigates additional considerations when network functions are virtualized and performed in general-purpose hardware.', changed standardization level to Informational, changed state to RFC, added RFC published event at 2017-07-31, changed IESG state to RFC Published)
2017-07-31
05 (System) RFC published
2017-07-12
05 (System) RFC Editor state changed to AUTH48-DONE from AUTH48
2017-05-30
05 (System) RFC Editor state changed to AUTH48 from RFC-EDITOR
2017-05-03
05 (System) RFC Editor state changed to RFC-EDITOR from EDIT
2017-04-11
05 Wesley Eddy Closed request for Last Call review by TSVART with state 'Overtaken by Events'
2017-03-20
05 (System) RFC Editor state changed to EDIT
2017-03-20
05 (System) IESG state changed to RFC Ed Queue from Approved-announcement sent
2017-03-20
05 (System) Announcement was received by RFC Editor
2017-03-20
05 (System) IANA Action state changed to No IC
2017-03-20
05 Amy Vezza IESG state changed to Approved-announcement sent from Approved-announcement to be sent
2017-03-20
05 Amy Vezza IESG has approved the document
2017-03-20
05 Amy Vezza Closed "Approve" ballot
2017-03-20
05 Amy Vezza Ballot approval text was generated
2017-03-17
05 (System) IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed
2017-03-17
05 Cindy Morgan New version available: draft-ietf-bmwg-virtual-net-05.txt
2017-03-17
05 (System) Secretariat manually posting. Approvals already received
2017-03-17
05 Cindy Morgan Uploaded new revision
2017-03-16
04 Cindy Morgan IESG state changed to Approved-announcement to be sent from IESG Evaluation
2017-03-16
04 Joel Jaeggli Changed consensus to Yes from Unknown
2017-03-16
04 Stephen Farrell [Ballot Position Update] New position, No Objection, has been recorded for Stephen Farrell
2017-03-16
04 Kathleen Moriarty [Ballot comment]
Thanks for a well-written draft!
2017-03-16
04 Kathleen Moriarty Ballot comment text updated for Kathleen Moriarty
2017-03-16
04 Kathleen Moriarty [Ballot Position Update] New position, Yes, has been recorded for Kathleen Moriarty
2017-03-16
04 Benoît Claise [Ballot Position Update] New position, No Objection, has been recorded for Benoit Claise
2017-03-15
04 Suresh Krishnan [Ballot Position Update] New position, No Objection, has been recorded for Suresh Krishnan
2017-03-15
04 Ben Campbell
[Ballot comment]
I have a few mostly editorial comments:

- Abstract and Introduction: Missing "the" before "Benchmarking..."

-Abstract: Will the paragraph about new version history …
[Ballot comment]
I have a few mostly editorial comments:

- Abstract and Introduction: Missing "the" before "Benchmarking..."

-Abstract: Will the paragraph about new version history stay in the RFC?

-1: Much of this section, especially the 2nd paragraph, reads like a commercial, or a marketing white paper. I'm not going to put this in the way of publication, but an IETF RFC should generally take a more neutral tone. It's enough to acknowledge that people are doing (or plan to do) NFV.

-2: Language of the form of "BMG will consider" will quickly become dated. Consider something to the effect of "At the time of this writing, BMG is considering/plans to consider..."

Can you offer a definition or citation for "bare metal"?

"Also,  benchmarking combinations of physical and virtual devices and functions in a System Under Test.": Sentence fragment.

- 4.2, first paragraph, last sentence: Can you offer a citation for the 4x3 matrix?
:
2017-03-15
04 Ben Campbell [Ballot Position Update] New position, No Objection, has been recorded for Ben Campbell
2017-03-15
04 Jari Arkko [Ballot Position Update] New position, No Objection, has been recorded for Jari Arkko
2017-03-15
04 Alia Atlas [Ballot comment]
A clear and well-written doc - thanks!
2017-03-15
04 Alia Atlas [Ballot Position Update] New position, No Objection, has been recorded for Alia Atlas
2017-03-15
04 Deborah Brungard [Ballot Position Update] New position, No Objection, has been recorded for Deborah Brungard
2017-03-14
04 Alvaro Retana [Ballot Position Update] New position, No Objection, has been recorded for Alvaro Retana
2017-03-14
04 Mirja Kühlewind [Ballot comment]
Thanks for the well written doc!
2017-03-14
04 Mirja Kühlewind [Ballot Position Update] New position, No Objection, has been recorded for Mirja Kühlewind
2017-02-20
04 Joel Jaeggli IESG state changed to IESG Evaluation from Waiting for Writeup
2017-02-20
04 Joel Jaeggli Ballot has been issued
2017-02-20
04 Joel Jaeggli [Ballot Position Update] New position, Yes, has been recorded for Joel Jaeggli
2017-02-20
04 Joel Jaeggli Created "Approve" ballot
2017-02-20
04 Joel Jaeggli Ballot writeup was changed
2017-02-20
04 Joel Jaeggli Placed on agenda for telechat - 2017-03-16
2017-02-15
04 Jouni Korhonen Request for Last Call review by OPSDIR Completed: Ready. Reviewer: Jouni Korhonen. Sent review to list.
2017-02-15
04 (System) IESG state changed to Waiting for Writeup from In Last Call
2017-02-10
04 Martin Stiemerling Request for Last Call review by TSVART is assigned to Janardhan Iyengar
2017-02-10
04 Martin Stiemerling Request for Last Call review by TSVART is assigned to Janardhan Iyengar
2017-02-05
04 Joel Halpern Request for Last Call review by GENART Completed: Ready. Reviewer: Joel Halpern. Sent review to list.
2017-02-03
04 Gunter Van de Velde Request for Last Call review by OPSDIR is assigned to Jouni Korhonen
2017-02-03
04 Gunter Van de Velde Request for Last Call review by OPSDIR is assigned to Jouni Korhonen
2017-02-02
04 Jean Mahoney Request for Last Call review by GENART is assigned to Joel Halpern
2017-02-02
04 Jean Mahoney Request for Last Call review by GENART is assigned to Joel Halpern
2017-02-02
04 (System) IANA Review state changed to IANA OK - No Actions Needed from IANA - Review Needed
2017-02-02
04 Sabrina Tanamal
(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has reviewed draft-ietf-bmwg-virtual-net-04.txt, which is currently in Last Call, and has the following comments:

We …
(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has reviewed draft-ietf-bmwg-virtual-net-04.txt, which is currently in Last Call, and has the following comments:

We understand that this document doesn't require any registry actions.

While it's often helpful for a document's IANA Considerations section to remain in place upon publication even if there are no actions, if the authors strongly prefer to remove it, we do not object.

If this assessment is not accurate, please respond as soon as possible.

Thank you,

Sabrina Tanamal
IANA Services Specialist
PTI
2017-02-02
04 Sean Turner Request for Last Call review by SECDIR Completed: Ready. Reviewer: Sean Turner. Sent review to list.
2017-02-02
04 Tero Kivinen Request for Last Call review by SECDIR is assigned to Sean Turner
2017-02-02
04 Tero Kivinen Request for Last Call review by SECDIR is assigned to Sean Turner
2017-02-01
04 Cindy Morgan IANA Review state changed to IANA - Review Needed
2017-02-01
04 Cindy Morgan
The following Last Call announcement was sent out:

From: The IESG
To: "IETF-Announce"
CC: bmwg-chairs@ietf.org, joelja@gmail.com, draft-ietf-bmwg-virtual-net@ietf.org, "Sarah Banks" , bmwg@ietf.org, …
The following Last Call announcement was sent out:

From: The IESG
To: "IETF-Announce"
CC: bmwg-chairs@ietf.org, joelja@gmail.com, draft-ietf-bmwg-virtual-net@ietf.org, "Sarah Banks" , bmwg@ietf.org, sbanks@encrypted.net
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Considerations for Benchmarking Virtual Network Functions and Their Infrastructure) to Informational RFC


The IESG has received a request from the Benchmarking Methodology WG
(bmwg) to consider the following document:
- 'Considerations for Benchmarking Virtual Network Functions and Their
  Infrastructure'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2017-02-15. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  Benchmarking Methodology Working Group has traditionally conducted
  laboratory characterization of dedicated physical implementations of
  internetworking functions.  This memo investigates additional
  considerations when network functions are virtualized and performed
  in general purpose hardware.

  See the new version history section for updates.





The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-virtual-net/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-bmwg-virtual-net/ballot/


No IPR declarations have been submitted directly on this I-D.




2017-02-01
04 Cindy Morgan IESG state changed to In Last Call from Last Call Requested
2017-02-01
04 Joel Jaeggli Last call was requested
2017-02-01
04 Joel Jaeggli Last call announcement was generated
2017-02-01
04 Joel Jaeggli Ballot approval text was generated
2017-02-01
04 Joel Jaeggli Ballot writeup was generated
2017-02-01
04 Joel Jaeggli IESG state changed to Last Call Requested from AD Evaluation
2017-01-21
04 Joel Jaeggli IESG state changed to AD Evaluation from Publication Requested
2017-01-20
04 Sarah Banks
This is the Publication Request and document shepherd write-up for   

Considerations for Benchmarking Virtual Network Functions and Their Infrastructure
  https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-04

This version is …
This is the Publication Request and document shepherd write-up for   

Considerations for Benchmarking Virtual Network Functions and Their Infrastructure
  https://tools.ietf.org/html/draft-ietf-bmwg-virtual-net-04

This version is dated 9 January 2017.

Sarah Banks is the Document Shepherd, and prepared this form: January 2017.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Informational, as indicated on the title page. All BMWG RFCs are traditionally Informational, in part because they do not define protocols and the traditional conditions for standards track advancement did not apply.  However, they are specifications and the RFC 2119 terms are applicable to identify the level of requirements.

(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:
This draft examines considerations for benchmarking virtualized network functions (VNFs) and their supported infrastructure, with consideration for general-purpose hardware, and considers benchmarking both physical and virtual network functions in the same fashion, allowing for comparison between the two, where possible.

Working Group Summary:
There has been a fair amount of work done on this draft, and progress made on revisions, feedback, and comments. Several presentations have been made in
the room during IETF meetings, and followup and discussion taken to the BMWG list. This draft is particularly useful, given the popularity of VNF's within the industry.


Document Quality:
This document is ready for publication.

Personnel:

Who is the Document Shepherd? Who is the Responsible Area Director?
Sarah Banks is Shepherd, Joel Jaeggli is AD.


(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

I've reviewed this draft at WGLC. Nits check is clean, with 2 warnings that have no impact on publication.
All comments and feedback by the WG have been addressed by the Author.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No specific reason to call out these reviews here, but the document itself was reviewed in ETSI NFV, and used as a reference by at least one of the TST WG and IFA WF specifications (2 references in total). The document has also been reviewed by several individuals from the OPNFV community, and NFVRG (IRTF) is aware of the draft.


(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

None.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

No IPR has been filed.


(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

No IPR disclosed.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

A WGLC was called on March 23, 2016, and closed June 20, 2016. There was significant support for this work at IETF 95, and consensus was achieved amongst the WG.


(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

No nits

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, media type, and URI type reviews.

NA

(13) Have all references within this document been identified as either normative or informative?

All references are either Normative or Informative, and marked as such.


(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 5226).

There are no IANA actions required.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

None.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, etc.

NA.
2017-01-20
04 Sarah Banks Responsible AD changed to Joel Jaeggli
2017-01-20
04 Sarah Banks IETF WG state changed to Submitted to IESG for Publication from WG Consensus: Waiting for Write-Up
2017-01-20
04 Sarah Banks IESG state changed to Publication Requested
2017-01-20
04 Sarah Banks IESG process started in state Publication Requested
2017-01-20
04 Sarah Banks Changed document writeup
2016-11-12
04 Sarah Banks IETF WG state changed to WG Consensus: Waiting for Write-Up from WG Document
2016-11-12
04 Al Morton Notification list changed to "Sarah Banks" <sbanks@encrypted.net>
2016-11-12
04 Al Morton Document shepherd changed to Sarah Banks
2016-11-09
04 Al Morton Added to session: IETF-97: bmwg  Tue-0930
2016-08-14
04 Al Morton New version available: draft-ietf-bmwg-virtual-net-04.txt
2016-06-19
03 Al Morton New version available: draft-ietf-bmwg-virtual-net-03.txt
2016-04-04
02 Al Morton Added to session: IETF-95: bmwg  Thu-1000
2016-03-21
02 Al Morton New version available: draft-ietf-bmwg-virtual-net-02.txt
2015-11-04
01 Al Morton Intended Status changed to Informational from None
2015-09-23
01 Al Morton New version available: draft-ietf-bmwg-virtual-net-01.txt
2015-06-18
00 Al Morton This document now replaces draft-morton-bmwg-virtual-net instead of None
2015-06-02
00 Al Morton New version available: draft-ietf-bmwg-virtual-net-00.txt