Considerations for Benchmarking Virtual Network Functions and Their Infrastructure
RFC 8172

• Status
• IESG evaluation record
• IESG writeups
• Email expansions
• History

• Versions
• 00
• 01
• 02
• 03
• 04
• 05

Internet Engineering Task Force (IETF)                         A. Morton
Request for Comments: 8172                                     AT&T Labs
Category: Informational                                        July 2017
ISSN: 2070-1721

       Considerations for Benchmarking Virtual Network Functions
                        and Their Infrastructure

Abstract

   The Benchmarking Methodology Working Group has traditionally
   conducted laboratory characterization of dedicated physical
   implementations of internetworking functions.  This memo investigates
   additional considerations when network functions are virtualized and
   performed in general-purpose hardware.

Status of This Memo

   This document is not an Internet Standards Track specification; it is
   published for informational purposes.

   This document is a product of the Internet Engineering Task Force
   (IETF).  It represents the consensus of the IETF community.  It has
   received public review and has been approved for publication by the
   Internet Engineering Steering Group (IESG).  Not all documents
   approved by the IESG are a candidate for any level of Internet
   Standard; see Section 2 of RFC 7841.

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at
   http://www.rfc-editor.org/info/rfc8172.

Copyright Notice

   Copyright (c) 2017 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Morton                        Informational                     [Page 1]
RFC 8172      Benchmarking VNFs and Related Infrastructure     July 2017

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
   2.  Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . .   4
   3.  Considerations for Hardware and Testing . . . . . . . . . . .   4
     3.1.  Hardware Components . . . . . . . . . . . . . . . . . . .   4
     3.2.  Configuration Parameters  . . . . . . . . . . . . . . . .   5
     3.3.  Testing Strategies  . . . . . . . . . . . . . . . . . . .   6
     3.4.  Attention to Shared Resources . . . . . . . . . . . . . .   7
   4.  Benchmarking Considerations . . . . . . . . . . . . . . . . .   8
     4.1.  Comparison with Physical Network Functions  . . . . . . .   8
     4.2.  Continued Emphasis on Black-Box Benchmarks  . . . . . . .   8
     4.3.  New Benchmarks and Related Metrics  . . . . . . . . . . .   9
     4.4.  Assessment of Benchmark Coverage  . . . . . . . . . . . .  10
     4.5.  Power Consumption . . . . . . . . . . . . . . . . . . . .  12
   5.  Security Considerations . . . . . . . . . . . . . . . . . . .  12
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  13
   7.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  13
     7.1.  Normative References  . . . . . . . . . . . . . . . . . .  13
     7.2.  Informative References  . . . . . . . . . . . . . . . . .  14
   Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  15
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  15

1.  Introduction

   The Benchmarking Methodology Working Group (BMWG) has traditionally
   conducted laboratory characterization of dedicated physical
   implementations of internetworking functions (or physical network
   functions (PNFs)).  The black-box benchmarks of throughput, latency,
   forwarding rates, and others have served our industry for many years.
   [RFC1242] and [RFC2544] are the cornerstones of the work.

   A set of service provider and vendor development goals has emerged:
   reduce costs while increasing flexibility of network devices and
   drastically reduce deployment time.  Network Function Virtualization
   (NFV) has the promise to achieve these goals and therefore has
   garnered much attention.  It now seems certain that some network
   functions will be virtualized following the success of cloud
   computing and virtual desktops supported by sufficient network path
   capacity, performance, and widespread deployment; many of the same
   techniques will help achieve NFV.

   In the context of Virtual Network Functions (VNFs), the supporting
   Infrastructure requires general-purpose computing systems, storage
   systems, networking systems, virtualization support systems (such as
   hypervisors), and management systems for the virtual and physical