Benchmarking Methodology Working Group                            K. Sun
Internet-Draft                                                   H. Yang
Intended status: Informational                                   Y. Park
Expires: September 8, 2019                                        Y. Kim
                                                     Soongsil University
                                                                  W. Lee
                                                                    ETRI
                                                           March 7, 2019


  Considerations for Benchmarking Network Performance in Containerized
                            Infrastructures
                 draft-dcn-bmwg-containerized-infra-00

Abstract

   This draft describes benchmarking considerations for a containerized
   infrastructure.  In a containerized infrastructure, Virtualized
   Network Functions(VNFs) are deployed on operating-system-level
   virtualization platform by abstracting the user namespace as opposed
   to virtualization using a hypervisor.  Leveraging this, the system
   configurations and networking scenarios for VNF benchmarking will be
   partially changed by way of resource allocation and network port
   binding between a physical host and VNFs.  In this draft we compare
   the state of the art in container networking architecture with
   networking on VM-based virtualized systems, and provide several test
   scenarios for network performance in containerized infrastructure.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 8, 2019.







Sun, et al.             Expires September 8, 2019               [Page 1]


Internet-Draft      Benchmarking Containerized Infra          March 2019


Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Benchmarking Consideration  . . . . . . . . . . . . . . . . .   3
     3.1.  Comparison with VM based Infrastructure . . . . . . . . .   3
     3.2.  Additional Considerations for Container Networking  . . .   5
   4.  Test Scenarios  . . . . . . . . . . . . . . . . . . . . . . .   7
   5.  Security Considerations . . . . . . . . . . . . . . . . . . .   7
   6.  Informative References  . . . . . . . . . . . . . . . . . . .   7
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .   7

1.  Introduction

   The Benchmarking Methodology Working Group(BMWG) has recently
   expanded its benchmarking scope from Physical Network Function(PNF)
   running on dedicated hardware system to Network Function
   Virtualization(NFV) infrastructure and Virtualized Network
   Function(VNF).  [RFC8172] described considerations for configuring
   NFV infrastructure and benchmarking metrics, and [RFC8204] gives
   guidelines for benchmarking virtual switch which connects VNFs in
   Open Platform for NFV(OPNFV).

   Recently NFV infrastructure has evolved to include a lightweight
   virtualized platform called the containerized infrastructure, where
   VNFs share the same host Operating System(OS) and they are logically
   isolated by using a different namespace.  While previous NFV
   infrastructure uses a hypervisor to allocate resources for Virtual
   Machine(VMs) and instantiate VNFs on it, the containerized
   infrastructure virtualizes resources without a hypervisor, therefore
   making containers very lightweight and more efficient in
   infrastructure resource utilization compared to a VM based NFV
   infrastructure.  When we consider benchmarking for VNFs in the



Sun, et al.             Expires September 8, 2019               [Page 2]


Internet-Draft      Benchmarking Containerized Infra          March 2019


   containerized infrastructure, it may have a different Device Under
   Test(DUT) configuration compared with both black-box benchmarking and
   VM-based NFV infrastructure as described in [RFC8172].  Accordingly,
   additional configuration parameters and testing strategies may be
   required.

   In the containerized infrastructure, a VNF network is implemented by
   running both switch and router functions in the host system.  For
   example, the internal communication between VNFs in the same host
   uses the L2 bridge function, while communication with external
   node(s) uses the L3 router function.  For container networking, the
   host system may use a virtual switch(vSwitch), but other options
   exist.  In the [ETSI-TST-009], they describe differences in
   networking structure between VM-based and container-based
   infrastructure.  Occasioned by these differences, deployment
   scenarios for testing network performance described in [RFC8204] may
   be partially applied to the containerized infrastructure, but other
   scenarios may be required.

   In this draft, we describe differences and additional considerations
   for benchmarking containerized infrastructure based on [RFC8172] and
   [RFC8204].  In particular, we focus on differences in system
   configuration parameters and networking configurations of the
   containerized infrastructure compared with VM-based NFV
   infrastructure.  Note that, although the detailed configurations of
   both infrastructures differ, the new benchmarks and metrics defined
   in [RFC8172] can be equally applied in containerized infrastructure
   from a generic-NFV point of view, and therefore defining additional
   metrics or methodologies is out of scope.

2.  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document is to be interpreted as described in [RFC2119].  This
   document uses the terminology described in [RFC8172], [RFC8204],
   [ETSI-TST-009].

3.  Benchmarking Consideration

3.1.  Comparison with VM based Infrastructure

   For benchmarking of containerized infrastructure, as mentioned in
   [RFC8172], the basic approach is to reuse existing benchmarks
   developed within the BMWG.  Various network function specifications
   already defined in BMWG should still be applied to containerized VNFs
   for performance comparison with physical network functions and VM
   based VNFs.



Sun, et al.             Expires September 8, 2019               [Page 3]


Internet-Draft      Benchmarking Containerized Infra          March 2019


 +---------------------------------+  +--------------------------------+
 |+--------------+ +--------------+|  |+------------+    +------------+|
 ||   Guest VM   | |   Guest VM   ||  || Container  |    | Container  ||
 ||+------------+| |+------------+||  ||+----------+|    |+----------+||
 |||     APP    || ||     APP    |||  |||   APP    ||    ||   APP    |||
 ||+------------+| |+------------+||  ||+----------+|    |+----------+||
 ||+------------+| |+------------+||  ||+----------+|    |+----------+||
 |||Guest Kernel|| ||Guest Kernel|||  ||| Bin/Libs ||    || Bin/Libs |||
 ||+------------+| |+------------+||  ||+----------+|    |+----------+||
 |+------^-------+ +-------^------+|  |+-----^------+    +------^-----+|
 |+------|-----------------|------+|  |+-----|------------------|-----+|
 ||      |    Hypervisor   |      ||  ||     |+----------------+|     ||
 |+------|-----------------|------+|  ||     ||Container Engine||     ||
 |+------|-----------------|------+|  ||     |+----------------+|     ||
 ||      | Host OS Kernel  |      ||  ||     |  Host OS Kernel  |     ||
 |+------|-----------------|-----+||  |+-----|------------------|-----+|
 |    +--v-----------------v--+    |  |  +---v------------------v---+  |
 +----|    physical network   |----+  +--|    physical network      |--+
      +-----------------------+            +--------------------------+
     (a) VM-Based Infrastructure       (b) Containerized Infrastructure

                Figure 1: Comparison of NFV Infrastructures

   In Figure 1, we describe two different NFV architectures: VM-based
   and Containerized.  A major distinction between containerized
   infrastructure and VM based infrastructure is that with the former,
   all VNFs share the same host resources including but not limited to
   computing, storage and networking resources, as well as the host
   Operating System(OS), kernel and libraries.  The absence of the guest
   OS and the hypervisor, necessitates the following considerations that
   occur in the test environment:

   o Concerning hardware for containerized infrastructure, all
   components described in [RFC8172] can be part of the test setup.
   While the capabilities of servers and storage should meet the minimum
   requirements for testing, it is possible to deploy a test environment
   with less capabilities than in a VM based infrastructure.

   o About configuration parameters, containerized infrastructure needs
   specified management system instead of hypervisor(e.g.  Linux
   Container, Docker Engine).

   o In the VM based infrastructure, each VM has packet processing in
   the kernel of the guest OS through its own CPU threads, virtualized
   and assigned by hypervisor.  On the other hand, containerized VNFs
   use the host CPU without virtualization.  Different CPU resource
   assignment methods may have different CPU utilization perspectives
   for VNF performance benchmarking.



Sun, et al.             Expires September 8, 2019               [Page 4]


Internet-Draft      Benchmarking Containerized Infra          March 2019


   o From a Memory Management Unit(MMU) point of view, there is a
   difference in how the paging process is conducted between two
   environments.  The main difference lies in the isolated nature of the
   OS for VM-based VNFs.  In the containerized infrastructure, memory
   paging which processes conversion between physical address and
   virtual address is affected by the host resource directly.  Thus,
   memory usage of each VNFs is more dependent on the host resource
   capabilities than in VM-based VNFs.

   o Some network drivers may have varying dependencies for each
   environment.  For example, a vhost-net driver used in a guest OS
   cannot be used for a container; on the other hand, a veth driver can
   be only applicable within a containerized infrastructure.

3.2.  Additional Considerations for Container Networking

   In the containerized infrastructure, there are various network
   architectures depending on the deployment environment and models.
   Since container networking typically involves using virtual switch
   functions, base network configuration parameters for container
   networking benchmarks are mostly similar with VM based VNF networking
   described in [RFC8204].  Additional considerations for container
   networking are described as follows:

   o Networking depends on deployment models: Containerized VNFs have
   several deployment models.  Containerized VNFs can be deployed as a
   cluster called POD by Kubernetes, otherwise each VNF can be deployed
   separately using Docker.  In former case, there is only one external
   network interface for a POD which contains more than one VNF.  An
   alternative deployment model considers a scenario in which
   containerized VNFs or PODs are running on VM-based infrastructure.
   Figure 2 shows briefly differences of network architectures based on
   deployment models.  [ETSI-TST-009] describes in more detail the
   differences between them.  Other deployment models are classified
   bases on whether containerized VNFs are deployed on baremetal or
   inside of the VM.















Sun, et al.             Expires September 8, 2019               [Page 5]


Internet-Draft      Benchmarking Containerized Infra          March 2019


 +---------------------------------------------------------------------+
 |                          Baremetal Node                             |
 |                                                                     |
 | +--------------+ +--------------+ +-------------- + +-------------+ |
 | |              | |     POD      | |      VM       | |     VM      | |
 | |              | |+------------+| |+-------------+| |  +------+   | |
 | |   Container  | ||  Container || ||Container VNF|| |  | PODs |   | |
 | |     VNF      | ||    VNFs    || |+-----^-------+| |  +---^--+   | |
 | |              | |+------------+| |      |        | |      |      | |
 | |   +------+   | |   +------+   | |   +--v---+    | |  +---v--+   | |
 | +---| veth |---+ +---| veth |---+ +---|virtio|----+ +--|virtio|---+ |
 |     +--^---+         +---^--+         +--^---+         +---^--+     |
 |        |                 |               |                 |        |
 |        |                 |            +--v---+         +---v--+     |
 | +------|-----------------|------------|vhost |---------|vhost |---+ |
 | |      |                 |            +--^---+         +---^--+   | |
 | |      |                 |               |                 |      | |
 | |   +--v---+         +---v--+         +--v---+         +---v--+   | |
 | | +-| veth |---------| veth |---------| Tap  |---------| Tap  |-+ | |
 | | | +--^---+         +---^--+         +--^---+         +---^--+ | | |
 | | |    |                 |    vSwitch    |                 |    | | |
 | | | +--|-----------------|---------------|-----------------|--+ | | |
 | | +-|  |                 |    Bridge     |                 |  |-+ | |
 | |   +--|-----------------|---------------|-----------------|--+   | |
 | |      |   +---------+   |            +--|-----------------|---+  | |
 | |      |   |Container|   |            |  |    Hypervisor   |   |  | |
 | |      |   | Engine  |   |            |  |                 |   |  | |
 | |      |   +---------+   |            +--|-----------------|---+  | |
 | |      |                 |  Host Kernel  |                 |      | |
 | +------|-----------------|---------------|-----------------|------+ |
 |     +--v-----------------v---------------v-----------------v--+     |
 +-----|                      physical network                   |-----+
       +---------------------------------------------------------+

     Figure 2: Examples of Networking Architecture based on Deployment
                                  Models

   o Network Plug-ins: In the containerized infrastructure, specific
   networking functions can be supported by attaching various plug-ins.
   Container Network Model(CNM) and Container Network Interface(CNI) are
   currently the most popular network plug-ins.  According each network
   plug-in, they have different runtime structure or accessibilities to
   namespace.  Actual testing results may vary depending on plug-in
   types and its supporting drivers.

   o Network Types: To enhance forwarding capabilities, similar to the
   VM based infrastructure, the containerized infrastructure can also
   employ use of specific networking technologies such as SR-IOV.



Sun, et al.             Expires September 8, 2019               [Page 6]


Internet-Draft      Benchmarking Containerized Infra          March 2019


4.  Test Scenarios

   TBD

5.  Security Considerations

   TBD

6.  Informative References

   [ETSI-TST-009]
              "Network Functions Virtualisation (NFV) Release 3;
              Testing; Specification of Networking Benchmarks and
              Measurement Methods for NFVI", October 2018.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", RFC 2119, March 1997.

   [RFC8172]  Morton, A., "Considerations for Benchmarking Virtual
              Network Functions and Their Infrastructure", RFC 8172,
              July 2017.

   [RFC8204]  Tahhan, M., O'Mahony, B., and A. Morton, "Benchmarking
              Virtual Switches in the Open Platform for NFV (OPNFV)",
              RFC 8204, September 2017.

Authors' Addresses

   Kyoungjae Sun
   School of Electronic Engineering
   Soongsil University
   369, Sangdo-ro, Dongjak-gu
   Seoul, Seoul  06978
   Republic of Korea

   Phone: +82 10 3643 5627
   EMail: gomjae@dcn.ssu.ac.kr


   Hyunsik Yang
   School of Electronic Engineering
   Soongsil University
   369, Sangdo-ro, Dongjak-gu
   Seoul, Seoul  06978
   Republic of Korea

   Phone: +82 10 9005 7439
   EMail: yangun@dcn.ssu.ac.kr



Sun, et al.             Expires September 8, 2019               [Page 7]


Internet-Draft      Benchmarking Containerized Infra          March 2019


   Youngki Park
   School of Electronic Engineering
   Soongsil University
   369, Sangdo-ro, Dongjak-gu
   Seoul, Seoul  06978
   Republic of Korea

   Phone: +82 10 4281 0720
   EMail: ykpark@dcn.ssu.ac.kr


   Younghan Kim
   School of Electronic Engineering
   Soongsil University
   369, Sangdo-ro, Dongjak-gu
   Seoul, Seoul  06978
   Republic of Korea

   Phone: +82 10 2691 0904
   EMail: younghak@ssu.ac.kr


   Wangbong Lee
   ETRI
   ETRI
   161, Gajeong-ro, Yoosung-gu
   Dajeon, Dajeon  34129
   Republic of Korea

   Phone: +82 10 5336 2323
   EMail: leewb@etri.re.kr




















Sun, et al.             Expires September 8, 2019               [Page 8]