Benchmarking Methodology Working Group K. Sun
Internet-Draft H. Yang
Intended status: Informational Y. Park
Expires: September 8, 2019 Y. Kim
Soongsil University
W. Lee
ETRI
March 7, 2019
Considerations for Benchmarking Network Performance in Containerized
Infrastructures
draft-dcn-bmwg-containerized-infra-00
Abstract
This draft describes benchmarking considerations for a containerized
infrastructure. In a containerized infrastructure, Virtualized
Network Functions(VNFs) are deployed on operating-system-level
virtualization platform by abstracting the user namespace as opposed
to virtualization using a hypervisor. Leveraging this, the system
configurations and networking scenarios for VNF benchmarking will be
partially changed by way of resource allocation and network port
binding between a physical host and VNFs. In this draft we compare
the state of the art in container networking architecture with
networking on VM-based virtualized systems, and provide several test
scenarios for network performance in containerized infrastructure.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 8, 2019.
Sun, et al. Expires September 8, 2019 [Page 1]
Internet-Draft Benchmarking Containerized Infra March 2019
Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Benchmarking Consideration . . . . . . . . . . . . . . . . . 3
3.1. Comparison with VM based Infrastructure . . . . . . . . . 3
3.2. Additional Considerations for Container Networking . . . 5
4. Test Scenarios . . . . . . . . . . . . . . . . . . . . . . . 7
5. Security Considerations . . . . . . . . . . . . . . . . . . . 7
6. Informative References . . . . . . . . . . . . . . . . . . . 7
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 7
1. Introduction
The Benchmarking Methodology Working Group(BMWG) has recently
expanded its benchmarking scope from Physical Network Function(PNF)
running on dedicated hardware system to Network Function
Virtualization(NFV) infrastructure and Virtualized Network
Function(VNF). [RFC8172] described considerations for configuring
NFV infrastructure and benchmarking metrics, and [RFC8204] gives
guidelines for benchmarking virtual switch which connects VNFs in
Open Platform for NFV(OPNFV).
Recently NFV infrastructure has evolved to include a lightweight
virtualized platform called the containerized infrastructure, where
VNFs share the same host Operating System(OS) and they are logically
isolated by using a different namespace. While previous NFV
infrastructure uses a hypervisor to allocate resources for Virtual
Machine(VMs) and instantiate VNFs on it, the containerized
infrastructure virtualizes resources without a hypervisor, therefore
making containers very lightweight and more efficient in
infrastructure resource utilization compared to a VM based NFV
infrastructure. When we consider benchmarking for VNFs in the
Sun, et al. Expires September 8, 2019 [Page 2]
Internet-Draft Benchmarking Containerized Infra March 2019
containerized infrastructure, it may have a different Device Under
Test(DUT) configuration compared with both black-box benchmarking and
VM-based NFV infrastructure as described in [RFC8172]. Accordingly,
additional configuration parameters and testing strategies may be
required.
In the containerized infrastructure, a VNF network is implemented by
running both switch and router functions in the host system. For
example, the internal communication between VNFs in the same host
uses the L2 bridge function, while communication with external
node(s) uses the L3 router function. For container networking, the
host system may use a virtual switch(vSwitch), but other options
exist. In the [ETSI-TST-009], they describe differences in
networking structure between VM-based and container-based
infrastructure. Occasioned by these differences, deployment
scenarios for testing network performance described in [RFC8204] may
be partially applied to the containerized infrastructure, but other
scenarios may be required.
In this draft, we describe differences and additional considerations
for benchmarking containerized infrastructure based on [RFC8172] and
[RFC8204]. In particular, we focus on differences in system
configuration parameters and networking configurations of the
containerized infrastructure compared with VM-based NFV
infrastructure. Note that, although the detailed configurations of
both infrastructures differ, the new benchmarks and metrics defined
in [RFC8172] can be equally applied in containerized infrastructure
from a generic-NFV point of view, and therefore defining additional
metrics or methodologies is out of scope.
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document is to be interpreted as described in [RFC2119]. This
document uses the terminology described in [RFC8172], [RFC8204],
[ETSI-TST-009].
3. Benchmarking Consideration
3.1. Comparison with VM based Infrastructure
For benchmarking of containerized infrastructure, as mentioned in
[RFC8172], the basic approach is to reuse existing benchmarks
developed within the BMWG. Various network function specifications
already defined in BMWG should still be applied to containerized VNFs
for performance comparison with physical network functions and VM
based VNFs.
Sun, et al. Expires September 8, 2019 [Page 3]
Internet-Draft Benchmarking Containerized Infra March 2019
+---------------------------------+ +--------------------------------+
|+--------------+ +--------------+| |+------------+ +------------+|
|| Guest VM | | Guest VM || || Container | | Container ||
||+------------+| |+------------+|| ||+----------+| |+----------+||
||| APP || || APP ||| ||| APP || || APP |||
||+------------+| |+------------+|| ||+----------+| |+----------+||
||+------------+| |+------------+|| ||+----------+| |+----------+||
|||Guest Kernel|| ||Guest Kernel||| ||| Bin/Libs || || Bin/Libs |||
||+------------+| |+------------+|| ||+----------+| |+----------+||
|+------^-------+ +-------^------+| |+-----^------+ +------^-----+|
|+------|-----------------|------+| |+-----|------------------|-----+|
|| | Hypervisor | || || |+----------------+| ||
|+------|-----------------|------+| || ||Container Engine|| ||
|+------|-----------------|------+| || |+----------------+| ||
|| | Host OS Kernel | || || | Host OS Kernel | ||
|+------|-----------------|-----+|| |+-----|------------------|-----+|
| +--v-----------------v--+ | | +---v------------------v---+ |
+----| physical network |----+ +--| physical network |--+
+-----------------------+ +--------------------------+
(a) VM-Based Infrastructure (b) Containerized Infrastructure
Figure 1: Comparison of NFV Infrastructures
In Figure 1, we describe two different NFV architectures: VM-based
and Containerized. A major distinction between containerized
infrastructure and VM based infrastructure is that with the former,
all VNFs share the same host resources including but not limited to
computing, storage and networking resources, as well as the host
Operating System(OS), kernel and libraries. The absence of the guest
OS and the hypervisor, necessitates the following considerations that
occur in the test environment:
o Concerning hardware for containerized infrastructure, all
components described in [RFC8172] can be part of the test setup.
While the capabilities of servers and storage should meet the minimum
requirements for testing, it is possible to deploy a test environment
with less capabilities than in a VM based infrastructure.
o About configuration parameters, containerized infrastructure needs
specified management system instead of hypervisor(e.g. Linux
Container, Docker Engine).
o In the VM based infrastructure, each VM has packet processing in
the kernel of the guest OS through its own CPU threads, virtualized
and assigned by hypervisor. On the other hand, containerized VNFs
use the host CPU without virtualization. Different CPU resource
assignment methods may have different CPU utilization perspectives
for VNF performance benchmarking.
Sun, et al. Expires September 8, 2019 [Page 4]
Internet-Draft Benchmarking Containerized Infra March 2019
o From a Memory Management Unit(MMU) point of view, there is a
difference in how the paging process is conducted between two
environments. The main difference lies in the isolated nature of the
OS for VM-based VNFs. In the containerized infrastructure, memory
paging which processes conversion between physical address and
virtual address is affected by the host resource directly. Thus,
memory usage of each VNFs is more dependent on the host resource
capabilities than in VM-based VNFs.
o Some network drivers may have varying dependencies for each
environment. For example, a vhost-net driver used in a guest OS
cannot be used for a container; on the other hand, a veth driver can
be only applicable within a containerized infrastructure.
3.2. Additional Considerations for Container Networking
In the containerized infrastructure, there are various network
architectures depending on the deployment environment and models.
Since container networking typically involves using virtual switch
functions, base network configuration parameters for container
networking benchmarks are mostly similar with VM based VNF networking
described in [RFC8204]. Additional considerations for container
networking are described as follows:
o Networking depends on deployment models: Containerized VNFs have
several deployment models. Containerized VNFs can be deployed as a
cluster called POD by Kubernetes, otherwise each VNF can be deployed
separately using Docker. In former case, there is only one external
network interface for a POD which contains more than one VNF. An
alternative deployment model considers a scenario in which
containerized VNFs or PODs are running on VM-based infrastructure.
Figure 2 shows briefly differences of network architectures based on
deployment models. [ETSI-TST-009] describes in more detail the
differences between them. Other deployment models are classified
bases on whether containerized VNFs are deployed on baremetal or
inside of the VM.
Sun, et al. Expires September 8, 2019 [Page 5]
Internet-Draft Benchmarking Containerized Infra March 2019
+---------------------------------------------------------------------+
| Baremetal Node |
| |
| +--------------+ +--------------+ +-------------- + +-------------+ |
| | | | POD | | VM | | VM | |
| | | |+------------+| |+-------------+| | +------+ | |
| | Container | || Container || ||Container VNF|| | | PODs | | |
| | VNF | || VNFs || |+-----^-------+| | +---^--+ | |
| | | |+------------+| | | | | | | |
| | +------+ | | +------+ | | +--v---+ | | +---v--+ | |
| +---| veth |---+ +---| veth |---+ +---|virtio|----+ +--|virtio|---+ |
| +--^---+ +---^--+ +--^---+ +---^--+ |
| | | | | |
| | | +--v---+ +---v--+ |
| +------|-----------------|------------|vhost |---------|vhost |---+ |
| | | | +--^---+ +---^--+ | |
| | | | | | | |
| | +--v---+ +---v--+ +--v---+ +---v--+ | |
| | +-| veth |---------| veth |---------| Tap |---------| Tap |-+ | |
| | | +--^---+ +---^--+ +--^---+ +---^--+ | | |
| | | | | vSwitch | | | | |
| | | +--|-----------------|---------------|-----------------|--+ | | |
| | +-| | | Bridge | | |-+ | |
| | +--|-----------------|---------------|-----------------|--+ | |
| | | +---------+ | +--|-----------------|---+ | |
| | | |Container| | | | Hypervisor | | | |
| | | | Engine | | | | | | | |
| | | +---------+ | +--|-----------------|---+ | |
| | | | Host Kernel | | | |
| +------|-----------------|---------------|-----------------|------+ |
| +--v-----------------v---------------v-----------------v--+ |
+-----| physical network |-----+
+---------------------------------------------------------+
Figure 2: Examples of Networking Architecture based on Deployment
Models
o Network Plug-ins: In the containerized infrastructure, specific
networking functions can be supported by attaching various plug-ins.
Container Network Model(CNM) and Container Network Interface(CNI) are
currently the most popular network plug-ins. According each network
plug-in, they have different runtime structure or accessibilities to
namespace. Actual testing results may vary depending on plug-in
types and its supporting drivers.
o Network Types: To enhance forwarding capabilities, similar to the
VM based infrastructure, the containerized infrastructure can also
employ use of specific networking technologies such as SR-IOV.
Sun, et al. Expires September 8, 2019 [Page 6]
Internet-Draft Benchmarking Containerized Infra March 2019
4. Test Scenarios
TBD
5. Security Considerations
TBD
6. Informative References
[ETSI-TST-009]
"Network Functions Virtualisation (NFV) Release 3;
Testing; Specification of Networking Benchmarks and
Measurement Methods for NFVI", October 2018.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", RFC 2119, March 1997.
[RFC8172] Morton, A., "Considerations for Benchmarking Virtual
Network Functions and Their Infrastructure", RFC 8172,
July 2017.
[RFC8204] Tahhan, M., O'Mahony, B., and A. Morton, "Benchmarking
Virtual Switches in the Open Platform for NFV (OPNFV)",
RFC 8204, September 2017.
Authors' Addresses
Kyoungjae Sun
School of Electronic Engineering
Soongsil University
369, Sangdo-ro, Dongjak-gu
Seoul, Seoul 06978
Republic of Korea
Phone: +82 10 3643 5627
EMail: gomjae@dcn.ssu.ac.kr
Hyunsik Yang
School of Electronic Engineering
Soongsil University
369, Sangdo-ro, Dongjak-gu
Seoul, Seoul 06978
Republic of Korea
Phone: +82 10 9005 7439
EMail: yangun@dcn.ssu.ac.kr
Sun, et al. Expires September 8, 2019 [Page 7]
Internet-Draft Benchmarking Containerized Infra March 2019
Youngki Park
School of Electronic Engineering
Soongsil University
369, Sangdo-ro, Dongjak-gu
Seoul, Seoul 06978
Republic of Korea
Phone: +82 10 4281 0720
EMail: ykpark@dcn.ssu.ac.kr
Younghan Kim
School of Electronic Engineering
Soongsil University
369, Sangdo-ro, Dongjak-gu
Seoul, Seoul 06978
Republic of Korea
Phone: +82 10 2691 0904
EMail: younghak@ssu.ac.kr
Wangbong Lee
ETRI
ETRI
161, Gajeong-ro, Yoosung-gu
Dajeon, Dajeon 34129
Republic of Korea
Phone: +82 10 5336 2323
EMail: leewb@etri.re.kr
Sun, et al. Expires September 8, 2019 [Page 8]