Internet Engineering Task Force M. Hamilton
Internet-Draft BreakingPoint Systems
Intended status: Informational S. Banks
Expires: September 8, 2011 Cisco Systems
March 7, 2011
Benchmarking Terminology for Content-Aware Network Devices
draft-hamilton-bmwg-ca-bench-term-00
Abstract
The purpose of this document is to define and outline the terminology
necessary to appropriately follow and implement "Benchmarking
Methodology for Content-Aware Network Devices". Relevant terms will
be defined and discussed throughout this document in order to ensure
the comprehension of the previously mentioned methodology.
Status of this Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 8, 2011.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
Hamilton & Banks Expires September 8, 2011 [Page 1]
Internet-Draft Terminology for Content-Aware Devices March 2011
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Requirements Language . . . . . . . . . . . . . . . . . . 4
2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.1. Application Flow . . . . . . . . . . . . . . . . . . . . . 5
3.2. Application Throughput . . . . . . . . . . . . . . . . . . 5
3.3. Average Time to TCP Session Establishment . . . . . . . . 6
3.4. Content-Aware Device . . . . . . . . . . . . . . . . . . . 6
3.5. Deep Packet Inspection . . . . . . . . . . . . . . . . . . 7
3.6. Network 5-Tuple . . . . . . . . . . . . . . . . . . . . . 7
3.7. Session Establishment Rate . . . . . . . . . . . . . . . . 8
3.8. Session Establishment Time . . . . . . . . . . . . . . . . 8
3.9. Simultaneous TCP Sessions . . . . . . . . . . . . . . . . 9
3.10. Time To SYN . . . . . . . . . . . . . . . . . . . . . . . 9
4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10
5. Security Considerations . . . . . . . . . . . . . . . . . . . 10
6. References . . . . . . . . . . . . . . . . . . . . . . . . . . 10
6.1. Normative References . . . . . . . . . . . . . . . . . . . 10
6.2. Informative References . . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 11
Hamilton & Banks Expires September 8, 2011 [Page 2]
Internet-Draft Terminology for Content-Aware Devices March 2011
1. Introduction
Content-aware and deep packet inspection (DPI) device penetration has
grown significantly over the last decade. No longer are devices
simply using Ethernet headers and IP headers to make forwarding
decisions. Devices that could historically be classified as
'stateless' or raw forwarding devices are now seeing more DPI
functionality. Devices such as core and edge routers are now being
developed with DPI functionality to make more intelligent routing and
forwarding decisions.
The Benchmarking Working Group (BMWG) has historically produced
Internet Drafts and Requests for Comment that are focused
specifically on creating output metrics that are derived from a very
specific and well-defined set of input parameters that are completely
and unequivocally reproducible from testbed to testbed. The end goal
of such methodologies is to, in the words of the BMWG charter "reduce
specmanship" from network equipment manufacturers(NEM's). Existing
BMWG work has certainly met this stated goal.
Today, device sophistication has expanded beyond existing
methodologies, allowing vendors to reengage in specmanship. In order
to achieve the stated BMWG goals, the methodologies designed to hold
vendors accountable must evolve with the enhanced device
functionality.
The BMWG has historically avoided the use of the term "realistic"
throughout all of its drafts and RFCs. While this document will not
explicitly use this term, the end goal of the terminology and
methodology is to generate performance metrics that will be as close
as possible to equivalent metrics in a production environment. It
should be further noted than any metrics acquired from a production
network MUST be captured according to the policies and procedures of
the IPPM or PMOL working groups.
An explicit non-goal of this document is to replace existing
methodology/terminology pairs such as RFC 2544 [1]/RFC 1242 [2] or
RFC 3511 [3]/RFC 2647 [4]. The explicit goal of this document is to
create a methodology and terminology pair that is more suited for
modern devices while complementing the data acquired using existing
BMWG methodologies. Existing BMWG work generally revolves around
completely repeatable input stimulus, expecting fully repeatable
output. This document departs from this mantra due to the nature of
modern traffic and is more focused on output repeatability than on
static input stimulus.
Some of the terms used throughout this draft have previously been
defined in "Benchmarking Terminology for Firewall Performance" RFC
Hamilton & Banks Expires September 8, 2011 [Page 3]
Internet-Draft Terminology for Content-Aware Devices March 2011
2647 [4]. This document SHOULD be consulted prior to using this
document.
1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [5].
2. Scope
Content-aware devices take many forms, shapes and architectures.
These devices are advanced network interconnect devices that inspect
deep into the application payload of network data packets to do
classification. They may be as simple as a firewall that uses
application data inspection for rule set enforcement, or they may
have advanced functionality such as performing protocol decoding and
validation, anti-virus, anti-spam and even application exploit
filtering.
This document is strictly focused on examining performance and
robustness across a focused set of metrics that may be used to more
accurately predict device performance when deployed in modern
networks. These metrics will be implementation independent.
It should also be noted that the purpose of this document is not to
perform functional testing of the potential features in the Device/
System Under Test (DUT/SUT)[4] nor specify the configurations that
should be tested. Various definitions of proper operation and
configuration may be appropriate within different contexts. While
the definition of these parameters are outside the scope of this
document, the specific configuration of both the DUT and tester
SHOULD be published with the test results for repeatability and
comparison purposes.
While a list of devices that fall under this category will quickly
become obsolete, an initial list of devices that would be well served
by utilizing this type of methodology should prove useful. Devices
such as firewalls, intrusion detection and prevention devices,
application delivery controllers, deep packet inspection devices, and
unified threat management systems generally fall into the content-
aware category.
3. Definitions
Hamilton & Banks Expires September 8, 2011 [Page 4]
Internet-Draft Terminology for Content-Aware Devices March 2011
3.1. Application Flow
Definition:
An application flow is the virtual connection between two network
hosts that is used to exchange user data above the transport
layer.
Discussion:
Content-aware devices may potentially proxy session-layer
connections, acting as a virtual server to the client and a
virtual client to the server. In this mode, the SUT/DUT may
modify members of the network 5-tuple or act on their behalf, thus
each end host is actually disconnected at the session layer.
Application flows are virtual connections that are between the two
hosts, irrespective of the nature of the session layer semantics.
Unit of Measurement:
N/A
Issues:
N/A
See Also:
5-Tuple
3.2. Application Throughput
Definition:
The rate at which data associated with an application flow is
transmitted through the SUT/DUT.
Discussion:
Throughput metrics may be calculated at various layers in the
network protocol stack. Each layer does contain associated
overhead necessary to maintain that layer. Application throughput
is the number of bits transmitted through a SUT/DUT, not including
the overhead associated with lower layer protocols. Measurement
should be taken at the receiver side to minimize the impact of
session layer retransmissions.
Unit of Measurement:
N/A
Issues:
Some applications may not rely on session layer reliability
mechanisms. This definition does not cover the case where an
application may utilize its own specific reliability/
retransmission algorithm.
Hamilton & Banks Expires September 8, 2011 [Page 5]
Internet-Draft Terminology for Content-Aware Devices March 2011
See Also:
N/A
3.3. Average Time to TCP Session Establishment
Definition:
The average time that a SUT/DUT requires to complete the TCP
session establishment process.
Discussion:
The average time to TCP session establishment is calculated by
taking the sum of all "TCP Session Establishment Time" values
acquired in the specified time frame and divide by the total
number of sessions established within that timeframe. The
timeframe in which the average is taken will depend on the
methodology itself and what is trying to be measured.
Unit of Measurement:
Seconds.
Issues:
Depending on how the DUT/SUT handles TCP session establishment,
the client and server may have different values for the same TCP
session. A client-side session may be established prior to the
server-side session being established.
See Also:
See Also.
3.4. Content-Aware Device
Definition:
A networking device which performs deep packet inspection.
Discussion:
For a more detailed discussion, please see "deep packet
inspection".
Unit of Measurement:
Not Applicable.
Issues:
Not Applicable.
See Also:
Deep Packet Inspection
Hamilton & Banks Expires September 8, 2011 [Page 6]
Internet-Draft Terminology for Content-Aware Devices March 2011
3.5. Deep Packet Inspection
Definition:
The process by which a network device inspects layer 7 payload as
well as protocol headers when making processing decisions.
Discussion:
Deep packet inspection (DPI) has grown from a feature reserved for
Intrusion Prevention Devices into functionality that is shared
across many next generation networking devices. Devices
traditionally classified as firewalls are now looking at layer 7
payloads to make decisions, whether it is classification, rate-
shaping, or actually deeming whether a flow is allowed. Many
deep-packet inspection devices utilize proxy behavior as a
functional choice for performing inspection.
Unit of Measurement:
Not Applicable.
Issues:
Not Applicable.
See Also:
Content-Aware Device
3.6. Network 5-Tuple
Definition:
The set of 5 metrics which distinguish two session layer
connections from each other.
Discussion:
When discussing data transfer between hosts, a Network 5-tuple is
typically used to differentiate between multiple session layer
connections. Source and destination IP addresses, source and
destination session-layer ports, and the session layer protocol
make up the network 5-tuple. The session layer protocol is
typically TCP or UDP, but may be SCTP or another session layer
protocol.
Unit of Measurement:
N/A
Issues:
N/A
Hamilton & Banks Expires September 8, 2011 [Page 7]
Internet-Draft Terminology for Content-Aware Devices March 2011
3.7. Session Establishment Rate
Definition:
The rate at which TCP sessions may be established through a given
DUT/SUT.
Discussion:
The session establishment rate is a measurement of how many TCP
sessions the DUT/SUT is able to establish in a given unit of time.
If within a 1 second time interval the tester is able to establish
10,000 sessions, that rate will be measured at 10,000 sessions per
second. The session must be established in accordance with the
policy set forth in "Session Establishment Time".
Unit of Measurement:
TCP session(s) per second
Issues:
Issues.
See Also:
See Also.
3.8. Session Establishment Time
Definition:
Session establishment time is the difference in time between the
first TCP SYN packet sent from the client and when TCP ACK
packet's arrival at the server interface.
Discussion:
This metric is calculated between the time the first bit of the
TCP SYN packet is sent from the client and the time the last bit
of the TCP ACK packet arrives on the server interface.
Unit of Measurement:
Seconds.
Issues:
Depending on how the DUT/SUT handles TCP session establishment,
the client and server may have different values for the same
logical TCP session. A client-side session may be established
prior to the server-side session being established.
See Also:
Hamilton & Banks Expires September 8, 2011 [Page 8]
Internet-Draft Terminology for Content-Aware Devices March 2011
3.9. Simultaneous TCP Sessions
Definition:
The number of TCP sessions which are in the 'Established State' as
defined by RFC 793 [6].
Discussion:
This measurement counts the number of TCP sessions which are in
the 'Established State'. Sessions which are in this state must be
able to maintain data transfer between client and server, bi-
directionally.
Unit of Measurement:
Sessions.
Issues:
Depending on the nature of the SUT/DUT, the number of simultaneous
sessions may instantaneously be different when counted from the
client and server sides of the SUT/DUT.
See Also:
See Also.
3.10. Time To SYN
Definition:
The Time to SYN is a one-way metric, which is the difference
between the that that the first TCP SYN packet is sent by the
client and the time at which the server receives the TCP SYN
packet from the client.
Discussion:
This metric is more important with content-aware devices due to
the potential proxying issues. Content-aware devices may proxy a
TCP session on behalf of the server. Many times, the client will
receive the SYN/ACK from the DUT/SUT and complete the TCP
handshake before the SYN has been forwarded to the server. This
measurement is actually a proxy measure for client-side session
establishment time through the DUT/SUT, if the session is in fact
proxied.
Unit of Measurement:
Seconds.
See Also:
Hamilton & Banks Expires September 8, 2011 [Page 9]
Internet-Draft Terminology for Content-Aware Devices March 2011
4. IANA Considerations
This memo includes no request to IANA.
All drafts are required to have an IANA considerations section (see
the update of RFC 2434 [9] for a guide). If the draft does not
require IANA to do anything, the section contains an explicit
statement that this is the case (as above). If there are no
requirements for IANA, the section will be removed during conversion
into an RFC by the RFC Editor.
5. Security Considerations
Benchmarking activities as described in this memo are limited to
technology characterization using controlled stimuli in a laboratory
environment, with dedicated address space and the other constraints
RFC 2544 [1].
The benchmarking network topology will be an independent test setup
and MUST NOT be connected to devices that may forward the test
traffic into a production network, or misroute traffic to the test
management network
6. References
6.1. Normative References
[1] Bradner, S. and J. McQuaid, "Benchmarking Methodology for
Network Interconnect Devices", RFC 2544, March 1999.
[2] Bradner, S., "Benchmarking terminology for network
interconnection devices", RFC 1242, July 1991.
[3] Hickman, B., Newman, D., Tadjudin, S., and T. Martin,
"Benchmarking Methodology for Firewall Performance", RFC 3511,
April 2003.
[4] Newman, D., "Benchmarking Terminology for Firewall Performance",
RFC 2647, August 1999.
[5] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
[6] Postel, J., "Transmission Control Protocol", STD 7, RFC 793,
September 1981.
Hamilton & Banks Expires September 8, 2011 [Page 10]
Internet-Draft Terminology for Content-Aware Devices March 2011
[7] Popoviciu, C., Hamza, A., Van de Velde, G., and D. Dugatkin,
"IPv6 Benchmarking Methodology for Network Interconnect
Devices", RFC 5180, May 2008.
[8] Brownlee, N., Mills, C., and G. Ruth, "Traffic Flow Measurement:
Architecture", RFC 2722, October 1999.
6.2. Informative References
[9] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA
Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.
Authors' Addresses
Mike Hamilton
BreakingPoint Systems
Austin, TX 78717
US
Phone: +1 512 636 2303
Email: mhamilton@breakingpoint.com
Sarah Banks
Cisco Systems
San Jose, CA 95134
US
Email: sabanks@cisco.com
Hamilton & Banks Expires September 8, 2011 [Page 11]