\
[Search] [txt|pdfized|bibtex] [Tracker] [Email] [Nits]
Versions: 00                                                            
Internet Engineering Task Force                              M. Hamilton
Internet-Draft                                     BreakingPoint Systems
Intended status: Informational                                  S. Banks
Expires: September 8, 2011                                 Cisco Systems
                                                           March 7, 2011


       Benchmarking Terminology for Content-Aware Network Devices
                  draft-hamilton-bmwg-ca-bench-term-00

Abstract

   The purpose of this document is to define and outline the terminology
   necessary to appropriately follow and implement "Benchmarking
   Methodology for Content-Aware Network Devices".  Relevant terms will
   be defined and discussed throughout this document in order to ensure
   the comprehension of the previously mentioned methodology.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 8, 2011.

Copyright Notice

   Copyright (c) 2011 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as



Hamilton & Banks        Expires September 8, 2011               [Page 1]


Internet-Draft    Terminology for Content-Aware Devices       March 2011


   described in the Simplified BSD License.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
     1.1.  Requirements Language  . . . . . . . . . . . . . . . . . .  4
   2.  Scope  . . . . . . . . . . . . . . . . . . . . . . . . . . . .  4
   3.  Definitions  . . . . . . . . . . . . . . . . . . . . . . . . .  4
     3.1.  Application Flow . . . . . . . . . . . . . . . . . . . . .  5
     3.2.  Application Throughput . . . . . . . . . . . . . . . . . .  5
     3.3.  Average Time to TCP Session Establishment  . . . . . . . .  6
     3.4.  Content-Aware Device . . . . . . . . . . . . . . . . . . .  6
     3.5.  Deep Packet Inspection . . . . . . . . . . . . . . . . . .  7
     3.6.  Network 5-Tuple  . . . . . . . . . . . . . . . . . . . . .  7
     3.7.  Session Establishment Rate . . . . . . . . . . . . . . . .  8
     3.8.  Session Establishment Time . . . . . . . . . . . . . . . .  8
     3.9.  Simultaneous TCP Sessions  . . . . . . . . . . . . . . . .  9
     3.10. Time To SYN  . . . . . . . . . . . . . . . . . . . . . . .  9
   4.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 10
   5.  Security Considerations  . . . . . . . . . . . . . . . . . . . 10
   6.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 10
     6.1.  Normative References . . . . . . . . . . . . . . . . . . . 10
     6.2.  Informative References . . . . . . . . . . . . . . . . . . 11
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 11


























Hamilton & Banks        Expires September 8, 2011               [Page 2]


Internet-Draft    Terminology for Content-Aware Devices       March 2011


1.  Introduction

   Content-aware and deep packet inspection (DPI) device penetration has
   grown significantly over the last decade.  No longer are devices
   simply using Ethernet headers and IP headers to make forwarding
   decisions.  Devices that could historically be classified as
   'stateless' or raw forwarding devices are now seeing more DPI
   functionality.  Devices such as core and edge routers are now being
   developed with DPI functionality to make more intelligent routing and
   forwarding decisions.

   The Benchmarking Working Group (BMWG) has historically produced
   Internet Drafts and Requests for Comment that are focused
   specifically on creating output metrics that are derived from a very
   specific and well-defined set of input parameters that are completely
   and unequivocally reproducible from testbed to testbed.  The end goal
   of such methodologies is to, in the words of the BMWG charter "reduce
   specmanship" from network equipment manufacturers(NEM's).  Existing
   BMWG work has certainly met this stated goal.

   Today, device sophistication has expanded beyond existing
   methodologies, allowing vendors to reengage in specmanship.  In order
   to achieve the stated BMWG goals, the methodologies designed to hold
   vendors accountable must evolve with the enhanced device
   functionality.

   The BMWG has historically avoided the use of the term "realistic"
   throughout all of its drafts and RFCs.  While this document will not
   explicitly use this term, the end goal of the terminology and
   methodology is to generate performance metrics that will be as close
   as possible to equivalent metrics in a production environment.  It
   should be further noted than any metrics acquired from a production
   network MUST be captured according to the policies and procedures of
   the IPPM or PMOL working groups.

   An explicit non-goal of this document is to replace existing
   methodology/terminology pairs such as RFC 2544 [1]/RFC 1242 [2] or
   RFC 3511 [3]/RFC 2647 [4].  The explicit goal of this document is to
   create a methodology and terminology pair that is more suited for
   modern devices while complementing the data acquired using existing
   BMWG methodologies.  Existing BMWG work generally revolves around
   completely repeatable input stimulus, expecting fully repeatable
   output.  This document departs from this mantra due to the nature of
   modern traffic and is more focused on output repeatability than on
   static input stimulus.

   Some of the terms used throughout this draft have previously been
   defined in "Benchmarking Terminology for Firewall Performance" RFC



Hamilton & Banks        Expires September 8, 2011               [Page 3]


Internet-Draft    Terminology for Content-Aware Devices       March 2011


   2647 [4].  This document SHOULD be consulted prior to using this
   document.

1.1.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [5].


2.  Scope

   Content-aware devices take many forms, shapes and architectures.
   These devices are advanced network interconnect devices that inspect
   deep into the application payload of network data packets to do
   classification.  They may be as simple as a firewall that uses
   application data inspection for rule set enforcement, or they may
   have advanced functionality such as performing protocol decoding and
   validation, anti-virus, anti-spam and even application exploit
   filtering.

   This document is strictly focused on examining performance and
   robustness across a focused set of metrics that may be used to more
   accurately predict device performance when deployed in modern
   networks.  These metrics will be implementation independent.

   It should also be noted that the purpose of this document is not to
   perform functional testing of the potential features in the Device/
   System Under Test (DUT/SUT)[4] nor specify the configurations that
   should be tested.  Various definitions of proper operation and
   configuration may be appropriate within different contexts.  While
   the definition of these parameters are outside the scope of this
   document, the specific configuration of both the DUT and tester
   SHOULD be published with the test results for repeatability and
   comparison purposes.

   While a list of devices that fall under this category will quickly
   become obsolete, an initial list of devices that would be well served
   by utilizing this type of methodology should prove useful.  Devices
   such as firewalls, intrusion detection and prevention devices,
   application delivery controllers, deep packet inspection devices, and
   unified threat management systems generally fall into the content-
   aware category.


3.  Definitions





Hamilton & Banks        Expires September 8, 2011               [Page 4]


Internet-Draft    Terminology for Content-Aware Devices       March 2011


3.1.  Application Flow

   Definition:
      An application flow is the virtual connection between two network
      hosts that is used to exchange user data above the transport
      layer.

   Discussion:
      Content-aware devices may potentially proxy session-layer
      connections, acting as a virtual server to the client and a
      virtual client to the server.  In this mode, the SUT/DUT may
      modify members of the network 5-tuple or act on their behalf, thus
      each end host is actually disconnected at the session layer.
      Application flows are virtual connections that are between the two
      hosts, irrespective of the nature of the session layer semantics.

   Unit of Measurement:
      N/A

   Issues:
      N/A

   See Also:
      5-Tuple

3.2.  Application Throughput

   Definition:
      The rate at which data associated with an application flow is
      transmitted through the SUT/DUT.

   Discussion:
      Throughput metrics may be calculated at various layers in the
      network protocol stack.  Each layer does contain associated
      overhead necessary to maintain that layer.  Application throughput
      is the number of bits transmitted through a SUT/DUT, not including
      the overhead associated with lower layer protocols.  Measurement
      should be taken at the receiver side to minimize the impact of
      session layer retransmissions.

   Unit of Measurement:
      N/A

   Issues:
      Some applications may not rely on session layer reliability
      mechanisms.  This definition does not cover the case where an
      application may utilize its own specific reliability/
      retransmission algorithm.



Hamilton & Banks        Expires September 8, 2011               [Page 5]


Internet-Draft    Terminology for Content-Aware Devices       March 2011


   See Also:
      N/A

3.3.  Average Time to TCP Session Establishment

   Definition:
      The average time that a SUT/DUT requires to complete the TCP
      session establishment process.

   Discussion:
      The average time to TCP session establishment is calculated by
      taking the sum of all "TCP Session Establishment Time" values
      acquired in the specified time frame and divide by the total
      number of sessions established within that timeframe.  The
      timeframe in which the average is taken will depend on the
      methodology itself and what is trying to be measured.

   Unit of Measurement:
      Seconds.

   Issues:
      Depending on how the DUT/SUT handles TCP session establishment,
      the client and server may have different values for the same TCP
      session.  A client-side session may be established prior to the
      server-side session being established.

   See Also:
      See Also.

3.4.  Content-Aware Device

   Definition:
      A networking device which performs deep packet inspection.

   Discussion:
      For a more detailed discussion, please see "deep packet
      inspection".

   Unit of Measurement:
      Not Applicable.

   Issues:
      Not Applicable.

   See Also:
      Deep Packet Inspection





Hamilton & Banks        Expires September 8, 2011               [Page 6]


Internet-Draft    Terminology for Content-Aware Devices       March 2011


3.5.  Deep Packet Inspection

   Definition:
      The process by which a network device inspects layer 7 payload as
      well as protocol headers when making processing decisions.

   Discussion:
      Deep packet inspection (DPI) has grown from a feature reserved for
      Intrusion Prevention Devices into functionality that is shared
      across many next generation networking devices.  Devices
      traditionally classified as firewalls are now looking at layer 7
      payloads to make decisions, whether it is classification, rate-
      shaping, or actually deeming whether a flow is allowed.  Many
      deep-packet inspection devices utilize proxy behavior as a
      functional choice for performing inspection.

   Unit of Measurement:
      Not Applicable.

   Issues:
      Not Applicable.

   See Also:
      Content-Aware Device

3.6.  Network 5-Tuple

   Definition:
      The set of 5 metrics which distinguish two session layer
      connections from each other.

   Discussion:
      When discussing data transfer between hosts, a Network 5-tuple is
      typically used to differentiate between multiple session layer
      connections.  Source and destination IP addresses, source and
      destination session-layer ports, and the session layer protocol
      make up the network 5-tuple.  The session layer protocol is
      typically TCP or UDP, but may be SCTP or another session layer
      protocol.

   Unit of Measurement:
      N/A

   Issues:
      N/A






Hamilton & Banks        Expires September 8, 2011               [Page 7]


Internet-Draft    Terminology for Content-Aware Devices       March 2011


3.7.  Session Establishment Rate

   Definition:
      The rate at which TCP sessions may be established through a given
      DUT/SUT.

   Discussion:
      The session establishment rate is a measurement of how many TCP
      sessions the DUT/SUT is able to establish in a given unit of time.
      If within a 1 second time interval the tester is able to establish
      10,000 sessions, that rate will be measured at 10,000 sessions per
      second.  The session must be established in accordance with the
      policy set forth in "Session Establishment Time".

   Unit of Measurement:
      TCP session(s) per second

   Issues:
      Issues.

   See Also:
      See Also.

3.8.  Session Establishment Time

   Definition:
      Session establishment time is the difference in time between the
      first TCP SYN packet sent from the client and when TCP ACK
      packet's arrival at the server interface.

   Discussion:
      This metric is calculated between the time the first bit of the
      TCP SYN packet is sent from the client and the time the last bit
      of the TCP ACK packet arrives on the server interface.

   Unit of Measurement:
      Seconds.

   Issues:
      Depending on how the DUT/SUT handles TCP session establishment,
      the client and server may have different values for the same
      logical TCP session.  A client-side session may be established
      prior to the server-side session being established.

   See Also:






Hamilton & Banks        Expires September 8, 2011               [Page 8]


Internet-Draft    Terminology for Content-Aware Devices       March 2011


3.9.  Simultaneous TCP Sessions

   Definition:
      The number of TCP sessions which are in the 'Established State' as
      defined by RFC 793 [6].

   Discussion:
      This measurement counts the number of TCP sessions which are in
      the 'Established State'.  Sessions which are in this state must be
      able to maintain data transfer between client and server, bi-
      directionally.

   Unit of Measurement:
      Sessions.

   Issues:
      Depending on the nature of the SUT/DUT, the number of simultaneous
      sessions may instantaneously be different when counted from the
      client and server sides of the SUT/DUT.

   See Also:
      See Also.

3.10.  Time To SYN

   Definition:
      The Time to SYN is a one-way metric, which is the difference
      between the that that the first TCP SYN packet is sent by the
      client and the time at which the server receives the TCP SYN
      packet from the client.

   Discussion:
      This metric is more important with content-aware devices due to
      the potential proxying issues.  Content-aware devices may proxy a
      TCP session on behalf of the server.  Many times, the client will
      receive the SYN/ACK from the DUT/SUT and complete the TCP
      handshake before the SYN has been forwarded to the server.  This
      measurement is actually a proxy measure for client-side session
      establishment time through the DUT/SUT, if the session is in fact
      proxied.

   Unit of Measurement:
      Seconds.

   See Also:






Hamilton & Banks        Expires September 8, 2011               [Page 9]


Internet-Draft    Terminology for Content-Aware Devices       March 2011


4.  IANA Considerations

   This memo includes no request to IANA.

   All drafts are required to have an IANA considerations section (see
   the update of RFC 2434 [9] for a guide).  If the draft does not
   require IANA to do anything, the section contains an explicit
   statement that this is the case (as above).  If there are no
   requirements for IANA, the section will be removed during conversion
   into an RFC by the RFC Editor.


5.  Security Considerations

   Benchmarking activities as described in this memo are limited to
   technology characterization using controlled stimuli in a laboratory
   environment, with dedicated address space and the other constraints
   RFC 2544 [1].

   The benchmarking network topology will be an independent test setup
   and MUST NOT be connected to devices that may forward the test
   traffic into a production network, or misroute traffic to the test
   management network


6.  References

6.1.  Normative References

   [1]  Bradner, S. and J. McQuaid, "Benchmarking Methodology for
        Network Interconnect Devices", RFC 2544, March 1999.

   [2]  Bradner, S., "Benchmarking terminology for network
        interconnection devices", RFC 1242, July 1991.

   [3]  Hickman, B., Newman, D., Tadjudin, S., and T. Martin,
        "Benchmarking Methodology for Firewall Performance", RFC 3511,
        April 2003.

   [4]  Newman, D., "Benchmarking Terminology for Firewall Performance",
        RFC 2647, August 1999.

   [5]  Bradner, S., "Key words for use in RFCs to Indicate Requirement
        Levels", BCP 14, RFC 2119, March 1997.

   [6]  Postel, J., "Transmission Control Protocol", STD 7, RFC 793,
        September 1981.




Hamilton & Banks        Expires September 8, 2011              [Page 10]


Internet-Draft    Terminology for Content-Aware Devices       March 2011


   [7]  Popoviciu, C., Hamza, A., Van de Velde, G., and D. Dugatkin,
        "IPv6 Benchmarking Methodology for Network Interconnect
        Devices", RFC 5180, May 2008.

   [8]  Brownlee, N., Mills, C., and G. Ruth, "Traffic Flow Measurement:
        Architecture", RFC 2722, October 1999.

6.2.  Informative References

   [9]  Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA
        Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.


Authors' Addresses

   Mike Hamilton
   BreakingPoint Systems
   Austin, TX  78717
   US

   Phone: +1 512 636 2303
   Email: mhamilton@breakingpoint.com


   Sarah Banks
   Cisco Systems
   San Jose, CA  95134
   US

   Email: sabanks@cisco.com





















Hamilton & Banks        Expires September 8, 2011              [Page 11]