Benchmarking Methodology for Network Interconnect Devices
RFC 2544

• Status
• IESG evaluation record
• IESG writeups
• Email expansions
• History

• Versions

Network Working Group                                         S. Bradner
Request for Comments: 2544                            Harvard University
Obsoletes: 1944                                               J. McQuaid
Category: Informational                                 NetScout Systems
                                                              March 1999

       Benchmarking Methodology for Network Interconnect Devices

Status of this Memo

   This memo provides information for the Internet community.  It does
   not specify an Internet standard of any kind.  Distribution of this
   memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (1999).  All Rights Reserved.

IESG Note

   This document is a republication of RFC 1944 correcting the values
   for the IP addresses which were assigned to be used as the default
   addresses for networking test equipment. (See section C.2.2 ).  This
   RFC replaces and obsoletes RFC 1944.

Abstract

   This document discusses and defines a number of tests that may be
   used to describe the performance characteristics of a network
   interconnecting  device.  In addition to defining the tests this
   document also describes specific formats for reporting the results of
   the tests.  Appendix A lists the tests and conditions that we believe
   should be included for specific cases and gives additional
   information about testing practices.  Appendix B is a reference
   listing of maximum frame rates to be used with specific frame sizes
   on various media and Appendix C gives some examples of frame formats
   to be used in testing.

1. Introduction

   Vendors often engage in "specsmanship" in an attempt to give their
   products a better position in the marketplace.  This often involves
   "smoke & mirrors" to confuse the potential users of the products.

Bradner & McQuaid            Informational                      [Page 1]
RFC 2544                Benchmarking Methodology              March 1999

   This document defines a specific set of tests that vendors can use to
   measure and report the performance characteristics of network
   devices.  The results of these tests will provide the user comparable
   data from different vendors with which to evaluate these devices.

   A previous document, "Benchmarking Terminology for Network
   Interconnect Devices" (RFC 1242), defined many of the terms that are
   used in this document.  The terminology document should be consulted
   before attempting to make use of this document.

2. Real world

   In producing this document the authors attempted to keep in mind the
   requirement that apparatus to perform the described tests must
   actually be built.  We do not know of "off the shelf" equipment
   available to implement all of the tests but it is our opinion that
   such equipment can be constructed.

3. Tests to be run

   There are a number of tests described in this document.  Not all of
   the tests apply to all types of devices under test (DUTs). Vendors
   should perform all of the tests that can be supported by a specific
   type of product.  The authors understand that it will take a
   considerable period of time to perform all of the recommended tests
   nder  all of the recommended conditions. We believe that the results
   are worth the effort.  Appendix A lists some of the tests and
   conditions that we believe should be included for specific cases.

4. Evaluating the results

   Performing all of the recommended tests will result in a great deal
   of data. Much of this data will not apply to the evaluation of the
   devices under each circumstance.  For example, the rate at which a
   router forwards IPX frames will be of little use in selecting a
   router for an environment that does not (and will not) support that
   protocol.  Evaluating even that data which is relevant to a
   particular network installation will require experience which may not
   be readily available. Furthermore, selection of the tests to be run
   and evaluation of the test data must be done with an understanding of
   generally accepted testing practices regarding repeatability,
   variance and statistical significance of small numbers of trials.

Bradner & McQuaid            Informational                      [Page 2]
RFC 2544                Benchmarking Methodology              March 1999

5. Requirements

   In this document, the words that are used to define the significance
   of each particular requirement are capitalized. These words are:

      * "MUST" This word, or the words "REQUIRED" and "SHALL" mean that
         the item is an absolute requirement of the specification.

      * "SHOULD" This word or the adjective "RECOMMENDED" means that
         there may exist valid reasons in particular circumstances to
         ignore this item, but the full implications should be
         understood and the case carefully weighed before choosing a
         different course.