OSPF Benchmarking Terminology and Concepts
RFC 4062

• Status
• IESG evaluation record
• IESG writeups
• Email expansions
• History

• Versions
• 10

Network Working Group                                          V. Manral
Request for Comments: 4062                                  SiNett Corp.
Category: Informational                                         R. White
                                                           Cisco Systems
                                                               A. Shaikh
                                                    AT&T Labs (Research)
                                                              April 2005

               OSPF Benchmarking Terminology and Concepts

Status of This Memo

   This memo provides information for the Internet community.  It does
   not specify an Internet standard of any kind.  Distribution of this
   memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2005).

Abstract

   This document explains the terminology and concepts used in OSPF
   benchmarking.  Although some of these terms may be defined elsewhere
   (and we will refer the reader to those definitions in some cases) we
   include discussions concerning these terms, as they relate
   specifically to the tasks involved in benchmarking the OSPF protocol.

1.  Introduction

   This document is a companion to [BENCHMARK], which describes basic
   Open Shortest Path First [OSPF] testing methods.  This document
   explains terminology and concepts used in OSPF Testing Framework
   Documents, such as [BENCHMARK].

2.  Specification of Requirements

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].
   [RFC2119] key words in this document are used to ensure
   methodological control, which is very important in the specification
   of benchmarks.  This document does not specify a network-related
   protocol.

Manral, et al.               Informational                      [Page 1]
RFC 4062             OSPF Benchmarking Terminology            April 2005

3.  Common Definitions

   Definitions in this section are well-known industry and benchmarking
   terms that may be defined elsewhere.

   o    White Box (Internal) Measurements

        -    Definition

             White box measurements are those reported and collected on
             the Device Under Test (DUT) itself.

        -    Discussion

             These measurements rely on output and event recording,
             along with the clocking and time stamping available on the
             DUT itself.  Taking measurements on the DUT may impact the
             actual outcome of the test, since it can increase processor
             loading, memory utilization, and timing factors.  Some
             devices may not have the required output readily available
             for taking internal measurements.

             Note: White box measurements can be influenced by the
             vendor's implementation of various timers and processing
             models.  Whenever possible, internal measurements should be
             compared to external measurements to verify and validate
             them.

             Because of the potential for variations in collection and
             presentation methods across different DUTs, white box
             measurements MUST NOT be used as a basis for comparison in
             benchmarks.  This has been a guiding principle of the
             Benchmarking Methodology Working Group.

   o    Black Box (External) Measurements

        -    Definition

             Black box measurements infer the performance of the DUT
             through observation of its communications with other
             devices.

        -    Discussion

             One example of a black box measurement is when a downstream
             device receives complete routing information from the DUT,
             it can be inferred that the DUT has transmitted all the
             routing information available.  External measurements of

Manral, et al.               Informational                      [Page 2]
RFC 4062             OSPF Benchmarking Terminology            April 2005

             internal operations may suffer in that they include not
             just the protocol action times, but also propagation
             delays, queuing delays, and other such factors.

             For the purposes of [BENCHMARK], external techniques are
             more readily applicable.

   o    Multi-device Measurements

        -    Measurements assessing communications (usually in
             combination with internal operations) between two or more
             DUTs.  Multi-device measurements may be internal or
             external.

4.  Terms Defined Elsewhere

   Terms in this section are defined elsewhere and are included only as
   they apply to [BENCHMARK].

   o    Point-to-Point Links

        -    Definition

             See [OSPF], Section 1.2.

        -    Discussion