Network Working Group V. Manral
Request for Comments: 4062 SiNett Corp.
Category: Informational R. White
Cisco Systems
A. Shaikh
AT&T Labs (Research)
April 2005
OSPF Benchmarking Terminology and Concepts
Status of This Memo
This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2005).
Abstract
This document explains the terminology and concepts used in OSPF
benchmarking. Although some of these terms may be defined elsewhere
(and we will refer the reader to those definitions in some cases) we
include discussions concerning these terms, as they relate
specifically to the tasks involved in benchmarking the OSPF protocol.
1. Introduction
This document is a companion to [BENCHMARK], which describes basic
Open Shortest Path First [OSPF] testing methods. This document
explains terminology and concepts used in OSPF Testing Framework
Documents, such as [BENCHMARK].
2. Specification of Requirements
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
[RFC2119] key words in this document are used to ensure
methodological control, which is very important in the specification
of benchmarks. This document does not specify a network-related
protocol.
Manral, et al. Informational [Page 1]
RFC 4062 OSPF Benchmarking Terminology April 2005
3. Common Definitions
Definitions in this section are well-known industry and benchmarking
terms that may be defined elsewhere.
o White Box (Internal) Measurements
- Definition
White box measurements are those reported and collected on
the Device Under Test (DUT) itself.
- Discussion
These measurements rely on output and event recording,
along with the clocking and time stamping available on the
DUT itself. Taking measurements on the DUT may impact the
actual outcome of the test, since it can increase processor
loading, memory utilization, and timing factors. Some
devices may not have the required output readily available
for taking internal measurements.
Note: White box measurements can be influenced by the
vendor's implementation of various timers and processing
models. Whenever possible, internal measurements should be
compared to external measurements to verify and validate
them.
Because of the potential for variations in collection and
presentation methods across different DUTs, white box
measurements MUST NOT be used as a basis for comparison in
benchmarks. This has been a guiding principle of the
Benchmarking Methodology Working Group.
o Black Box (External) Measurements
- Definition
Black box measurements infer the performance of the DUT
through observation of its communications with other
devices.
- Discussion
One example of a black box measurement is when a downstream
device receives complete routing information from the DUT,
it can be inferred that the DUT has transmitted all the
routing information available. External measurements of
Manral, et al. Informational [Page 2]
RFC 4062 OSPF Benchmarking Terminology April 2005
internal operations may suffer in that they include not
just the protocol action times, but also propagation
delays, queuing delays, and other such factors.
For the purposes of [BENCHMARK], external techniques are
more readily applicable.
o Multi-device Measurements
- Measurements assessing communications (usually in
combination with internal operations) between two or more
datatracker.ietf.org