Network Working Group R. Mandeville
Internet-Draft European Network Laboratories
Expiration Date: February 1999 J. Keene
Netcom Systems
August 1998
Benchmarking Methodology for LAN Switching Devices
<draft-ietf-bmwg-mswitch-00.txt>
Status of this Memo
This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF),its areas,
and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
To view the entire list of current Internet-Drafts, please check
the "1id-abstracts.txt" listing contained in the Internet-Drafts
Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
(Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au
(Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu
(US West Coast).
This memo provides information for the Internet community. This memo
does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.
Abstract
This document is intended to provide methodology for the benchmarking
of local area network (LAN) switching devices. It extends the
methodology already defined for benchmarking network interconnecting
devices in RFC 1944 to switching devices.
This RFC primarily deals with devices which switch frames at the
Medium Access Control (MAC) layer. It provides a methodology for
benchmarking switching devices, forwarding performance, congestion
control, latency, address handling and filtering. In addition to
defining the tests, this document also describes specific formats for
reporting the results of the tests.
1. Introduction
Mandeville, Keene [Page 1]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
This document defines a specific set of tests to measure and report
the performance characteristics of network switching devices.
A previous document, "Benchmarking Terminology for LAN Switching
Devices" (RFC 2285), defined many of the terms that are used in this
document. The terminology document SHOULD be consulted before
attempting to make use of this document.
2. Requirements
The following RFCs SHOULD be consulted before attempting to make use
of this document:
* RFC 1242 "Benchmarking Terminology for Network Interconnect
Devices"
* RFC 1944 "Benchmarking Methodology for Network Interconnect
Devices"
* RFC 2285 "Benchmarking Terminology for LAN Switching Devices"
For the sake of clarity and continuity, this RFC adopts the template
for benchmarking tests set out in Section 26 of RFC 1944.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this document are to be interpreted as described in RFC 2119.
3. Test setup
This document extends the general test setup described in section 6
of RFC 1944 to the benchmarking of LAN switching devices. RFC 1944
primarily describes non-meshed traffic where input and output
interfaces are grouped in mutually exclusive sending and
receiving pairs. In fully meshed traffic, each interface of a
DUT/SUT is set up to both receive and transmit frames to all
the other interfaces under test.
Prior to each test run, the DUT/SUT MUST learn the MAC addresses used
in
the test and the address learning MUST be verified to avoid flooded
frames being counted as correctly received frames. The forwarding
rate, namely the rate at which address learning frames are offered
may
have to be adjusted to be as low as 50 frames per second or even
less,
to guarantee successful learning. The DUT/SUT address aging time
SHOULD be
configured to be greater than the period between the learning phase
of
the test and the test run; in non-meshed and partially meshed tests,
the aging time SHOULD at a minimum be set to at least the length of
the test period. More than one trial may be needed for the
association
Mandeville, Keene [Page 2]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
of the address to the port to occur.
If a DUT/SUT uses a hashing algorithm with address learning, the
DUT/SUT may not learn the necessary addresses to perform the tests.
The format of the MAC addresses MUST be adjustable so that the
address
mapping may be re-arranged to make a DUT/SUT learn addresses
without confusion.
It is recommended that SNMP and Spanning Tree be disabled when bench-
marking switching devices unless investigating overhead behavior. If
such protocols cannot be turned off, it is recommended that the
levels of offered load be reduced (less than 100%) to allow for
the additional management frames.
4. Frame formats and sizes
For frame formats and sizes, refer to RFC 1944, sections 8 and 9 and
Appendix C.
There are three frame formats for layer 2 Ethernet switches:
standard MAC Ethernet frames, standard MAC Ethernet frames
with vendor-specific tags added to them, and IEEE 802.3ab
frames tagged to accommodate 802.3p,q. The two types of tagged
frames may exceed the standard maximum length frame of 1518 bytes,
and
may not be accepted by the interface controllers of some DUT/SUTs.
It is recommended to check the compatibility of the DUT/SUT
with tagged frames before testing.
Devices switching tagged frames of over 1518 bytes will have a lower
maximum forwarding rate than standard untagged frames.
5. Benchmarking Tests
The following tests offer objectives, procedures, and reporting
formats for benchmarking LAN switching devices.
5.1 Fully meshed throughput, frame loss and forwarding rates
5.2 Partially meshed overloading
5.3 Head of line blocking
5.4 Partially meshed multiple devices
5.5 Multiple streams of unidirectional traffic
5.6 Filter illegal frames
5.7 Broadcast frame handling and latency
5.8 Maximum forwarding rate and minimum interframe gap
5.9 Address caching capacity
5.10 Address learning rate
Mandeville, Keene [Page 3]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
5.1 Fully meshed throughput, frame loss and forwarding rates
Objective:
To determine the throughput, frame loss and forwarding rates of
DUT/SUTs
offered fully meshed traffic as defined in RFC 2285.
Procedure:
RFC 2285 points out that when offering bursty meshed traffic, the
variables which MUST be defined are frame size, bust size,
interframe gap, interburst gap, and load. Each variable is
configured with the following considerations.
Interframe Gap (IFG) - The IFG between frames inside a burst
MUST
be at the minimum specified by the standard (9.6 us for 10Mbps
Ethernet and 0.96 us for 100Mbps Ethernet).
Interburst Gap (IBG) - This is the interval between bursts of
traffic. Refer to Appendix A, Calculating Interburst Gap, for
the formula used to compute IBG.
Load / Port - The test SHOULD be run multiple times with a
different load per port in each case. The 100% load translates to
a transmit load of 50% for half duplex. This type of test SHOULD
also be run at higher than 100% loads.
In half duplex mode, exactly half of the target load SHOULD be
sent to each of the ports under test. For example, with a 100%
load of 64-byte frames, the target load for each port under test
is 7440 frames received per second and 7440 frames transmitted per
second (for 10Mbps Ethernet).
Frame Size - Recommended frame sizes are 64, 128, 256, 512, 1024,
1280 and 1518 bytes, per RFC 1944 section 9.
Burst Size - The burst size defines the number of packets sent
back-to-back at the minimum legal IFG (96 bit times) before
pausing transmission to receive frames. Burst sizes
SHOULD vary between 1 and 930 frames.
To avoid truncating bursts, a burst size SHOULD be an even
multiple into 7440. 7440 is the maximum frames per second for half
duplex mode; 14880 is the maximum frames per second for full
duplex mode. Refer to Appendix A for a table of recommended burst
values.
Mandeville, Keene [Page 4]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
Each port in the test sends frames to all other ports in a round-
robin type fashion. The following table shows how each port in a test
SHOULD transmit frames to all other ports in the test. In this
example, there are six ports with 1 address per port:
Source Port, then Destination Ports (in order of transmission)
Port #1 2 3 4 5 6 2...
Port #2 3 4 5 6 1 3...
Port #3 4 5 6 1 2 4...
Port #4 5 6 1 2 3 5...
Port #5 6 1 2 3 4 6...
Port #6 1 2 3 4 5 1...
As shown in the table, there is an equal distribution of destination
addresses for each transmit opportunity. This keeps the test balanced
so that one destination port is not overloaded by the test algorithm
and all ports are equally and fully loaded throughout the test. For
tests using multiple addresses per port, the actual port destinations
are the same as described above and the actual source/destination
address
pairs are chosen randomly to exercise the DUT/SUT's ability to
perform
address lookups.
For every address, the testing device sends learning packets to allow
the DUT/SUT to load its address tables properly.
To measure the DUT/SUT's ability to switch traffic while performing
many
different address lookups, the number of addresses per port SHOULD be
increased in a series of tests.
Reporting format:
In these tests, a port SHOULD transmit and receive the same amount of
packets. Each port MUST count the packets received with a valid
address.
Any packet received which does not have a valid address MUST not be
counted as a received packet and can be counted as part of a flood
count as described in 3.8.3 in RFC 2285.
The results for these tests SHOULD be reported in the form of
numerical
data or a graph with text to indicate the data type. The data types
for
the throughput test and for the frame loss rate test are described in
RFC 1944.
Forwarding rate (FR) of the DUT/SUT SHOULD be reported as the number
of
frames per second that the device is observed to successfully
transmit
to the correct destination interface in response to a specified
offered
load. The offered load MUST also be cited.
Mandeville, Keene [Page 5]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
Forwarding rate at maximum offered load (FRMOL ) SHOULD be reported
as
the number of frames per second that a device can successfully
transmit
to the correct destination interface in response to the maximum
offered
load as defined in RFC 2285, section 3.6. The maximum offered load
MUST
also be cited.
Maximum forwarding rate (MFR) SHOULD be reported as the highest
forwarding
rate of a DUT/SUT taken from an iterative set of forwarding rate
measurements. The load applied to the device MUST also be cited.
5. 2 Partially meshed overloading
To be done.
5. 3 Head-of-line blocking
To be done.
5.4 Partially Meshed Multiple Devices
To be done.
5.5 Multiple streams of unidirectional traffic
To be done.
5.6 Filter illegal frames
To be done.
5.7 Broadcast frame handling and latency test
To be done.
5.8 Maximum forwarding rate and minimum interframe gap
To be done.
5.9 Address Caching Capacity
Objective:
To determine the address caching capacity of a LAN switching device
as defined in RFC 2285, section 3.8.1.
Procedure:
Mandeville, Keene [Page 6]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
This test SHOULD at a minimum be performed in a three-port
configuration as described below.
The first port (port 1) of the testing device is connected to the
DUT/SUT
and is the port from which a testing device sends frames with varying
source addresses and a fixed destination address corresponding to the
MAC
address of the receiving port. By receiving frames with varying
source
addresses, the DUT/SUT will learn these new addresses from the
sending port
of the test device.
A second port (port 2) of the testing device is connected to the
DUT/SUT
and acts as the receiving port for the address learning frames. This
port also sends "control" frames back to the addresses learned
on the first port. The algorithm for this is explained below.
-- A third port (port 3) on the testing device MUST be connected to
a
port on the DUT/SUT and act as a monitoring port to listen for
flooded frames.
The algorithm for the test is as follows:
BEGIN
Set Initial Value of N to user specified number, where N is the
number of addresses to be verified in each iteration
WHILE NOT Finished DO
PAUSE for the aging time specified
Address learning: Port 1 sends N frames with varying
source addresses to Port 2 to attempt to fill the DUT/SUT
address table for Port #1.
Controlling: Port 2 sends N frames with varying
destination addresses corresponding to Port 1.
IF (Port 3 received a frame during Control phase) OR
(Port 1 did not receive the correct # of frames)
THEN
Address Table of DUT/SUT was full
Set N to lower number (in binary search method)
ELSE
Address Table of DUT/SUT was NOT full
Set N to higher number (in binary search method)
IF High and Low Values of N Meet
THEN
Test is Finished, Value of N equals number of
addresses supported by DUT/SUT
ELSE
Continue Test
END WHILE
DONE
Mandeville, Keene [Page 7]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
Using a binary search approach, the test targets the exact number
of addresses supported per port with minimal test iterations. Due
to the aging time of DUT/SUT address tables, each iteration may take
some time during the waiting period for the addresses to clear. If
possible, configure the DUT/SUT for a low value for the aging time.
Once the high and low values of N meet, then the threshold of address
handling has been found.
For this test, the aging time of the DUT/SUT MUST be known. Use a
short
aging time if possible to reduce the time needed to run the test.
The aging time MUST be longer than the time necessary to produce
packets at the specified rate. If a low frame rate is used for
the test, then it may be possible that sending a large amount of
frames may actually take longer than the aging time. Keep in mind
that the test actually sends twice as many frames as the number
of addresses being tested due to the learning phase and the
controlling phase.
Set the initial value of addresses per port to a number
slightly higher than the number of addresses that the DUT/SUT can
handle.
This step primes the binary search and can reduce the number of
iterations required to determine the exact number supported.
Set the forwarding rate to a number that is reasonable to be handled
by the DUT/SUT and one that is high enough so that the test
iterations are
not too long.
Reporting format:
After the test is run, results for each iteration SHOULD be displayed
in a table to include:
-- the number of addresses used for each test iteration.
-- the frame rate used for each test iteration.
-- number of control frames that were transmitted by test port
number 2. Control frames are the frames sent with varying destination
addresses to confirm that the DUT/SUT has learned all of the
addresses
for each test iteration.
-- the number of frames received by test port 2 during the control
portion of each test. If the number is non-zero, this is an
indication
of the DUT/SUT flooding a frame in which the destination address is
not
in the address table.
-- the number of frames that test port 1 received during the control
Mandeville, Keene [Page 8]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
portion of the test. This number will include those frames which are
Spanning tree frames. For a normal test iteration, this number SHOULD
be equal to the number of frames transmitted by port 2 during the
control phase.
-- the number of frames that test port 1 received during the control
phase not including Spanning Tree or other non test generated frames.
In all cases, this number SHOULD be equal to the number of frames
transmitted by port 2 during the control phase of the test.
-- the number of frames test port 3 received during the control
phase
of the test. If the value is not zero, then this indicates that for
that test iteration, the DUT/SUT could not determine the proper
destination
port for that many frames. In other words, the DUT/SUT flooded the
frame to
all ports since its address table was full.
5.10 Address Learning Rate
Objective:
Once the maximum number of addresses supported per port by the
DUT/SUT is
known, the addressing learning rate (the rate at which the DUT/SUT
learns
these addresses) can be determined.
Procedure:
An algorithm similar to the one used to determine address caching
capacity can be used to determine the address learning rate. This
test iterates the rate at which address learning frames are offered
by the test device connected to the DUT/SUT. It is recommended to
set
the number of addresses offered to the DUT/SUT in this test to the
maximum caching capacity. However, the address learning rate might
be
determined for different numbers of addresses but in each test run,
the number MUST remain constant.
Initializing the forwarding rate primes the binary search algorithm
and can help to shorten the overall test duration.
A third port on the DUT/SUT MUST listen for flooded frames.
In this test, the forwarding rate SHOULD be varied until a rate is
found that is the threshold for the rate supported by the DUT/SUT. It
may be useful to pick a value slightly higher than the advertised
forwarding rate.
Reporting format:
Mandeville, Keene [Page 9]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
To be done.
Section 6. Security Considerations
This document does not yet address Security Considerations.
Section 7. Authors' Address
Robert Mandeville
European Network Laboratories (ENL)
2, rue Helene Boucher
87286 Guyancourt Cedex
France
Phone: + 33 1 39 44 12 05
EMail: bob@enl.net
Judy Keene
Netcom Systems
20550 Nordhoff St.
Chatsworth, CA 91311
USA
Phone: +1 818 700 5100
Email: judy_keene@netcomsystems.com
Appendix A: Calculating the Interburst Gap
IBG is defined in RFC 2285 as a function of maximum media rate (also
known as line rate), the length of the frames in the bursts with the
preamble and the interframe gap (IFG), the number of frames in the
bursts,
and the intended load.
Using the burst size, frame size and the load per port, the IBG can
be
calculated:
Burst size * ((preamble 64 + (frame size * 8 bits) + 96 IFG bit
times))
* 1 (for full duplex) or .5 (for half duplex)
(10 bit times equal 1 microsecond)
Example:
Burst size = 24
Mandeville, Keene [Page 10]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
Frame size = 64
24 * (64 + 64*8 + 96) = 24 * (672) = 16,128 bit/times =
1612.8 us IBG
In half duplex mode, exactly half of the target load SHOULD be sent
to each of the ports under test. For example, with a 100% load of
64-byte frames, the target load for each port under test is 7440
frames received per second and 7440 frames transmitted per second
(for 10Mbps Ethernet).
Recommended characteristic values for burst size are:
2 15 60 240 930
3 16 62 248
4 20 80 310
5 24 93 372
6 30 120 465
8 31 124 496
10 40 155 620
12 48 186 744
Mandeville, Keene [Page 11]
Network Working Group R. Mandeville
Internet-Draft European Network Laboratories
Expiration Date: February 1999 J. Keene
Netcom Systems
August 1998
Benchmarking Methodology for LAN Switching Devices
<draft-ietf-bmwg-mswitch-00.txt>
Status of this Memo
This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF),its areas,
and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
To view the entire list of current Internet-Drafts, please check
the "1id-abstracts.txt" listing contained in the Internet-Drafts
Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
(Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au
(Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu
(US West Coast).
This memo provides information for the Internet community. This memo
does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.
Abstract
This document is intended to provide methodology for the benchmarking
of local area network (LAN) switching devices. It extends the
methodology already defined for benchmarking network interconnecting
devices in RFC 1944 to switching devices.
This RFC primarily deals with devices which switch frames at the
Medium Access Control (MAC) layer. Itprovides a methodology for
benchmarking switching devices, forwarding performance, congestion
control, latency, address handling and filtering. In addition to
defining the tests, this document also describes specific formats for
reporting the results of the tests.
1. Introduction
Mandeville, Keene [Page 1]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
This document defines a specific set of tests to measure and report
the performance characteristics of network switching devices.
A previous document, "Benchmarking Terminology for LAN Switching
Devices" (RFC 2285), defined many of the terms that are used in this
document. The terminology document SHOULD be consulted before
attempting to make use of this document.
2. Requirements
The following RFCs SHOULD be consulted before attempting to make use
of this document:
* RFC 1242 "Benchmarking Terminology for Network Interconnect
Devices"
* RFC 1944 "Benchmarking Methodology for Network Interconnect
Devices"
* RFC 2285 "Benchmarking Terminology for LAN Switching Devices"
For the sake of clarity and continuity, this RFC adopts the template
for benchmarking tests set out in Section 26 of RFC 1944.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this document are to be interpreted as described in RFC 2119.
3. Test setup
This document extends the general test setup described in section 6
of RFC 1944 to the benchmarking of LAN switching devices. RFC 1944
primarily describes non-meshed traffic where input and output
interfaces are grouped in mutually exclusive sending and
receiving pairs. In fully meshed traffic, each interface of a
DUT/SUT is set up to both receive and transmit frames to all
the other interfaces under test.
Prior to each test run, the DUT/SUT MUST learn the MAC addresses used
in
the test and the address learning MUST be verified to avoid flooded
frames being counted as correctly received frames. The forwarding
rate, namely the rate at which address learning frames are offered
may
have to be adjusted to be as low as 50 frames per second or even
less,
to guarantee successful learning. The DUT/SUT address aging time
SHOULD be
configured to be greater than the period between the learning phase
of
the test and the test run; in non-meshed and partially meshed tests,
the aging time SHOULD at a minimum be set to at least the length of
the test period. More than one trial may be needed for the
association
Mandeville, Keene [Page 2]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
of the address to the port to occur.
If a DUT/SUT uses a hashing algorithm with address learning, the
DUT/SUT may not learn the necessary addresses to perform the tests.
The format of the MAC addresses MUST be adjustable so that the
address
mapping may be re-arranged to make a DUT/SUT learn addresses
without confusion.
It is recommended that SNMP and Spanning Tree be disabled when bench-
marking switching devices unless investigating overhead behavior. If
such protocols cannot be turned off, it is recommended that the
levels of offered load be reduced (less than 100%) to allow for
the additional management frames.
4. Frame formats and sizes
For frame formats and sizes, refer to RFC 1944, sections 8 and 9 and
Appendix C.
There are three frame formats for layer 2 Ethernet switches:
standard MAC Ethernet frames, standard MAC Ethernet frames
with vendor-specific tags added to them, and IEEE 802.3ab
frames tagged to accommodate 802.3p,q. The two types of tagged
frames may exceed the standard maximum length frame of 1518 bytes,
and
may not be accepted by the interface controllers of some DUT/SUTs.
It is recommended to check the compatibility of the DUT/SUT
with tagged frames before testing.
Devices switching tagged frames of over 1518 bytes will have a lower
maximum forwarding rate than standard untagged frames.
5. Benchmarking Tests
The following tests offer objectives, procedures, and reporting
formats for benchmarking LAN switching devices.
5.1 Fully meshed throughput, frame loss and forwarding rates
5.2 Partially meshed overloading
5.3 Head of line blocking
5.4 Partially meshed multiple devices
5.5 Multiple streams of unidirectional traffic
5.6 Filter illegal frames
5.7 Broadcast frame handling and latency
5.8 Maximum forwarding rate and minimum interframe gap
5.9 Address caching capacity
5.10 Address learning rate
Mandeville, Keene [Page 3]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
5.1 Fully meshed throughput, frame loss and forwarding rates
Objective:
To determine the throughput, frame loss and forwarding rates of
DUT/SUTs
offered fully meshed traffic as defined in RFC 2285.
Procedure:
RFC 2285 points out that when offering bursty meshed traffic, the
variables which MUST be defined are frame size, bust size,
interframe gap, interburst gap, and load. Each variable is
configured with the following considerations.
Interframe Gap (IFG) - The IFG between frames inside a burst
MUST
be at the minimum specified by the standard (9.6 us for 10Mbps
Ethernet and 0.96 us for 100Mbps Ethernet).
Interburst Gap (IBG) - This is the interval between bursts of
traffic. Refer to Appendix A, Calculating Interburst Gap, for
the formula used to compute IBG.
Load / Port - The test SHOULD be run multiple times with a
different load per port in each case. The 100% load translates to
a transmit load of 50% for half duplex. This type of test SHOULD
also be run at higher than 100% loads.
In half duplex mode, exactly half of the target load SHOULD be
sent to each of the ports under test. For example, with a 100%
load of 64-byte frames, the target load for each port under test
is 7440 frames received per second and 7440 frames transmitted per
second (for 10Mbps Ethernet).
Frame Size - Recommended frame sizes are 64, 128, 256, 512, 1024,
1280 and 1518 bytes, per RFC 1944 section 9.
Burst Size - The burst size defines the number of packets sent
back-to-back at the minimum legal IFG (96 bit times) before
pausing transmission to receive frames. Burst sizes
SHOULD vary between 1 and 930 frames.
To avoid truncating bursts, a burst size SHOULD be an even
multiple into 7440. 7440 is the maximum frames per second for half
duplex mode; 14880 is the maximum frames per second for full
duplex mode. Refer to Appendix A for a table of recommended burst
values.
Mandeville, Keene [Page 4]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
Each port in the test sends frames to all other ports in a round-
robin type fashion. The following table shows how each port in a test
SHOULD transmit frames to all other ports in the test. In this
example, there are six ports with 1 address per port:
Source Port, then Destination Ports (in order of transmission)
Port #1 2 3 4 5 6 2...
Port #2 3 4 5 6 1 3...
Port #3 4 5 6 1 2 4...
Port #4 5 6 1 2 3 5...
Port #5 6 1 2 3 4 6...
Port #6 1 2 3 4 5 1...
As shown in the table, there is an equal distribution of destination
addresses for each transmit opportunity. This keeps the test balanced
so that one destination port is not overloaded by the test algorithm
and all ports are equally and fully loaded throughout the test. For
tests using multiple addresses per port, the actual port destinations
are the same as described above and the actual source/destination
address
pairs are chosen randomly to exercise the DUT/SUT's ability to
perform
address lookups.
For every address, the testing device sends learning packets to allow
the DUT/SUT to load its address tables properly.
To measure the DUT/SUT's ability to switch traffic while performing
many
different address lookups, the number of addresses per port SHOULD be
increased in a series of tests.
Reporting format:
In these tests, a port SHOULD transmit and receive the same amount of
packets. Each port MUST count the packets received with a valid
address.
Any packet received which does not have a valid address MUST not be
counted as a received packet and can be counted as part of a flood
count as described in 3.8.3 in RFC 2285.
The results for these tests SHOULD be reported in the form of
numerical
data or a graph with text to indicate the data type. The data types
for
the throughput test and for the frame loss rate test are described in
RFC 1944.
Forwarding rate (FR) of the DUT/SUT SHOULD be reported as the number
of
frames per second that the device is observed to successfully
transmit
to the correct destination interface in response to a specified
offered
load. The offered load MUST also be cited.
Mandeville, Keene [Page 5]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
Forwarding rate at maximum offered load (FRMOL ) SHOULD be reported
as
the number of frames per second that a device can successfully
transmit
to the correct destination interface in response to the maximum
offered
load as defined in RFC 2285, section 3.6. The maximum offered load
MUST
also be cited.
Maximum forwarding rate (MFR) SHOULD be reported as the highest
forwarding
rate of a DUT/SUT taken from an iterative set of forwarding rate
measurements. The load applied to the device MUST also be cited.
5. 2 Partially meshed overloading
To be done.
5. 3 Head-of-line blocking
To be done.
5.4 Partially Meshed Multiple Devices
To be done.
5.5 Multiple streams of unidirectional traffic
To be done.
5.6 Filter illegal frames
To be done.
5.7 Broadcast frame handling and latency test
To be done.
5.8 Maximum forwarding rate and minimum interframe gap
To be done.
5.9 Address Caching Capacity
Objective:
To determine the address caching capacity of a LAN switching device
as defined in RFC 2285, section 3.8.1.
Procedure:
Mandeville, Keene [Page 6]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
This test SHOULD at a minimum be performed in a three-port
configuration as described below.
The first port (port 1) of the testing device is connected to the
DUT/SUT
and is the port from which a testing device sends frames with varying
source addresses and a fixed destination address corresponding to the
MAC
address of the receiving port. By receiving frames with varying
source
addresses, the DUT/SUT will learn these new addresses from the
sending port
of the test device.
A second port (port 2) of the testing device is connected to the
DUT/SUT
and acts as the receiving port for the address learning frames. This
port also sends "control" frames back to the addresses learned
on the first port. The algorithm for this is explained below.
-- A third port (port 3) on the testing device MUST be connected to
a
port on the DUT/SUT and act as a monitoring port to listen for
flooded frames.
The algorithm for the test is as follows:
BEGIN
Set Initial Value of N to user specified number, where N is the
number of addresses to be verified in each iteration
WHILE NOT Finished DO
PAUSE for the aging time specified
Address learning: Port 1 sends N frames with varying
source addresses to Port 2 to attempt to fill the DUT/SUT
address table for Port #1.
Controlling: Port 2 sends N frames with varying
destination addresses corresponding to Port 1.
IF (Port 3 received a frame during Control phase) OR
(Port 1 did not receive the correct # of frames)
THEN
Address Table of DUT/SUT was full
Set N to lower number (in binary search method)
ELSE
Address Table of DUT/SUT was NOT full
Set N to higher number (in binary search method)
IF High and Low Values of N Meet
THEN
Test is Finished, Value of N equals number of
addresses supported by DUT/SUT
ELSE
Continue Test
END WHILE
DONE
Mandeville, Keene [Page 7]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
Using a binary search approach, the test targets the exact number
of addresses supported per port with minimal test iterations. Due
to the aging time of DUT/SUT address tables, each iteration may take
some time during the waiting period for the addresses to clear. If
possible, configure the DUT/SUT for a low value for the aging time.
Once the high and low values of N meet, then the threshold of address
handling has been found.
For this test, the aging time of the DUT/SUT MUST be known. Use a
short
aging time if possible to reduce the time needed to run the test.
The aging time MUST be longer than the time necessary to produce
packets at the specified rate. If a low frame rate is used for
the test, then it may be possible that sending a large amount of
frames may actually take longer than the aging time. Keep in mind
that the test actually sends twice as many frames as the number
of addresses being tested due to the learning phase and the
controlling phase.
Set the initial value of addresses per port to a number
slightly higher than the number of addresses that the DUT/SUT can
handle.
This step primes the binary search and can reduce the number of
iterations required to determine the exact number supported.
Set the forwarding rate to a number that is reasonable to be handled
by the DUT/SUT and one that is high enough so that the test
iterations are
not too long.
Reporting format:
After the test is run, results for each iteration SHOULD be displayed
in a table to include:
-- the number of addresses used for each test iteration.
-- the frame rate used for each test iteration.
-- number of control frames that were transmitted by test port
number 2. Control frames are the frames sent with varying destination
addresses to confirm that the DUT/SUT has learned all of the
addresses
for each test iteration.
-- the number of frames received by test port 2 during the control
portion of each test. If the number is non-zero, this is an
indication
of the DUT/SUT flooding a frame in which the destination address is
not
in the address table.
-- the number of frames that test port 1 received during the control
Mandeville, Keene [Page 8]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
portion of the test. This number will include those frames which are
Spanning tree frames. For a normal test iteration, this number SHOULD
be equal to the number of frames transmitted by port 2 during the
control phase.
-- the number of frames that test port 1 received during the control
phase not including Spanning Tree or other non test generated frames.
In all cases, this number SHOULD be equal to the number of frames
transmitted by port 2 during the control phase of the test.
-- the number of frames test port 3 received during the control
phase
of the test. If the value is not zero, then this indicates that for
that test iteration, the DUT/SUT could not determine the proper
destination
port for that many frames. In other words, the DUT/SUT flooded the
frame to
all ports since its address table was full.
5.10 Address Learning Rate
Objective:
Once the maximum number of addresses supported per port by the
DUT/SUT is
known, the addressing learning rate (the rate at which the DUT/SUT
learns
these addresses) can be determined.
Procedure:
An algorithm similar to the one used to determine address caching
capacity can be used to determine the address learning rate. This
test iterates the rate at which address learning frames are offered
by the test device connected to the DUT/SUT. It is recommended to
set
the number of addresses offered to the DUT/SUT in this test to the
maximum caching capacity. However, the address learning rate might
be
determined for different numbers of addresses but in each test run,
the number MUST remain constant.
Initializing the forwarding rate primes the binary search algorithm
and can help to shorten the overall test duration.
A third port on the DUT/SUT MUST listen for flooded frames.
In this test, the forwarding rate SHOULD be varied until a rate is
found that is the threshold for the rate supported by the DUT/SUT. It
may be useful to pick a value slightly higher than the advertised
forwarding rate.
Reporting format:
Mandeville, Keene [Page 9]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
To be done.
Section 6. Security Considerations
This document does not yet address Security Considerations.
Section 7. Authors' Address
Robert Mandeville
European Network Laboratories (ENL)
2, rue Helene Boucher
87286 Guyancourt Cedex
France
Phone: + 33 1 39 44 12 05
EMail: bob@enl.net
Judy Keene
Netcom Systems
20550 Nordhoff St.
Chatsworth, CA 91311
USA
Phone: +1 818 700 5100
Email: judy_keene@netcomsystems.com
Appendix A: Calculating the Interburst Gap
IBG is defined in RFC 2285 as a function of maximum media rate (also
known as line rate), the length of the frames in the bursts with the
preamble and the interframe gap (IFG), the number of frames in the
bursts,
and the intended load.
Using the burst size, frame size and the load per port, the IBG can
be
calculated:
Burst size * ((preamble 64 + (frame size * 8 bits) + 96 IFG bit
times))
* 1 (for full duplex) or .5 (for half duplex)
(10 bit times equal 1 microsecond)
Example:
Burst size = 24
Mandeville, Keene [Page 10]
INTERNET-DRAFT LAN Switch Benchmarking Methodology August 1998
Frame size = 64
24 * (64 + 64*8 + 96) = 24 * (672) = 16,128 bit/times =
1612.8 us IBG
In half duplex mode, exactly half of the target load SHOULD be sent
to each of the ports under test. For example, with a 100% load of
64-byte frames, the target load for each port under test is 7440
frames received per second and 7440 frames transmitted per second
(for 10Mbps Ethernet).
Recommended characteristic values for burst size are:
2 15 60 240 930
3 16 62 248
4 20 80 310
5 24 93 372
6 30 120 465
8 31 124 496
10 40 155 620
12 48 186 744
Mandeville, Keene [Page 11]
