Network Working Group
INTERNET-DRAFT
Expires in: December 2003
Scott Poretsky
Avici Systems
Brent Imhoff
Wiltel Communications
June 2003
Benchmarking Methodology for
IGP Data Plane Route Convergence
<draft-ietf-bmwg-igp-dataplane-conv-meth-00.txt>
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as "work in
progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Table of Contents
1. Introduction ...............................................2
2. Existing definitions .......................................2
3. Test Setup..................................................2
3.1 Test Topologies............................................2
3.2 Test Considerations........................................3
4. Test Cases..................................................4
4.1 Local Events...............................................4
4.1.1 Convergence Due to SONET Link Failure....................4
4.1.2 Convergence Due to PPP Session Failure...................5
4.1.3 Convergence Due to IGP Adjacency Failure.................5
4.1.4 Convergence Due to Route Withdrawal......................6
4.1.5 Convergence Due to Cost Change...........................7
4.2 Remote Events..............................................7
4.2.1 Convergence Due to Remote SONET Link Failure.............7
Poretsky, Imhoff [Page 1]
INTERNET-DRAFT Benchmarking Methodology for June 2003
IGP Data Plane Route Convergence
5. Measuring Convergence Times.................................8
5.1 Measuring Peak-to-Peak Convergence Time....................8
5.2 Measuring Impact of Components for Convergence.............8
6. Security Considerations.....................................9
7. Acknowledgements............................................9
8. References..................................................9
9. Author's Address............................................9
10. Full Copyright Statement...................................10
1. Introduction
This draft describes the methodology for benchmarking IGP Route
Convergence. The applicability of this testing is described in
[1] and the new terminology that it introduces is defined in [2].
Service Providers use IGP Convergence time as a key metric of
router design and architecture. Customers of Service Providers
observe convergence time by packet loss. IGP Route Convergence
is a Direct Measure of Quality (DMOQ) when benchmarking the data
plane and not the control plane. The test cases in this document
are black-box tests that emulate the network events that cause
route convergence, as described in [1]. Black-box test design
accounts for all of the factors for route convergence time, as
provided in [1]. The methodology and terminology is to be used
for benchmarking route convergence and can be applied to any
link-state IGP such as ISIS [3] and OSPF [4].
2. Existing definitions
For the sake of clarity and continuity this RFC adopts the template
for definitions set out in Section 2 of RFC 1242. Definitions are
indexed and grouped together in sections for ease of reference.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this document are to be interpreted as described in RFC 2119.
3. Test Setup
3.1 Test Topologies
--------- Ingress Traffic Path ---------
| |<------------------------------| |
| | | |
| | Preferred Egress Path | |
| DUT |------------------------------>|Tester |
| | | |
| |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>| |
| | Backup Egress Path | |
--------- ---------
Figure 1. IGP Route Convergence Test Topology
for Local Changes
Poretsky, Imhoff [Page 2]
INTERNET-DRAFT Benchmarking Methodology for June 2003
IGP Data Plane Route Convergence
Figure 1 shows the test topology to measure IGP Route Convergence due
to local changes such as SONET Link Failure, PPP Session Failure, IGP
Adjacency Failure, Route Withdrawal, and Route cost change. These
test cases are described in section 4.1. These test cases provide
IGP Route Convergence times that consider the Event Detection time,
SPF Processing time, and FIB Update time. These times are measured
by observing packet loss in the data plane. Physical Links may be of
any type, such as Sonet or Ethernet, and any speed.
Figure 2 shows the test topology to measure IGP Route Convergence
time due to remote changes in the network topology. These times are
measured by observing packet loss in the data plane. Physical Links
may be of any type, such as Sonet or Ethernet, and any speed. In this
topology, the three routers are considered a System Under Test (SUT).
Application of this topology and test cases described in section 4.2
account for the impact of IGP Advertisement on Route Convergence, as
described in [1].
----- -----------
| | Preferred | |
----- |R2 |------------->| |
| |---->| | Egress Path | |
| | ----- | |
|R1 | | Tester |
| | ----- | |
| |---->| | Backup | |
----- |R3 |~~~~~~~~~~~~~>| |
^ | | Egress Path | |
| ----- -----------
| |
|--------------------------------
Ingress Traffic Path
Figure 2. IGP Route Convergence Test Topology
for Remote Changes
3.2 Test Considerations
3.2.1 IGP Selection
The test cases described in section 4 can be used for ISIS or
OSPF. The Route Convergence test methodology for both is
identical. The IGP adjacencies are established on the Preferred
Egress Path and Backup Egress Path.
3.2.2 BGP Configuration
The obtained results for IGP Route Convergence may vary if
BGP routes are installed. For results similar to those that
would be observed in an operational network it is recommended
that a BGP session be established on the Ingress Traffic Path
with routes installed.
Poretsky, Imhoff [Page 3]
INTERNET-DRAFT Benchmarking Methodology for June 2003
IGP Data Plane Route Convergence
3.2.3 IGP Route Scaling
The number of IGP routes will impact the measured IGP Route
Convergence because convergence for the entire IGP route table is
measured. For results similar to those that would be observed in
an operational network it is recommended that the number of
installed routes closely approximate that for routers in the
network.
3.2.4 BGP Route Scaling
The number of installed BGP routes may impact the IGP Convergence
time. For results similar to those that would be observed in an
operational Network it is recommended that the number of installed
routes closely approximate that for routers in the network.
3.2.5 Timers
There are some timers that will impact the measured IGP Convergence
time. The following timers should be configured to the minimum value
prior to beginning execution of the test cases:
SONET Failure Indication Delay
IGP Hello Timer
IGP Dead-Interval
LSA Generation Delay
LSA Flood Packet Pacing
LSA Retransmission Packet Pacing
SPF Delay
4. Test Cases
4.1 Local Events
The test cases in this section use the test topology shown in
Figure 1.
4.1.1 Convergence Due to Local SONET Link Failure
Objective
To obtain the IGP Route Convergence due to a Local SONET Link
failure event.
Procedure
1. Advertise IGP routes from Tester to DUT on Ingress Traffic
Path.
2. Advertise matching IGP routes from Tester to DUT on
Preferred Egress Path and Backup Egress Path. Set the cost
of the routes so that the IGP routes along the Preferred
Egress Path is the preferred next-hop.
3. Send traffic at maximum forwarding rate to destinations
matching all IGP routes from Tester to DUT on Ingress Traffic
Path.
4. Verify traffic routed over Preferred Egress Path.
5. Remove SONET on Tester Interface connected to Preferred Egress
Path.
6. Measure Peak-to-Peak Convergence Time [2] as DUT detects
the link down event and converges all IGP routes and
traffic over the Backup Egress Path.
Poretsky, Imhoff [Page 4]
INTERNET-DRAFT Benchmarking Methodology for June 2003
IGP Data Plane Route Convergence
Results
The measured IGP Convergence time is influenced by the Local
SONET indication, SPF delay, SPF Holdtime, SPF Execution
Time, Tree Build Time, and Hardware Update Time.
4.1.2 Convergence Due to PPP Session Failure
Objective
To obtain the IGP Route Convergence due to a Local PPP Session
failure event.
Procedure
1. Advertise IGP routes from Tester to DUT on Ingress Traffic
Path.
2. Advertise matching IGP routes from Tester to DUT on
Preferred Egress Path and Backup Egress Path. Set the cost
of the routes so that the IGP routes along the Preferred
Egress Path is the preferred next-hop.
3. Send traffic at maximum forwarding rate to destinations
matching all IGP routes from Tester to DUT on Ingress
Traffic Path.
4. Verify traffic routed over Preferred Egress Path.
5. Remove PPP session from Tester Interface connected to
Preferred Egress Path.
6. Measure Peak-to-Peak Convergence Time as DUT detects the
PPP session down event and converges all IGP routes and
traffic over the Backup Egress Path.
Results
The measured IGP Convergence time is influenced by the Local
PPP failure indication, SPF delay, SPF Holdtime, SPF Execution
Time, Tree Build Time, and Hardware Update Time.
4.1.3 Convergence Due to IGP Adjacency Failure
Objective
To obtain the IGP Route Convergence due to a Local IGP Adjacency
failure event.
Procedure
1. Advertise IGP routes from Tester to DUT on Ingress Traffic
Path.
2. Advertise matching IGP routes from Tester to DUT on
Preferred Egress Path and Backup Egress Path. Set the cost
of the routes so that the IGP routes along the Preferred
Egress Path is the preferred next-hop.
3. Send traffic at maximum forwarding rate to destinations
matching all IGP routes from Tester to DUT on Ingress
Traffic Path.
4. Verify traffic routed over Preferred Egress Path.
5. Remove IGP adjacency from Tester interface connected to
Preferred Egress Path.
Poretsky, Imhoff [Page 5]
INTERNET-DRAFT Benchmarking Methodology for June 2003
IGP Data Plane Route Convergence
6. Measure Peak-to-Peak Convergence Time as DUT detects the
IGP session failure event and converges all IGP routes and
traffic over the Backup Egress Path.
Results
The measured IGP Convergence time is influenced by the IGP
Hello Interval, IGP Dead Interval, SPF delay, SPF Holdtime,
SPF Execution Time, Tree Build Time, and Hardware Update
Time.
4.1.4 Convergence Due to Route Withdrawal
Objective
To obtain the IGP Route Convergence due to Route Withdrawal.
Procedure
1. Advertise IGP routes from Tester to DUT on Ingress Traffic
Path.
2. Advertise matching IGP routes from Tester to DUT on
Preferred Egress Path and Backup Egress Path. Set the cost
of the routes so that the IGP routes along the Preferred
Egress Path is the preferred next-hop.
3. Send traffic at maximum forwarding rate to destinations
matching all IGP routes from Tester to DUT on Ingress
Traffic Path.
4. Verify traffic routed over Preferred Egress Path.
5. Tester withdraws all IGP routes from DUT's Local Interface
on Preferred Egress Path.
6. Measure Peak-to-Peak Convergence Time as DUT processes the
route withdrawal event and converges all IGP routes and
traffic over the Backup Egress Path.
Results
The measured IGP Convergence time is the SPF Processing and FIB
Update time as influenced by the SPF delay, SPF Holdtime,
SPF Execution Time, Tree Build Time, and Hardware Update Time.
4.1.5 Convergence Due to Cost Change
Objective
To obtain the IGP Route Convergence due to route cost change.
Procedure
1. Advertise IGP routes from Tester to DUT on Ingress Traffic
Path.
2. Advertise matching IGP routes from Tester to DUT on
Preferred Egress Path and Backup Egress Path. Set the cost
of the routes so that the IGP routes along the Preferred
Egress Path is the preferred next-hop.
3. Send traffic at maximum forwarding rate to destinations
matching all IGP routes from Tester to DUT on Ingress
Traffic Path.
Poretsky, Imhoff [Page 6]
INTERNET-DRAFT Benchmarking Methodology for June 2003
IGP Data Plane Route Convergence
4. Verify traffic routed over Preferred Egress Path.
5. Tester increases cost for all IGP routes at DUT's Local
Interface on Preferred Egress Path so that Backup Egress
Path have lower cost and becomes preferred path.
6. Measure Reroute Convergence Time [2] as DUT detects the
cost change event and converges all IGP routes and
traffic over the Backup Egress Path.
Results
The measured IGP Convergence time is the SPF Processing and FIB
Update time as influenced by the SPF delay, SPF Holdtime,
SPF Execution Time, Tree Build Time, and Hardware Update Time.
There should be no packet loss for this case.
4.2 Remote Events
The test cases in this section use the test topology shown in
Figure 2.
4.2.1 Convergence Due to Remote SONET Link Failure
Objective
To obtain the IGP Route Convergence due to a Remote
SONET Link failure event.
Procedure
1. Advertise IGP routes from Tester to DUT on Ingress Traffic
Path.
2. Advertise matching IGP routes from Tester to DUT on
Preferred Egress Path and Backup Egress Path. Set the cost
of the routes so that the IGP routes along the Preferred
Egress Path is the preferred next-hop.
3. Send traffic at maximum forwarding rate to destinations
matching all IGP routes from Tester to DUT on Ingress
Traffic Path.
4. Verify traffic routed over Preferred Egress Path.
5. Remove SONET on Neighbor Interface connected to
Preferred Egress Path.
6. Measure Peak-to-Peak Convergence time as DUT detects the
link down event and converges all IGP routes and traffic
over the Backup Egress Path.
Results
The measured IGP Convergence time is influenced by the
SONET failure indication, LSA/LSP Flood Packet Pacing,
LSA/LSP Retransmission Packet Pacing, LSA/LSP Generation
time, SPF delay, SPF Holdtime, SPF Execution Time, Tree
Build Time, and Hardware Update Time.
Poretsky, Imhoff [Page 7]
INTERNET-DRAFT Benchmarking Methodology for June 2003
IGP Data Plane Route Convergence
5. Measuring Convergence Times
5.1 Measuring Full Convergence Time
Figure 3 shows a graph model of Convergence Time as measured
from the data plane. Refer to [2] for definitions of the terms
used. IGP Route Convergence Time is the amount of time for the
Forwarding Rate to begin its downward slope upon occurrence of
a network event and then fully recover to the Maximum
Forwarding Rate.
Forwarding Rate versus Time
Time=Recovery Time=Network Event Time = 0sec
Maximum ^ ^ ^
Forwarding Rate--> ----\ /-----------
\ /<----Route Convergence
Route Convergence------->\ / Event Slope
Recovery Slope \_______/<------100% Packet Loss
X-axis = Time
Y-axis = Forwarding Rate
Figure 3. Convergence Graph
Maximum forwarding rate at a fixed packet size without packet
loss is required for accurate measurement. The test duration
must be greater than the convergence time. Full Convergence
Time is obtained directly from the graph in Figure 3 using
equation 1.
(eq 1) Convergence Time(Full)=Time(Recovery)-Time(Network Event).
Given a known constant rate of offered load in units packet per
second (pps), the Average Convergence Time can be obtained
using equation 2 or equation 3.
(eq 2) Convergence Time(Average)=Number Packets Lost/pps(Offered)
(eq 3) Convergence Time(Average)= (Number Packets Offered -
Number of Packets Received)/pps(Offered)
As discussed in [1], Full Convergence Time is the more accurate
measurement. Average Convergence Time does not account for the
angle of the Route Convergence Recovery Slope, so Full Convergence
Time > Average Convergence Time. Ideally, the Recovery Slope has
no angle so that it is vertical and Average Convergence Time =
Full Convergence Time.
5.2 Measuring Impact of Components for Convergence
The factors for IGP Route Convergence Time are provided in [1].
The results of the test cases in section 4 above can be used to
calculate the impact each factor has on the Convergence
Poretsky, Imhoff [Page 8]
INTERNET-DRAFT Benchmarking Methodology for June 2003
IGP Data Plane Route Convergence
results, as follow:
SPF Processing and FIB Update time = Result (4.1.4)
SONET failure indication time = Result(4.1.1) - Result(4.1.4)
PPP failure indication time = Result(4.1.2) - Result(4.1.4)
IGP failure indication time = Result(4.1.3) - Result(4.1.4)
IGP Advertisement time = Result(4.1.1) - Result(4.2.1)
6. Security Considerations
Documents of this type do not directly effect the security of
the Internet or of corporate networks as long as benchmarking
is not performed on devices or systems connected to operating
networks.
7. Acknowledgements
Thanks to Jayant Kulkarni for doing as most Test Engineers
do - working beyond the call of duty to help advance
technology. Especially thanks to the many Network Engineers
and Network Architects at the Service Providers who are always
eager to discuss Route Convergence.
8. References
[1] Poretsky, S., "Benchmarking Applicability for IGP
Convergence", draft-ietf-bmwg-igp-dataplane-conv-app-00, work
in progress, June 2003.
[2] Poretsky, S., "Benchmarking Terminology for IGP Convergence",
draft-ietf-bmwg-igp-dataplane-conv-term-00, work in progress,
June 2003.
[3] Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and Dual
Environments", RFC 1195, December 1990.
[4] Moy, J., "OSPF Version 2", RFC 2328, IETF, April 1998.
9. Author's Address
Scott Poretsky
Avici Systems, Inc.
101 Billerica Avenue
N. Billerica, MA 01862
USA
Phone: + 1 978 964 2287
EMail: sporetsky@avici.com
Poretsky, Imhoff [Page 9]
INTERNET-DRAFT Benchmarking Methodology for June 2003
IGP Data Plane Route Convergence
Brent Imhoff
WilTel Communications
3180 Rider Trail South
Bridgeton, MO 63045
USA
Phone: +1 314 595 6853
EMail: brent.imhoff@wcg.com
10. Full Copyright Statement
Copyright (C) The Internet Society (1998). All Rights
Reserved.
This document and translations of it may be copied and
furnished to others, and derivative works that comment on or
otherwise explain it or assist in its implementation may be
prepared, copied, published and distributed, in whole or in
part, without restriction of any kind, provided that the above
copyright notice and this paragraph are included on all such
copies and derivative works. However, this document itself may
not be modified in any way, such as by removing the copyright
notice or references to the Internet Society or other Internet
organizations, except as needed for the purpose of developing
Internet standards in which case the procedures for copyrights
defined in the Internet Standards process must be followed, or
as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will
not be revoked by the Internet Society or its successors or
assigns. This document and the information contained herein is
provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE
INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY
THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY
RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
Poretsky, Imhoff [Page 10]
