Network Working Group
   INTERNET-DRAFT
   Expires in: July 2004
                                                   Scott Poretsky
                                                   Quarry Technologies

                                                   Brent Imhoff
                                                   Wiltel Communications

                                                   January 2004

                        Terminology for Benchmarking
                      IGP Data Plane Route Convergence

                <draft-ietf-bmwg-igp-dataplane-conv-term-02.txt>

   Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force  (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time.  It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   Table of Contents

     1. Introduction .................................................2
     2. Existing definitions .........................................2
     3. Term definitions..............................................3
        3.1 Convergence Event.........................................3
        3.2 Network Convergence.......................................3
        3.3 Route Convergence.........................................4
        3.4 Full Convergence..........................................4
        3.5 Convergence Packet Loss...................................5
        3.6 Convergence Event Instant.................................5
        3.7 Convergence Recovery Instant..............................6
        3.8 Rate-Derived Convergence Time.............................6
        3.9 Convergence Event Transition..............................7
        3.10 Convergence Recovery Transition..........................7
        3.11 Loss-Derived Convergence Time............................8
        3.12 Sustained Forwarding Convergence Time...................................9

Poretsky, Imhoff                                                                [Page 1]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

        3.13 Restoration Convergence Time.............................9
        3.14 Packet Sampling Interval.................................10
        3.15 Local Interface..........................................10
        3.16 Neighbor Interface.......................................11
        3.17 Remote Interface.........................................11
        3.18 Preferred Egress Interface...............................11
        3.19 Next-Best Egress Interface...............................12
        3.20 Stale Forwarding.........................................12
     4. Security Considerations.......................................12
     5. References....................................................13
     6. Author's Address..............................................13
     7. Full Copyright Statement......................................14

   1. Introduction
   This draft describes the terminology for benchmarking IGP Route
   Convergence.  The motivation and applicability for this
   benchmarking is provided in [1].  The methodology to be used for
   this benchmarking is described in [2].  The methodology and
   terminology to be used for benchmarking route convergence can be
   applied to any link-state IGP such as ISIS [3] and OSPF [4].  The
   data plane is measured to obtain black-box (externally observable)
   convergence benchmarking metrics.  The purpose of this document is
   to introduce new terms required to complete execution of the IGP
   Route Convergence Methodology [2].

   An example of Route Convergence as observed and measured from the
   data plane is shown in Figure 1.  The graph in Figure 1 shows
   Forwarding Rate versus Time.  Time 0 on the X-axis is on the far
   right of the graph.  The components of the graph and metrics are
   defined in the Term Definitions section of this document.

                            Recovery  Convergence Event   Time = 0sec
        Maximum                ^              ^             ^
        Forwarding Rate--> ----\    Packet   /---------------
                                \    Loss   /<----Convergence
              Convergence------->\         /      Event Transition
        Recovery Transition       \       /
                                   \_____/<------100% Packet Loss

        X-axis = Time
        Y-axis = Forwarding Rate

                        Figure 1. Convergence Graph

   2.  Existing definitions
   For the sake of clarity and continuity this RFC adopts the template
   for definitions set out in Section 2 of RFC 1242.  Definitions are
   indexed and grouped together in sections for ease of reference.
   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC 2119.

Poretsky, Imhoff                                                                [Page 2]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

   3. Term Definitions

   3.1 Convergence Event

        Definition:
        The occurrence of a planned or unplanned action in the network
        that results in a change to an entry in the route table.

        Discussion:
        Convergence Events include link loss, routing protocol session
        loss, router failure, and better next-hop.

        Measurement Units:
        N/A

        Issues:
        None

        See Also:
        Convergence Packet Loss
        Convergence Event Instant

   3.2 Network Convergence

        Definition:
        The completion of updating of all routing tables, including the
        FIB, in all routers throughout the network.

        Discussion:
        Network Convergence can be approximated to the sum of Route
        Convergence for all routers in the network.  Network Convergence
        can be determined by recovery of the forwarding rate to equal
        the offer load, no stale forwarding, and no blenders[5][6].

        Measurement Units:
        N/A

        Issues:
        None

        See Also:
        Route Convergence
        Stale Forwarding









Poretsky, Imhoff                                                                [Page 3]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence


   3.3 Route Convergence

        Definition:
        Recovery from a Convergence Event indicated by the DUT
        forwarding rate equal to the offered load.

        Discussion:
        Route Convergence is the action of all components of the router
        being updated with the most recent route change(s) including the
        RIB and FIB, along with software and hardware tables. Route
        Convergence can be observed externally by the rerouting of data
        Traffic to a new egress interface.

        Measurement Units:
        N/A

        Issues:
        None

        See Also:
        Network Convergence
        Full Convergence
        Convergence Event

   3.4 Full Convergence
        Definition:
        Route Convergence for an entire FIB.

        Discussion:
        When benchmarking convergence it is useful to measure
        the time to converge an entire route table.  For example,

        a Convergence Event can be produced for an OSPF table of 5000
        routes so that the time to converge routes 1 through 5000
        is measured.

        Measurement Units:
        N/A

        Issues:
        None

        See Also:
        Network Convergence
        Route Convergence
        Convergence Event





Poretsky, Imhoff                                                                [Page 4]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

   3.5 Convergence Packet Loss

        Definition:
        The amount of packet loss produced by a Convergence Event
        until Route Convergence occurs.

        Discussion:
        Packet loss can be observed as a reduction of forwarded
        traffic from the maximum forwarding rate.

        Measurement Units:
        number of packets

        Issues:
        None

        See Also:
        Route Convergence
        Convergence Event
        Rate-Derived Convergence Time
        Loss-Derived Convergence Time

   3.6 Convergence Event Instant

        Definition:
        The time instant that a Convergence Event occurs.

        Discussion:
        Convergence Event Instant is observable from the data
        plane as the precise time that the device under test begins
        to exhibit packet loss.

        Measurement Units:
        hh:mm:ss:uuu

        Issues:
        None

        See Also:
        Convergence Event
        Convergence Packet Loss
        Convergence Recovery Instant










Poretsky, Imhoff                                                                [Page 5]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

   3.7 Convergence Recovery Instant
        Definition:
        The time instant that Full Convergence is measured
        and maintained for at least an additional five seconds.

        Discussion:
        Convergence Recovery Instant is measurable from the data
        plane as the precise time that the device under test
        achieves Full Convergence.

        Measurement Units:
        hh:mm:ss:uuu

        Issues:
        None

        See Also:
        Convergence Packet Loss
        Convergence Event Instant

   3.8 Rate-Derived Convergence Time

        Definition:
        The amount of time for Convergence Packet Loss to
        persist upon occurrence of a Convergence Event until
        occurrence of Route Convergence.

        Discussion:

        Rate-Derived Convergence Time can be measured as the time
        difference from the Convergence Event Instant to the
        Convergence Recovery Instant, as shown with Equation 1.

        (eq 1)  Rate-Derived Convergence Time =
                Convergence Recovery Instant - Convergence Event Instant.

        Rate-Derived Convergence Time should be measured at the maximum
        forwarding rate.  Failure to achieve Full Convergence results in
        a Rate-Derived Convergence Time benchmark of infinity.

        Measurement Units:
        seconds/milliseconds

        Issues:
        None

        See Also:
        Convergence Packet Loss
        Convergence Recovery Instant
        Convergence Event Instant
        Full Convergence

Poretsky, Imhoff                                                                [Page 6]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

   3.9 Convergence Event Transition

        Definition:
        The characteristic of a router in which forwarding rate
        gradually reduces to zero after a Convergence Event.

        Discussion:
        The Convergence Event Transition is best observed for
        Full Convergence.

        Measurement Units:
        seconds/milliseconds

        Issues:
        None

        See Also:
        Convergence Event
        Rate-Derived Convergence Time
        Convergence Packet Loss
        Convergence Recovery Transition

   3.10 Convergence Recovery Transition

        Definition:
        The characteristic of a router in which forwarding rate
        gradually increases to equal the offered load.

        Discussion:
        The Convergence Recovery Transition is best observed for
        Full Convergence.

        Measurement Units:
        seconds/milliseconds

        Issues:
        None

        See Also:
        Full Convergence
        Rate-Derived Convergence Time
        Convergence Packet Loss
        Convergence Event Transition









Poretsky, Imhoff                                                                [Page 7]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

   3.11 Loss-Derived Convergence Time

        Definition:
        The amount of time it takes for Route Convergence to
        to be achieved as calculated from the Convergence Packet
        Loss.

        Discussion:
        Loss-Derived Convergence Time can be calculated from
        Convergence Packet Loss that occurs due to a Convergence Event
        and Route Convergence, as shown with Equation 2.

        (eq 2) Loss-Derived Convergence Time =
                Convergence Packets Loss / Forwarding Rate

                NOTE: Units for this measurement are
                packets / packets/second = seconds

        Measurement Units:
        seconds/milliseconds

        Issues:
        Loss-Derived Convergence time gives a better than
        actual result when converging many routes simultaneously.
        Rate-Derived Convergence Time takes the Convergence Recovery
        Transition into account, but Loss-Derived Convergence Time
        ignores the Route Convergence Recovery Transition because
        it is obtained from the measured Convergence Packet Loss.
        Ideally, the Convergence Event Transition and Convergence
        Recovery Transition are instantaneous so that the
        Rate-Derived Convergence Time = Loss-Derived Convergence Time.
        However, router implementations are less than ideal.
        For these reasons the preferred reporting benchmark for IGP
        Route Convergence is the Rate-Derived Convergence Time.
        Guidelines for reporting Loss-Derived Convergence Time are
        provided in [2].

        See Also:
        Route Convergence
        Convergence Packet Loss
        Rate-Derived Convergence Time
        Convergence Event Transition
        Convergence Recovery Transition









Poretsky, Imhoff                                                                [Page 8]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

   3.12 Sustained Forwarding Convergence Time

        Definition:
        The amount of time for Route Convergence to be achieved for
        cases in which there is no packet loss.

        Discussion:
        Sustained Forwarding Convergence Time is the IGP Route Convergence
        benchmark to be used for Convergence Events that produce
        a change in next-hop without packet loss.

        Measurement Units:
        seconds/milliseconds

        Issues:
        None

        See Also:
        Route Convergence
        Rate-Derived Convergence Time
        Loss-Derived Convergence Time

   3.13 Restoration Convergence Time

        Definition:
        The amount of time for the router under test to restore
        traffic to the original outbound port after recovery from
        a Convergence Event.

        Discussion:
        Restoration Convergence Time is the amount of time to
        Converge back to the original outbound port.  This is achieved
        by recovering from the Convergence Event, such as restoring
        the failed link.  Restoration Convergence Time is measured
        using the Rate-Derived Convergence Time calculation technique,
        as provided in Equation 1.  It is possible, but not desired
        to have the Restoration Convergence Time differ from the
        Rate-Derived Convergence Time.

        Measurement Units:
        seconds or milliseconds

        Issues:
        None

        See Also:
        Convergence Event
        Rate-Derived Convergence Time




Poretsky, Imhoff                                                                [Page 9]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

   3.14 Packet Sampling Interval
        Definition:
        The rate at which the tester (test equipment) polls to make
        measurements for arriving packet flows.

        Discussion:
        Metrics measured at the Packet Sampling Interval include
        packets received and Convergence Packet Loss.

        Measurement Units:
        seconds or milliseconds

        Issues:
        Packet Sampling Interval can influence the Convergence Graph.
        This is particularly true as implementations achieve Full
        Convergence in less than 1 second.  The Convergence Event
        Transition and Convergence Recovery Transition can become
        exaggerated when the Packet Sampling Interval is too long.
        This will produce a larger than actual Rate-Derived
        Convergence Time.  The recommended value for configuration
        of the Packet Sampling Interval is provided in [2].

        See Also:
        Convergence Packet Loss
        Convergence Event Transition
        Convergence Recovery Transition

   3.15 Local Interface
        Definition:
        An interface on the DUT.

        Discussion:
        None

        Measurement Units:
        N/A

        Issues:
        None

        See Also:
        Neighbor Interface
        Remote interface









Poretsky, Imhoff                                                                [Page 10]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

   3.16 Neighbor Interface
        Definition:
        The interface on the neighbor router or tester that is
        directly linked to the DUT's Local Interface.

        Discussion:
        None

        Measurement Units:
        N/A

        Issues:
        None

        See Also:
        Local Interface
        Remote interface

   3.17 Remote Interface
        Definition:
        An interface on a neighboring router that is not directly
        connected to any interface on the DUT.

        Discussion:
        None

        Measurement Units:
        N/A

        Issues:
        None

        See Also:
        Local interface
        Neighbor Interface

   3.18 Preferred Egress Interface
        Definition:
        The outbound interface on DUT to the preferred next-hop.

        Discussion:
        Preferred Egress Interface is the egress interface prior to
        a Convergence Event

        Measurement Units:
        N/A

        Issues:
        None

        See Also:
        Next-Best Egress Interface

Poretsky, Imhoff                                                        [Page 11]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

   3.19 Next-Best Egress Interface

        Definition:
        The outbound interface on DUT to the second-best next-hop.

        Discussion:
        Next-Best Egress Interface is the egress interface after
        a Convergence Event.

        Measurement Units:
        N/A

        Issues:
        None

        See Also:
        Preferred Egress Interface

   3.20 Stale Forwarding

        Definition:
        Forwarding of traffic to route entries that no longer exist
        or to route entries with next-hops that are no longer preferred.

        Discussion:
        Stale Forwarding can be caused by a Convergence Event and is
        also known as a "black-hole" since it may produce packet loss.
        Stale Forwarding exists until Network Convergence is achieved.

        Measurement Units:
        N/A

        Issues:
        None

        See Also:
        Network Convergence



   4. Security Considerations

        Documents of this type do not directly affect the security of
        Internet or corporate networks as long as benchmarking
        is not performed on devices or systems connected to operating
        networks.






Poretsky, Imhoff                                                                [Page 12]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence

   5. References

   [1]   Poretsky, S., "Benchmarking Applicability for IGP Data Plane
         Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-app-02,
         work in progress, January 2004.

   [2]   Poretsky, S., "Benchmarking Methodology for IGP Data Plane
         Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-meth-02,
         work in progress, January 2004.

   [3]   Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and Dual
         Environments", RFC 1195, December 1990.

   [4]   Moy, J., "OSPF Version 2", RFC 2328, IETF, April 1998.

   [5]   S. Casner, C. Alaettinoglu, and C. Kuan, "A Fine-Grained View
         of High Performance Networking", NANOG 22, May 2001.

   [6]   L. Ciavattone, A. Morton, and G. Ramachandran, "Standardized
         Active Measurements on a Tier 1 IP Backbone", IEEE Communications
         Magazine, pp90-97, June, 2003.


  6. Author's Address

        Scott Poretsky
        Quarry Technologies
        8 New England Executive Park
        Burlington, MA 01803
        USA
        Phone: + 1 781 395 5090
        EMail: sporetsky@quarrytech.com

        Brent Imhoff
        WilTel Communications
        3180 Rider Trail South
        Bridgeton, MO 63045 USA
        Phone: +1 314 595 6853
        EMail: brent.imhoff@wcg.com













Poretsky, Imhoff                                                                [Page 13]


INTERNET-DRAFT          Benchmarking Terminology for            January 2004
                      IGP Data Plane Route Convergence


   7.  Full Copyright Statement

        Copyright (C) The Internet Society (1998).  All Rights
        Reserved.

        This document and translations of it may be copied and
        furnished to others, and derivative works that comment on or
        otherwise explain it or assist in its implementation may be
        prepared, copied, published and distributed, in whole or in
        part, without restriction of any kind, provided that the above
        copyright notice and this paragraph are included on all such
          copies and derivative works.  However, this document itself may
        not be modified in any way, such as by removing the copyright
        notice or references to the Internet Society or other Internet
        organizations, except as needed for the purpose of developing
        Internet standards in which case the procedures for copyrights
        defined in the Internet Standards process must be followed, or
        as required to translate it into languages other than English.

        The limited permissions granted above are perpetual and will
        not be revoked by the Internet Society or its successors or
        assigns.  This document and the information contained herein is
        provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE
        INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY
        THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY
        RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
        FOR A PARTICULAR PURPOSE.























Poretsky, Imhoff                                                                [Page 14]


   Network Working Group
   INTERNET-DRAFT
   Expires in: July 2004
                                                Scott Poretsky
                                                Quarry Technologies

                                                Brent Imhoff
                                                Wiltel Communications

                                                January 2004

                        Benchmarking Methodology for
                      IGP Data Plane Route Convergence

        <draft-ietf-bmwg-igp-dataplane-conv-meth-02.txt>


   Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force  (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time.  It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.


   Table of Contents
     1. Introduction ...............................................2
     2. Existing definitions .......................................2
     3. Test Setup..................................................2
     3.1 Test Topologies............................................2
     3.2 Test Considerations........................................4
     3.2.1 IGP Selection............................................4
     3.2.2 BGP Configuration........................................4
     3.2.3 IGP Route Scaling........................................5
     3.2.4 Timers...................................................5
     3.2.5 Convergence Time Metrics.................................5
     3.2.6 Offered Load.............................................5
     3.2.7 Interface Types..........................................5
     3.3 Reporting Format...........................................6
     4. Test Cases..................................................6

Poretsky, Imhoff                                                                [Page 1]


INTERNET-DRAFT    Benchmarking Methodology for  January 2004
                      IGP Data Plane Route Convergence

     4.1 Convergence Due to Link Failure............................6
     4.1.1 Convergence Due to Local Interface Failure...............6
     4.1.2 Convergence Due to Neighbor Interface Failure............7
     4.1.3 Convergence Due to Remote Interface Failure..............7
     4.2 Convergence Due to PPP Session Failure.....................8
     4.3 Convergence Due to IGP Adjacency Failure...................9
     4.4 Convergence Due to Route Withdrawal........................9
     4.5 Convergence Due to Cost Change.............................10
     4.6 Convergence Due to ECMP Member Interface Failure...........10
     4.7 Convergence Due to Parallel Link Interface Failure.........11
     5. Security Considerations.....................................12
     6. References..................................................12
     7. Author's Address............................................12
     8. Full Copyright Statement....................................13

   1. Introduction
   This draft describes the methodology for benchmarking IGP Route
   Convergence.  The applicability of this testing is described in
   [1] and the new terminology that it introduces is defined in [2].
   Service Providers use IGP Convergence time as a key metric of
   router design and architecture.  Customers of Service Providers
   observe convergence time by packet loss, so IGP Route Convergence
   is considered a Direct Measure of Quality (DMOQ).  The test cases
   in this document are black-box tests that emulate the network
   events that cause route convergence, as described in [1].  The
   black-box test designs benchmark the data plane accounting for
   all of the factors contributing to convergence time, as discussed
   in [1].  The methodology (and terminology) for benchmarking route
   convergence can be applied to any link-state  IGP such as ISIS [3]
   and OSPF [4].

   2.  Existing definitions

   For the sake of clarity and continuity this RFC adopts the template
   for definitions set out in Section 2 of RFC 1242.  Definitions are
   indexed and grouped together in sections for ease of reference.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC 2119.

   3.  Test Setup
   3.1 Test Topologies

   Figure 1 shows the test topology to measure IGP Route Convergence due
   to local Convergence Events such as SONET Link Failure, PPP Session
   Failure, IGP  Adjacency Failure, Route Withdrawal, and route cost
   change.  These test cases discussed in section 4 provide route
   convergence times that account for the Event Detection time, SPF
   Processing time, and FIB Update time.  These times are measured
   by observing packet loss in the data plane.

Poretsky, Imhoff                                                                [Page 2]


INTERNET-DRAFT    Benchmarking Methodology for  January 2004
                      IGP Data Plane Route Convergence


        ---------       Ingress Interface       ---------
        |       |<------------------------------|       |
        |       |                               |       |
        |       | Preferred Egress Interface    |       |
        |  DUT  |------------------------------>|Tester |
        |       |                               |       |
        |       |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>|       |
        |       | Next-Best Egress Interface    |       |
        ---------                               ---------

        Figure 1.  IGP Route Convergence Test Topology for Local Changes

   Figure 2 shows the test topology to measure IGP Route Convergence
   time due to remote changes in the network topology.  These times are
   measured by observing packet loss in the data plane.  In this
   topology the three routers are considered a System Under Test (SUT).
   NOTE: All routers in the SUT must be the same model and identically  configured.


                -----                       -----------
                |   |   Preferred           |         |
        -----   |R2 |---------------------->|         |
        |   |-->|   | Egress Interface      |         |
        |   |   -----                       |         |
        |R1 |                               |  Tester |
        |   |   -----                       |         |
        |   |-->|   |   Next-Best           |         |
        -----   |R3 |~~~~~~~~~~~~~~~~~~~~~~>|         |
          ^     |   |   Egress Interface    |         |
          |     -----                       -----------
          |                                     |
          |--------------------------------------
                Ingress Interface

        Figure 2.  IGP Route Convergence Test Topology
                        for Remote Changes

   Figure 3 shows the test topology to measure IGP Route Convergence
   time with members of an ECMP Set.  These times are measured by
   observing packet loss in the data plane.  In this topology, the DUT
   is configured with each Egress interface as a member of an ECMP set
   and the Tester emulates multiple next-hop routers (emulates one
   router for each member).







Poretsky, Imhoff                                                                [Page 3]


INTERNET-DRAFT     Benchmarking Methodology for January 2004
                      IGP Data Plane Route Convergence

        ---------       Ingress Interface         ---------
        |       |<--------------------------------|       |
        |       |                                 |       |
        |       |       ECMP Set Interface 1      |       |
        |  DUT  |-------------------------------->| Tester|
        |       |               .                 |       |
        |       |               .                 |       |
        |       |               .                 |       |
        |       |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>|       |
        |       |       ECMP Set Interface N      |       |
        ---------                                 ---------

        Figure 3.  IGP Route Convergence Test Topology
                        for ECMP Convergence

   Figure 4 shows the test topology to measure IGP Route Convergence
   time with members of a Parallel Link.  These times are measured by
   observing packet loss in the data plane.  In this topology, the DUT
   is configured with each Egress interface as a member of a Parallel
   Link and the Tester emulates the single next-hop router.

        ---------       Ingress Interface         ---------
        |       |<--------------------------------|       |
        |       |                                 |       |
        |       |       Parallel Link Interface 1 |       |
        |  DUT  |-------------------------------->| Tester|
        |       |               .                 |       |
        |       |               .                 |       |
        |       |               .                 |       |
        |       |~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~>|       |
        |       |       Parallel Link Interface N |       |
        ---------                                 ---------

        Figure 4.  IGP Route Convergence Test Topology
                      for Parallel Link Convergence

   3.2 Test Considerations

   3.2.1 IGP Selection
   The test cases described in section 4 can be used for ISIS or
   OSPF.  The Route Convergence test methodology for both is
   identical.  The IGP adjacencies are established on the Preferred
   Egress Interface and Next-Best Egress Interface.

   3.2.2 BGP Configuration
   The obtained results for IGP Route Convergence may vary if
   BGP routes are installed.  It is recommended that the IGP
   Convergence times be benchmarked without BGP routes installed.




Poretsky, Imhoff                                                                [Page 4]


INTERNET-DRAFT     Benchmarking Methodology for January 2004
                      IGP Data Plane Route Convergence

   3.2.3 IGP Route Scaling
   The number of IGP routes will impact the measured IGP Route
   Convergence because convergence for the entire IGP route table is
   measured.   For results similar to those that would be observed in
   an operational network it is recommended that the number of
   installed routes closely approximate that for routers in the
   network.

   3.2.4 Timers
   There are some timers that will impact the measured IGP Convergence
   time. The following timers should be configured to the minimum value
   prior to beginning execution of the test cases:

        Timer                                   Recommended Value
        -----                                   -----------------
        SONET Failure Indication Delay          <10milliseconds
        IGP Hello Timer                         1 second
        IGP Dead-Interval                       3 seconds
        LSA Generation Delay                    0
        LSA Flood Packet Pacing                 0
        LSA Retransmission Packet Pacing        0
        SPF Delay                               0

   3.2.5 Convergence Time Metrics
   The recommended value for the Packet Sampling Interval [2] is
   100 milliseconds.  Rate-Derived Convergence Time [2] is the
   preferred benchmark for IGP Route Convergence.  This benchmark
   must always be reported when the
   Packet Sampling Interval [2] <= 100 milliseconds.
   If the test equipment does not permit the Packet Sampling
   Interval to be set as low as 100 msec, then both the
   Rate-Derived Convergence Time and Loss-Derived Convergence
   Time [2] must be reported.

   3.2.6 Offered Load
   An offered Load of maximum forwarding rate at a fixed packet size
   is recommended for accurate measurement.  The duration of offered
   load must be greater than the convergence time.

   3.2.7 Interface Types
   All test cases in this methodology document may be executed with
   any interface type.  SONET is recommended and specifically
   mentioned in the procedures because it can be configured to have
   no or negligible affect on the measured convergence time.
   Ethernet (10Mb, 100Mb, 1Gb, and 10Gb) is not preferred since
   broadcast media are unable to detect loss of host and rely upon
   IGP Hellos to detect session loss.





Poretsky, Imhoff                                                                [Page 5]


INTERNET-DRAFT     Benchmarking Methodology for January 2004
                      IGP Data Plane Route Convergence

   3.3 Reporting Format
   For each test case, it is recommended that the following reporting
   format be completed:

        Parameter                                       Units
        ---------                                       -----
        IGP                                             (ISIS or OSPF)
        Interface Type                                  (GigE, POS, ATM, etc.)
        Packet Size                                     bytes
        IGP Routes                                      number of IGP routes
        Packet Sampling Interval                        seconds or milliseconds
        IGP Timer Values
                SONET Failure Indication Delay          seconds or milliseconds
                IGP Hello Timer                         seconds or milliseconds
                IGP Dead-Interval                       seconds or milliseconds
                LSA Generation Delay                    seconds or milliseconds
                LSA Flood Packet Pacing                 seconds or milliseconds
                LSA Retransmission Packet Pacing        seconds or milliseconds
                SPF Delay                               seconds or milliseconds
        Benchmarks
                Rate-Derived Convergence Time           seconds or milliseconds
                Loss-Derived Convergence Time           seconds or milliseconds
                Restoration Convergence Time            seconds or milliseconds

   4. Test Cases
   4.1 Convergence Due to Link Failure
   4.1.1 Convergence Due to Local Interface Failure
        Objective
        To obtain the IGP Route Convergence due to a local link
        failure event at the DUT's Local Interface.

        Procedure
        1. Advertise matching IGP routes from Tester to DUT on
           Preferred Egress Interface [2] and Next-Best Egress Interface
           [2] using the topology shown in Figure 1.  Set the cost of the
           routes so that the Preferred Egress Interface is the preferred
           next-hop.
        2. Send traffic at maximum forwarding rate to destinations
           matching all IGP routes from Tester to DUT on Ingress Interface
           [2].
        3. Verify traffic routed over Preferred Egress Interface.
        4. Remove SONET on DUT's Local Interface [2] by performing an
           administrative shutdown of the interface.
        5. Measure Rate-Derived Convergence Time [2] as DUT detects the
           link down event and converges all IGP routes and traffic over
           the Next-Best Egress Interface.
        6. Restore SONET on DUT's Local Interface by administratively
           enabling the interface.
        7. Measure Restoration Convergence Time [2] as DUT detects the link
           up event and converges all IGP routes and traffic back to the
           Preferred Egress Interface.

Poretsky, Imhoff                                                                [Page 6]


INTERNET-DRAFT     Benchmarking Methodology for January 2004
                      IGP Data Plane Route Convergence
        Results
        The measured IGP Convergence time is influenced by the Local
        SONET indication, SPF delay, SPF Holdtime, SPF Execution
        Time, Tree Build Time, and Hardware Update Time.

   4.1.2 Convergence Due to Neighbor Interface Failure
        Objective
        To obtain the IGP Route Convergence due to a local link
        failure event at the Tester's Neighbor Interface.

        Procedure
        1. Advertise matching IGP routes from Tester to DUT on
           Preferred Egress Interface [2] and Next-Best Egress Interface
           [2] using the topology shown in Figure 1.  Set the cost of
           the routes so that the Preferred Egress Interface is the
           preferred next-hop.
        2. Send traffic at maximum forwarding rate to destinations
           matching all IGP routes from Tester to DUT on Ingress
           Interface [2].
        3. Verify traffic routed over Preferred Egress Interface.
        4. Remove SONET on Tester's Neighbor Interface [2] connected to
           DUT' s Preferred Egress Interface.
        5. Measure Rate-Derived Convergence Time [2] as DUT detects the
           link down event and converges all IGP routes and traffic over
           the Next-Best Egress Interface.
        6. Restore SONET on Tester's Neighbor Interface connected to
           DUT's Preferred Egress Interface.
        7. Measure Restoration Convergence Time [2] as DUT detects the
           link up event and converges all IGP routes and traffic back to
           the Preferred Egress Interface.

        Results
        The measured IGP Convergence time is influenced by the Local
        SONET indication, SPF delay, SPF Holdtime, SPF Execution
        Time, Tree Build Time, and Hardware Update Time.

   4.1.3 Convergence Due to Remote Interface Failure
      Objective
        To obtain the IGP Route Convergence due to a Remote
        Interface failure event.

        Procedure
        1. Advertise matching IGP routes from Tester to SUT on
          Preferred Egress Interface [2] and Next-Best Egress Interface
           [2] using the topology shown in Figure 2.  Set the cost of the
           routes so that the Preferred Egress Interface is the preferred
           next-hop.
        2. Send traffic at maximum forwarding rate to destinations
           matching all IGP routes from Tester to DUT on Ingress Interface
           [2].
        3. Verify traffic is routed over Preferred Egress Interface.
        4. Remove SONET on Tester's Neighbor Interface [2] connected to
           SUT' s Preferred Egress Interface.

Poretsky, Imhoff                                                                [Page 7]


INTERNET-DRAFT     Benchmarking Methodology for January 2004
                      IGP Data Plane Route Convergence

        5. Measure Rate-Derived Convergence Time [2] as SUT detects
           the link down event and converges all IGP routes and traffic
           over the Next-Best Egress Interface.
        6. Restore SONET on Tester's Neighbor Interface connected to
           SUT's Preferred Egress Interface.
        7. Measure Restoration Convergence Time [2] as SUT detects the
           link up event and converges all IGP routes and traffic over
           the Preferred Egress Interface.

        Results
        The measured IGP Convergence time is influenced by the
        SONET failure indication, LSA/LSP Flood Packet Pacing,
        LSA/LSP Retransmission Packet Pacing, LSA/LSP Generation
        time, SPF delay, SPF Holdtime, SPF Execution Time, Tree
        Build Time, and Hardware Update Time.  The additional
        convergence time contributed by LSP Propagation can be
        obtained by subtracting the Rate-Derived Convergence Time
        measured in 4.1.2 (Convergence Due to Neighbor Interface
        Failure) from the Rate-Derived Convergence Time measured in
        this test case.

   4.2 Convergence Due to PPP Session Failure
        Objective
        To obtain the IGP Route Convergence due to a Local PPP Session
        failure event.

        Procedure
        1. Advertise matching IGP routes from Tester to DUT on
           Preferred Egress Interface [2] and Next-Best Egress Interface
           [2] using the topology shown in Figure 1.  Set the cost of
           the routes so that the IGP routes along the Preferred Egress
           Interface is the preferred next-hop.
        2. Send traffic at maximum forwarding rate to destinations
           matching all IGP routes from Tester to DUT on Ingress
           Interface [2].
        3. Verify traffic routed over Preferred Egress Interface.
        4. Remove PPP session from Tester's Neighbor Interface [2]
           connected to Preferred Egress Interface.
        5. Measure Rate-Derived Convergence Time [2] as DUT detects the
           PPP session down event and converges all IGP routes and
           traffic over the Next-Best Egress Interface.
        6. Restore PPP session on DUT's Preferred Egress Interface.
        7. Measure Restoration Convergence Time [2] as DUT detects the
           session up event and converges all IGP routes and traffic over
           the Preferred Egress Interface.

        Results
        The measured IGP Convergence time is influenced by the PPP
        failure indication, SPF delay, SPF Holdtime, SPF Execution
        Time, Tree Build Time, and Hardware Update Time.


Poretsky, Imhoff                                                                [Page 8]


INTERNET-DRAFT     Benchmarking Methodology for January 2004
                      IGP Data Plane Route Convergence

   4.3 Convergence Due to IGP Adjacency Failure

        Objective
        To obtain the IGP Route Convergence due to a Local IGP Adjacency
        failure event.

        Procedure
        1. Advertise matching IGP routes from Tester to DUT on
           Preferred Egress Interface [2] and Next-Best Egress Interface
           [2] using the topology shown in Figure 1.  Set the cost of
           the routes so that the Preferred Egress Interface is the
           preferred next-hop.
        2. Send traffic at maximum forwarding rate to destinations
           matching all IGP routes from Tester to DUT on Ingress
           Interface [2].
        3. Verify traffic routed over Preferred Egress Interface.
        4. Remove IGP adjacency from Tester's Neighbor Interface [2]
           connected to Preferred Egress Interface.
        5. Measure Rate-Derived Convergence Time [2] as DUT detects the
           IGP session failure event and converges all IGP routes and
           traffic over the Next-Best Egress Interface.
        6. Restore IGP session on DUT's Preferred Egress Interface.
        7. Measure Restoration Convergence Time [2] as DUT detects the
           session up event and converges all IGP routes and traffic over
           the Preferred Egress Interface.

        Results
        The measured IGP Convergence time is influenced by the IGP
        Hello Interval, IGP Dead Interval, SPF delay, SPF Holdtime,
        SPF Execution Time, Tree Build Time, and Hardware Update
        Time.

  4.4 Convergence Due to Route Withdrawal

        Objective
        To obtain the IGP Route Convergence due to Route Withdrawal.

        Procedure
        1. Advertise matching IGP routes from Tester to DUT on
         Preferred Egress Interface [2] and Next-Best Egress Interface
           [2] using the topology shown in Figure 1.  Set the cost of
           the routes so that the Preferred Egress Interface is the
           preferred next-hop.
        2. Send traffic at maximum forwarding rate to destinations
           matching all IGP routes from Tester to DUT on Ingress
           Interface [2].
        3. Verify traffic routed over Preferred Egress Interface.
        4. Tester withdraws all IGP routes from DUT's Local Interface
           on Preferred Egress Interface.



Poretsky, Imhoff                                                                [Page 9]


INTERNET-DRAFT     Benchmarking Methodology for January 2004
                      IGP Data Plane Route Convergence

        6. Re-advertise IGP routes to DUT's Preferred Egress Interface.
        7. Measure Restoration Convergence Time [2] as DUT converges all
           IGP routes and traffic over the Preferred Egress Interface.

        Results
        The measured IGP Convergence time is the SPF Processing and FIB
        Update time as influenced by the SPF delay, SPF Holdtime,
        SPF Execution Time, Tree Build Time, and Hardware Update Time.

   4.5 Convergence Due to Cost Change

        Objective
        To obtain the IGP Route Convergence due to route cost change.

        Procedure
        1. Advertise matching IGP routes from Tester to DUT on
           Preferred Egress Interface [2] and Next-Best Egress Interface
           [2] using the topology shown in Figure 1.  Set the cost of
           the routes so that the Preferred Egress Interface is the
           preferred next-hop.
        2. Send traffic at maximum forwarding rate to destinations
           matching all IGP routes from Tester to DUT on Ingress
           Interface [2].
        3. Verify traffic routed over Preferred Egress Interface.
        4. Tester increases cost for all IGP routes at DUT's Preferred
           Egress Interface so that the Next-Best Egress Interface
           has lower cost and becomes preferred path.
        5. Measure Rate-Derived Convergence Time [2] as DUT detects the
           cost change event and converges all IGP routes and traffic
           over the Next-Best Egress Interface.
        6. Re-advertise IGP routes to DUT's Preferred Egress Interface
           with original lower cost metric.
        7. Measure Restoration Convergence Time [2] as DUT converges all
           IGP routes and traffic over the Preferred Egress Interface.

        Results
        There should be no measured packet loss for this case.


    4.6 Convergence Due to ECMP Member Interface Failure

        Objective
        To obtain the IGP Route Convergence due to a local link
        failure event of an ECMP Member.

        Procedure
        1. Configure ECMP Set as shown in Figure 3.
        2. Advertise matching IGP routes from Tester to DUT on
           each ECMP member.



Poretsky, Imhoff                                                                [Page 10]


INTERNET-DRAFT     Benchmarking Methodology for January 2004
                      IGP Data Plane Route Convergence

        3. Send traffic at maximum forwarding rate to destinations
           matching all IGP routes from Tester to DUT on Ingress
           Interface [2].
        4. Verify traffic routed over all members of ECMP Set.
        5. Remove SONET on Tester's Neighbor Interface [2] connected to
           one of the DUT's ECMP member interfaces.
        6. Measure Rate-Derived Convergence Time [2] as DUT detects the
           link down event and converges all IGP routes and traffic
           over the other ECMP members.
        7. Restore SONET on Tester's Neighbor Interface connected to
           DUT's ECMP member interface.
        8. Measure Restoration Convergence Time [2] as DUT detects the
           link up event and converges IGP routes and some distribution
           of traffic over the restored ECMP member.

        Results
        The measured IGP Convergence time is influenced by the Local
        SONET indication, Tree Build Time, and Hardware Update Time.

   4.7 Convergence Due to Parallel Link Interface Failure

        Objective
        To obtain the IGP Route Convergence due to a local link
        failure event for a Member of a Parallel Link.

        Procedure
        1. Configure Parallel Link as shown in Figure 4.
        2. Advertise matching IGP routes from Tester to DUT on
           each Parallel Link member.
        3. Send traffic at maximum forwarding rate to destinations
           matching all IGP routes from Tester to DUT on Ingress
           Interface [2].
        4. Verify traffic routed over all members of Parallel Link.
        5. Remove SONET on Tester's Neighbor Interface [2] connected to
           one of the DUT's Parallel Link member interfaces.
        6. Measure Rate-Derived Convergence Time [2] as DUT detects the
           link down event and converges all IGP routes and traffic over
           the other Parallel Link members.
        7. Restore SONET on Tester's Neighbor Interface connected to
           DUT's Parallel Link member interface.
        8. Measure Restoration Convergence Time [2] as DUT detects the
           link up event and converges IGP routes and some distribution
           of traffic over the restored Parallel Link member.

        Results
        The measured IGP Convergence time is influenced by the Local
        SONET indication, Tree Build Time, and Hardware Update Time.





Poretsky, Imhoff                                                                [Page 11]


INTERNET-DRAFT     Benchmarking Methodology for January 2004
                      IGP Data Plane Route Convergence

   5. Security Considerations

        Documents of this type do not directly affect the security of
        the Internet or corporate networks as long as benchmarking
        is not performed on devices or systems connected to operating
        networks.

   6. References

      [1] Poretsky, S., "Benchmarking Applicability for IGP
            Convergence", draft-ietf-bmwg-igp-dataplane-conv-app-02, work
            in progress, January 2004.

      [2] Poretsky, S., Imhoff, B., "Benchmarking Terminology for IGP
            Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-02, work
            in progress, January 2004

      [3] Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and Dual
            Environments", RFC 1195, December 1990.

      [4] Moy, J., "OSPF Version 2", RFC 2328, IETF, April 1998.

   7. Author's Address

        Scott Poretsky
        Quarry Technologies
        8 New England Executive Park
        Burlington, MA 01803
        USA

        Phone: + 1 781 395 5090
        EMail: sporetsky@quarrytech.com

        Brent Imhoff
        WilTel Communications
        3180 Rider Trail South
        Bridgeton, MO 63045
        USA

        Phone: +1 314 595 6853
        EMail: brent.imhoff@wcg.com











Poretsky, Imhoff                                                                [Page 12]


INTERNET-DRAFT     Benchmarking Methodology for January 2004
                      IGP Data Plane Route Convergence

   8.  Full Copyright Statement
        Copyright (C) The Internet Society (1998).  All Rights
        Reserved.

        This document and translations of it may be copied and
        furnished to others, and derivative works that comment on or
        otherwise explain it or assist in its implementation may be
        prepared, copied, published and distributed, in whole or in
        part, without restriction of any kind, provided that the above
        copyright notice and this paragraph are included on all such
        copies and derivative works.  However, this document itself may
        not be modified in any way, such as by removing the copyright
        notice or references to the Internet Society or other Internet
        organizations, except as needed for the purpose of developing
        Internet standards in which case the procedures for copyrights
        defined in the Internet Standards process must be followed, or
        as required to translate it into languages other than English.

        The limited permissions granted above are perpetual and will
        not be revoked by the Internet Society or its successors or
        assigns.  This document and the information contained herein is
        provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE
        INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY
        THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY
        RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
        FOR A PARTICULAR PURPOSE.

























Poretsky, Imhoff                                                                [Page 13]   Network Working Group
   INTERNET-DRAFT
   Expires in: July 2004
                                                   Scott Poretsky
                                                   Quarry Technologies

                                                   January 2004

                 Benchmarking Applicability for
                IGP Data Plane Route Convergence

        <draft-ietf-bmwg-igp-dataplane-conv-app-02.txt>


   Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force  (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time.  It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.


   ABSTRACT
   This draft describes the applicability of IGP Route Convergence
   benchmarking methodology [1] and IGP Route Convergence benchmarking
   terminology [2].  The methodology and terminology is to be used
   for benchmarking route convergence and can be applied to any
   link-state IGP such as ISIS [3] and OSPF [4].  The data plane is
   measured to obtain the convergence benchmarking metrics described
   in [1].

   Table of Contents
     1. Introduction ...............................................2
     2. Existing definitions .......................................2
     3. Factors for IGP Route Convergence Time......................2
     4. Network Events that Cause Route Convergence.................3
     5. Use of Data Traffic for IGP Route Convergence Benchmarking..3
     6. Security Considerations.....................................4
     7. Acknowledgements............................................4
     8. References..................................................4

Poretsky                                                        [Page 1]


INTERNET-DRAFT          Benchmarking Applicability for          January 2004
                       IGP Data Plane Route Convergence

     9. Author's Address............................................5
     10. Full Copyright Statement...................................5

   1. Introduction
   IGP Convergence is a critical performance parameter.  Customers
   of Service Providers use packet loss due to IGP Convergence as a
   key metric of their network service quality.  Service Providers
   use IGP Convergence time as a key metric of router design and
   architecture.  Fast network convergence can be optimally achieved
   through deployment of fast converging routers.  The fundamental
   basis by which network users and operators benchmark convergence
   is packet loss, which is an externally observable event having
   direct impact on their application performance.

   IGP Route Convergence is a Direct Measure of Quality (DMOQ) when
   benchmarking the data plane.  For this reason it is important to
   develop a standard router benchmarking methodology and terminology
   for measuring IGP convergence that uses the data plane as described
   in [1] and [2].  This document describes all of the factors that
   influence a convergence measurement and how a purely black box test
   can be designed to account for all of these factors.  This enables
   accurate benchmarking and evaluation for route convergence time.

   2.  Existing definitions

   For the sake of clarity and continuity this RFC adopts the template
   for definitions set out in Section 2 of RFC 1242.  Definitions are
   indexed and grouped together in sections for ease of reference.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in
   this document are to be interpreted as described in RFC 2119.

   3. Factors for IGP Route Convergence Time

   There are four major categories of factors contributing to the
   measured Router IGP Convergence Time.   As discussed in [5], [6],
   [7], [8] and [9], these categories are Event Detection, SPF
   Processing, IGP Advertisement, and FIB Update.  These have numerous
   components that influence the convergence time.  These are listed
   as follow:

        -Event Detection-
        SONET failure indication time
        PPP failure indication time
        IGP Hello Dead Interval

        -SPF Processing-
        SPF Delay Time
        SPF Hold time
        SPF Execution time

Poretsky                                                        [Page 2]


INTERNET-DRAFT          Benchmarking Applicability for          January 2004
                       IGP Data Plane Route Convergence
        -IGP Advertisement-
        LSA/LSP Flood Packet Pacing
        LSA/LSP Retransmission Packet Pacing
        LSA/LSP Generation time

        -FIB Update-
        Tree Build time
        Hardware Update time

   The contribution of each of these factors listed above will vary
   with each router vendors' architecture and IGP implementation.
   It is therefore necessary to design a convergence test that
   considers all of these components, not just one or a few of these
   components.  The additional benefit of designing a test for all
   components is that it enables black-box testing in which knowledge
   of the routers' internal implementations is not required.  It is
   then possible to make valid use of the convergence benchmarking
   metrics when comparing routers from different vendors.

   4. Network Events that Cause Convergence

   There are different types of network events that can cause IGP
   convergence.  These network events are administrative link
   removal, unplanned link failure, line card failure, and route
   changes such as withdrawal, flap, next-hop change, and cost change.
   When benchmarking a router it is important to measure the
   convergence time for local and remote occurrence of these network
   events.  The convergence time measured will vary whether the network
   event occurred locally or remotely due to varying combinations of
   factors listed in the previous sections.  This behavior makes it
   possible to design purely black-box tests that isolate
   measurements for each of the components of convergence time.

   5. Use of Data Plane for IGP Route Convergence Benchmarking

   Customers of service providers use packet loss as the metric to
   calculate convergence time.  Packet loss is an externally observable
   event having direct impact on customers' application performance.
   For this reason it is important to develop a standard router
   benchmarking methodology and terminology that is a Direct Measure
   of Quality (DMOQ)for measuring IGP convergence.  Such a
   methodology uses the data plane as described in [1] and [2].

   An additional benefit of using packet loss for calculation of
   IGP Route Convergence time is that it enables black-box tests to
   be designed.  Data traffic can be offered to the
   device under test (DUT), an emulated network event can be forced
   to occur, and packet loss can be externally measured to calculate
   the convergence time.  Knowledge of the DUT architecture and IGP
   implementation is not required. There is no need to rely on the
   DUT to produce the test results.  There is no need to build
   intrusive test harnesses for the DUT.

Poretsky                                                        [Page 3]


INTERNET-DRAFT          Benchmarking Applicability for          January 2004
                       IGP Data Plane Route Convergence

   Use of data traffic and measurement of packet loss on the data
   plane also enables Route Convergence methodology test cases that
   consider the time for the Route Controller to update the FIB on
   the forwarding engine of the hardware.  A router is not fully
   converged until all components are updated and traffic is
   rerouted to the correct egress interface.  As long as there is
   packet loss, routes have not converged.  It is possible to send
   diverse traffic flows to destinations matching every route in the
   FIB so that the time it takes for the router to converge an entire
   route table can be benchmarked.

   6. Security Considerations

        Documents of this type do not directly effect the security of
        the Internet or of corporate networks as long as benchmarking
        is not performed on devices or systems connected to operating
        networks.

   7. Acknowledgements
        Thanks to Curtis Villamizar for sharing so much of his
        knowledge and experience through the years. Also, special
        thanks to the many Network Engineers and Network Architects
        at the Service Providers who are always eager to discuss
        Route Convergence.

   8. References

      [1]   Poretsky, S., "Benchmarking Methodology for IGP Data Plane
            Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-meth-01,
            work in progress, October 2004.

      [2]   Poretsky, S., "Benchmarking Terminology for IGP Data Plane
            Route Convergence", draft-ietf-bmwg-igp-dataplane-conv-term-01,
            work in progress, October 2004.

      [3]   Callon, R., "Use of OSI IS-IS for Routing in TCP/IP and Dual
            Environments", RFC 1195, December 1990.

      [4]   Moy, J., "OSPF Version 2", RFC 2328, IETF, April 1998.

      [5]   Villamizar, C., "Convergence and Restoration Techniques for
            ISP Interior Routing", NANOG 25, October 2002.

      [6]   Katz, D., "Why are we Scared of SPF?  IGP Scaling and
            Stability", NANOG 25, October 2002.

      [7]   Filsfils, C., "Deploying Tight-SLA Services on an Internet
            Backbone: ISIS Fast Convergence and Differentiated Services
            Design (tutorial)", NANOG 25, October 2002.


Poretsky                                                        [Page 4]


INTERNET-DRAFT          Benchmarking Applicability for          January 2004
                       IGP Data Plane Route Convergence

      [8]   Alaettinoglu, C. and Casner, S., "ISIS Routing on the Qwest
            Backbone: a Recipe for Subsecond ISIS Convergence", NANOG 24,
            October 2002.

      [9]   Alaettinoglu, C., Jacobson, V., and Yu, H., "Towards
            Millisecond IGP Convergence", NANOG 20, October 2000.


   9. Author's Address

        Scott Poretsky
        Quarry Technologies
        8 New England Executive Park
        Burlington, MA 01803
        USA

        Phone: + 1 781 395 5090
        EMail: sporetsky@quarrytech.com

   10.  Full Copyright Statement

        Copyright (C) The Internet Society (1998).  All Rights
        Reserved.

        This document and translations of it may be copied and
        furnished to others, and derivative works that comment on or
        otherwise explain it or assist in its implementation may be
        prepared, copied, published and distributed, in whole or in
        part, without restriction of any kind, provided that the above
        copyright notice and this paragraph are included on all such
        copies and derivative works.  However, this document itself may
        not be modified in any way, such as by removing the copyright
        notice or references to the Internet Society or other Internet
        organizations, except as needed for the purpose of developing
        Internet standards in which case the procedures for copyrights
        defined in the Internet Standards process must be followed, or
        as required to translate it into languages other than English.

        The limited permissions granted above are perpetual and will
        not be revoked by the Internet Society or its successors or
        assigns.  This document and the information contained herein is
        provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE
        INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY
        THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY
        RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
        FOR A PARTICULAR PURPOSE.

Poretsky                                                        [Page 5]