Network Working Group                                        S. Poretsky
Internet-Draft                                      Allot Communications
Intended status: Informational                                 B. Imhoff
Expires: April 29, 2010                                 Juniper Networks
                                                           K. Michielsen
                                                           Cisco Systems
                                                        October 26, 2009


Benchmarking Methodology for Link-State IGP Data Plane Route Convergence
               draft-ietf-bmwg-igp-dataplane-conv-meth-19

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.  This document may contain material
   from IETF Documents or IETF Contributions published or made publicly
   available before November 10, 2008.  The person(s) controlling the
   copyright in some of this material may not have granted the IETF
   Trust the right to allow modifications of such material outside the
   IETF Standards Process.  Without obtaining an adequate license from
   the person(s) controlling the copyright in such materials, this
   document may not be modified outside the IETF Standards Process, and
   derivative works of it may not be created outside the IETF Standards
   Process, except to format it for publication as an RFC or to
   translate it into languages other than English.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on April 29, 2010.

Copyright Notice

   Copyright (c) 2009 IETF Trust and the persons identified as the



Poretsky, et al.         Expires April 29, 2010                 [Page 1]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents in effect on the date of
   publication of this document (http://trustee.ietf.org/license-info).
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.

Abstract

   This document describes the methodology for benchmarking Link-State
   Interior Gateway Protocol (IGP) Route Convergence.  The methodology
   is to be used for benchmarking IGP convergence time through
   externally observable (black box) data plane measurements.  The
   methodology can be applied to any link-state IGP, such as ISIS and
   OSPF.



































Poretsky, et al.         Expires April 29, 2010                 [Page 2]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


Table of Contents

   1.  Introduction and Scope . . . . . . . . . . . . . . . . . . . .  5
   2.  Existing Definitions . . . . . . . . . . . . . . . . . . . . .  5
   3.  Test Topologies  . . . . . . . . . . . . . . . . . . . . . . .  5
     3.1.  Test topology for local changes  . . . . . . . . . . . . .  5
     3.2.  Test topology for remote changes . . . . . . . . . . . . .  6
     3.3.  Test topology for local ECMP changes . . . . . . . . . . .  7
     3.4.  Test topology for remote ECMP changes  . . . . . . . . . .  7
     3.5.  Test topology for Parallel Link changes  . . . . . . . . .  8
   4.  Convergence Time and Loss of Connectivity Period . . . . . . .  9
     4.1.  Convergence Events without instant traffic loss  . . . . . 10
     4.2.  Loss of Connectivity . . . . . . . . . . . . . . . . . . . 12
   5.  Test Considerations  . . . . . . . . . . . . . . . . . . . . . 13
     5.1.  IGP Selection  . . . . . . . . . . . . . . . . . . . . . . 13
     5.2.  Routing Protocol Configuration . . . . . . . . . . . . . . 13
     5.3.  IGP Topology . . . . . . . . . . . . . . . . . . . . . . . 13
     5.4.  Timers . . . . . . . . . . . . . . . . . . . . . . . . . . 14
     5.5.  Interface Types  . . . . . . . . . . . . . . . . . . . . . 14
     5.6.  Offered Load . . . . . . . . . . . . . . . . . . . . . . . 14
     5.7.  Measurement Accuracy . . . . . . . . . . . . . . . . . . . 15
     5.8.  Measurement Statistics . . . . . . . . . . . . . . . . . . 15
     5.9.  Tester Capabilities  . . . . . . . . . . . . . . . . . . . 15
   6.  Selection of Convergence Time Benchmark Metrics and Methods  . 16
     6.1.  Loss-Derived Method  . . . . . . . . . . . . . . . . . . . 16
       6.1.1.  Tester capabilities  . . . . . . . . . . . . . . . . . 16
       6.1.2.  Benchmark Metrics  . . . . . . . . . . . . . . . . . . 17
       6.1.3.  Measurement Accuracy . . . . . . . . . . . . . . . . . 17
     6.2.  Rate-Derived Method  . . . . . . . . . . . . . . . . . . . 17
       6.2.1.  Tester Capabilities  . . . . . . . . . . . . . . . . . 17
       6.2.2.  Benchmark Metrics  . . . . . . . . . . . . . . . . . . 17
       6.2.3.  Measurement Accuracy . . . . . . . . . . . . . . . . . 17
     6.3.  Route-Specific Loss-Derived Method . . . . . . . . . . . . 18
       6.3.1.  Tester Capabilities  . . . . . . . . . . . . . . . . . 18
       6.3.2.  Benchmark Metrics  . . . . . . . . . . . . . . . . . . 18
       6.3.3.  Measurement Accuracy . . . . . . . . . . . . . . . . . 18
   7.  Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 18
   8.  Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 20
     8.1.  Interface failures . . . . . . . . . . . . . . . . . . . . 21
       8.1.1.  Convergence Due to Local Interface Failure . . . . . . 21
       8.1.2.  Convergence Due to Remote Interface Failure  . . . . . 22
       8.1.3.  Convergence Due to ECMP Member Local Interface
               Failure  . . . . . . . . . . . . . . . . . . . . . . . 23
       8.1.4.  Convergence Due to ECMP Member Remote Interface
               Failure  . . . . . . . . . . . . . . . . . . . . . . . 25
       8.1.5.  Convergence Due to Parallel Link Interface Failure . . 26
     8.2.  Other failures . . . . . . . . . . . . . . . . . . . . . . 27
       8.2.1.  Convergence Due to Layer 2 Session Loss  . . . . . . . 27



Poretsky, et al.         Expires April 29, 2010                 [Page 3]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


       8.2.2.  Convergence Due to Loss of IGP Adjacency . . . . . . . 28
       8.2.3.  Convergence Due to Route Withdrawal  . . . . . . . . . 30
     8.3.  Administrative changes . . . . . . . . . . . . . . . . . . 31
       8.3.1.  Convergence Due to Local Adminstrative Shutdown  . . . 31
       8.3.2.  Convergence Due to Cost Change . . . . . . . . . . . . 32
   9.  Security Considerations  . . . . . . . . . . . . . . . . . . . 34
   10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 34
   11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 34
   12. Normative References . . . . . . . . . . . . . . . . . . . . . 35
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35









































Poretsky, et al.         Expires April 29, 2010                 [Page 4]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


1.  Introduction and Scope

   This document describes the methodology for benchmarking Link-State
   Interior Gateway Protocol (IGP) convergence.  The motivation and
   applicability for this benchmarking is described in [Po09a].  The
   terminology to be used for this benchmarking is described in [Po09t].

   IGP convergence time is measured on the data plane at the Tester by
   observing packet loss through the DUT.  All factors contributing to
   convergence time are accounted for by measuring on the data plane, as
   discussed in [Po09a].  The test cases in this document are black-box
   tests that emulate the network events that cause convergence, as
   described in [Po09a].

   The methodology described in this document can be applied to IPv4 and
   IPv6 traffic and link-state IGPs such as ISIS [Ca90][Ho08], OSPF
   [Mo98][Co08], and others.


2.  Existing Definitions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in BCP 14, RFC 2119
   [Br97].  RFC 2119 defines the use of these key words to help make the
   intent of standards track documents as clear as possible.  While this
   document uses these keywords, this document is not a standards track
   document.

   This document uses much of the terminology defined in [Po09t] and
   uses existing terminology defined in other BMWG work.  Examples
   include, but are not limited to:

      Throughput                         [Ref.[Br91], section 3.17]
      Device Under Test (DUT)            [Ref.[Ma98], section 3.1.1]
      System Under Test (SUT)            [Ref.[Ma98], section 3.1.2]
      Out-of-order Packet                [Ref.[Po06], section 3.3.2]
      Duplicate Packet                   [Ref.[Po06], section 3.3.3]
      Stream                             [Ref.[Po06], section 3.3.2]
      Loss Period                        [Ref.[Ko02], section 4]


3.  Test Topologies

3.1.  Test topology for local changes

   Figure 1 shows the test topology to measure IGP convergence time due
   to local Convergence Events such as Local Interface failure



Poretsky, et al.         Expires April 29, 2010                 [Page 5]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   (Section 8.1.1), layer 2 session failure (Section 8.2.1), and IGP
   adjacency failure (Section 8.2.2).  This topology is also used to
   measure IGP convergence time due to the route withdrawal
   (Section 8.2.3), and route cost change (Section 8.3.2) Convergence
   Events.  IGP adjancencies MUST be established between Tester and DUT,
   one on the Preferred Egress Interface and one on the Next-Best Egress
   Interface.  For this purpose the Tester emulates two routers, each
   establishing one adjacency with the DUT.  An IGP adjacency SHOULD be
   established on the Ingress Interface between Tester and DUT.

            ---------       Ingress Interface         ----------
            |       |<--------------------------------|        |
            |       |                                 |        |
            |       |    Preferred Egress Interface   |        |
            |  DUT  |-------------------------------->| Tester |
            |       |                                 |        |
            |       |-------------------------------->|        |
            |       |    Next-Best Egress Interface   |        |
            ---------                                 ----------

         Figure 1: IGP convergence test topology for local changes

3.2.  Test topology for remote changes

   Figure 2 shows the test topology to measure IGP convergence time due
   to Remote Interface failure (Section 8.1.2).  In this topology the
   two routers R1 and R2 are considered System Under Test (SUT) and
   SHOULD be identically configured devices of the same model.  IGP
   adjancencies MUST be established between Tester and SUT, one on the
   Preferred Egress Interface and one on the Next-Best Egress Interface.
   For this purpose the Tester emulates one or two routers.  An IGP
   adjacency SHOULD be established on the Ingress Interface between
   Tester and SUT.  In this topology there is a possibility of a
   transient microloop between R1 and R2 during convergence.

















Poretsky, et al.         Expires April 29, 2010                 [Page 6]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


                       ------                      ----------
                       |    |  Preferred           |        |
              ------   | R2 |--------------------->|        |
              |    |-->|    |  Egress Interface    |        |
              |    |   ------                      |        |
              | R1 |                               | Tester |
              |    |           Next-Best           |        |
              |    |------------------------------>|        |
              ------           Egress Interface    |        |
                 ^                                 ----------
                 |                                     |
                 ---------------------------------------
                             Ingress Interface

        Figure 2: IGP convergence test topology for remote changes

3.3.  Test topology for local ECMP changes

   Figure 3 shows the test topology to measure IGP convergence time due
   to local Convergence Events with members of an Equal Cost Multipath
   (ECMP) set (Section 8.1.3).  In this topology, the DUT is configured
   with each egress interface as a member of a single ECMP set and the
   Tester emulates N next-hop routers, one router for each member.  IGP
   adjancencies MUST be established between Tester and DUT, one on each
   member of the ECMP set.  For this purpose each of the N routers
   emulated by the Tester establishes one adjacency with the DUT.  An
   IGP adjacency SHOULD be established on the Ingress Interface between
   Tester and DUT.

            ---------       Ingress Interface         ----------
            |       |<--------------------------------|        |
            |       |                                 |        |
            |       |     ECMP set interface 1        |        |
            |       |-------------------------------->|        |
            |  DUT  |               .                 | Tester |
            |       |               .                 |        |
            |       |               .                 |        |
            |       |-------------------------------->|        |
            |       |     ECMP set interface N        |        |
            ---------                                 ----------

       Figure 3: IGP convergence test topology for local ECMP change

3.4.  Test topology for remote ECMP changes

   Figure 4 shows the test topology to measure IGP convergence time due
   to remote Convergence Events with members of an Equal Cost Multipath
   (ECMP) set (Section 8.1.4).  In this topology the two routers R1 and



Poretsky, et al.         Expires April 29, 2010                 [Page 7]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   R2 are considered System Under Test (SUT) and MUST be identically
   configured devices of the same model.  Router R1 is configured with
   each egress interface as a member of a single ECMP set and the Tester
   emulates N next-hop routers, one router for each member.  IGP
   adjancencies MUST be established between Tester and SUT, one on each
   egress interface of SUT.  For this purpose each of the N routers
   emulated by the Tester establishes one adjacency with the SUT.  An
   IGP adjacency SHOULD be established on the Ingress Interface between
   Tester and SUT.  In this topology there is a possibility of a
   transient microloop between R1 and R2 during convergence.

                                        ------     ----------
                                        |    |     |        |
              ------      ECMP set      | R2 |---->|        |
              |    |------------------->|    |     |        |
              |    |      Interface 1   ------     |        |
              |    |                               |        |
              |    |      ECMP set interface 2     |        |
              | R1 |------------------------------>| Tester |
              |    |               .               |        |
              |    |               .               |        |
              |    |               .               |        |
              |    |------------------------------>|        |
              ------      ECMP set interface N     |        |
                 ^                                 ----------
                 |                                     |
                 ---------------------------------------
                             Ingress Interface

    Figure 4: IGP convergence test topology for remote ECMP convergence

3.5.  Test topology for Parallel Link changes

   Figure 5 shows the test topology to measure IGP convergence time due
   to local Convergence Events with members of a Parallel Link
   (Section 8.1.5).  In this topology, the DUT is configured with each
   egress interface as a member of a Parallel Link and the Tester
   emulates the single next-hop router.  IGP adjancencies MUST be
   established on all N members of the Parallel Link between Tester and
   DUT.  For this purpose the router emulated by the Tester establishes
   N adjacencies with the DUT.  An IGP adjacency SHOULD be established
   on the Ingress Interface between Tester and DUT.









Poretsky, et al.         Expires April 29, 2010                 [Page 8]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


            ---------       Ingress Interface         ----------
            |       |<--------------------------------|        |
            |       |                                 |        |
            |       |     Parallel Link Interface 1   |        |
            |       |-------------------------------->|        |
            |  DUT  |               .                 | Tester |
            |       |               .                 |        |
            |       |               .                 |        |
            |       |-------------------------------->|        |
            |       |     Parallel Link Interface N   |        |
            ---------                                 ----------

     Figure 5: IGP convergence test topology for Parallel Link changes


4.  Convergence Time and Loss of Connectivity Period

   Two concepts will be highlighted in this section: convergence time
   and loss of connectivity period.

   The Route Convergence [Po09t] time indicates the period in time
   between the Convergence Event Instant [Po09t] and the instant in time
   the DUT is ready to forward traffic for a specific route on its Next-
   Best Egress Interface and maintains this state for the duration of
   the Sustained Convergence Validation Time [Po09t].  To measure Route
   Convergence time, the Convergence Event Instant and the traffic
   received from the Next-Best Egress Interface need to be observed.

   The Route Loss of Connectivity Period [Po09t] indicates the time
   during which traffic to a specific route is lost following a
   Convergence Event until Full Convergence [Po09t] completes.  This
   Route Loss of Connectivity Period can consist of one or more Loss
   Periods [Ko02].  For the testcases described in this document it is
   expected to have a single Loss Period.  To measure Route Loss of
   Connectivity Period, the traffic received from the Preferred Egress
   Interface and the traffic received from the Next-Best Egress
   Interface need to be observed.

   The Route Loss of Connectivity Period is most important since that
   has a direct impact on the network user's application performance.

   In general the Route Convergence time is larger than or equal to the
   Route Loss of Connectivity Period.  Depending on which Convergence
   Event occurs and how this Convergence Event is applied, traffic for a
   route may still be forwarded over the Preferred Egress Interface
   after the Convergence Event Instant, before converging to the Next-
   Best Egress Interface.  In that case the Route Loss of Connectivity
   Period is shorter than the Route Convergence time.



Poretsky, et al.         Expires April 29, 2010                 [Page 9]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   At least one condition needs to be fulfilled for Route Convergence
   time to be equal to Route Loss of Connectivity Period.  The condition
   is that the Convergence Event causes an instantaneous traffic loss
   for the measured route.  A fiber cut on the Preferred Egress
   Interface is an example of such a Convergence Event.

   A second condition applies to Route Convergence time measurements
   based on Connectivity Packet Loss [Po09t].  This second condition is
   that there is only a single Loss Period during Route Convergence.
   For the testcases described in this document this is expected to be
   the case.

4.1.  Convergence Events without instant traffic loss

   To measure convergence time benchmarks for Convergence Events caused
   by a Tester, such as an IGP cost change, the Tester MAY start to
   discard all traffic received from the Preferred Egress Interface at
   the Convergence Event Instant, or MAY separately observe packets
   received from the Preferred Egress Interface prior to the Convergence
   Event Instant.  This way these Convergence Events can be treated the
   same as Convergence Events that cause instantaneous traffic loss.

   To measure convergence time benchmarks without instantaneous traffic
   loss (either real or induced by the Tester) at the Convergence Event
   Instant, such as a reversion of a link failure Convergence Event, the
   Tester SHALL only observe packet statistics on the Next-Best Egress
   Interface.  If using the Rate-Derived method to benchmark convergence
   times for such Convergence Events, the Tester MUST collect a
   timestamp at the Convergence Event Instant.  If using a loss-derived
   method to benchmark convergence times for such Convergence Events,
   the Tester MUST measure the period in time between the Start Traffic
   Instant and the Convergence Event Instant.  To measure this period in
   time the Tester can collect timestamps at the Start Traffic Instant
   and the Convergence Event Instant.

   The Convergence Event Instant together with the receive rate
   observations on the Next-Best Egress Interface allow to derive the
   convergence time benchmarks using the Rate-Derived Method [Po09t].

   By observing lost packets on the Next-Best Egress Interface only, the
   observed packet loss is the number of lost packets between Traffic
   Start Instant and Convergence Recovery Instant.  To measure
   convergence times using a loss-derived method, packet loss between
   the Convergence Event Instant and the Convergence Recovery Instant is
   needed.  The time between Traffic Start Instant and Convergence Event
   Instant must be accounted for.  An example may clarify this.

   Figure 6 illustrates a Convergence Event without instantaneous



Poretsky, et al.         Expires April 29, 2010                [Page 10]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   traffic loss for all routes.  The top graph shows the Forwarding Rate
   over all routes, the bottom graph shows the Forwarding Rate for a
   single route Rta. Some time after the Convergence Event Instant,
   Forwarding Rate observed on the Preferred Egress Interface starts to
   decrease.  In the example, route Rta is the first route to experience
   packet loss at time Ta.  Some time later, the Forwarding Rate
   observed on the Next-Best Egress Interface starts to increase.  In
   the example, route Rta is the first route to complete convergence at
   time Ta'.

                ^
           Fwd  |
           Rate |-------------                    ............
                |             \                  .
                |              \                .
                |               \              .
                |                \            .
                |.................-.-.-.-.-.-.----------------
                +----+-------+---------------+----------------->
                ^    ^       ^               ^             time
               T0   CEI      Ta              Ta'

                ^
           Fwd  |
           Rate |-------------               .................
           Rta  |            |               .
                |            |               .
                |.............-.-.-.-.-.-.-.-.----------------
                +----+-------+---------------+----------------->
                ^    ^       ^               ^             time
               T0   CEI      Ta              Ta'

                Preferred Egress Interface: ---
                Next-Best Egress Interface: ...

   With T0 the Start Traffic Instant; CEI the Convergence Event Instant;
   Ta the time instant traffic loss for route Rta starts; Ta' the time
   instant traffic loss for route Rta ends.

                                 Figure 6

   If only packets received on the Next-Best Egress Interface are
   observed, the duration of the packet loss period for route Rta can be
   calculated from the received packets as in Equation 1.  Since the
   Convergence Event Instant is the start time for convergence time
   measurement, the period in time between T0 and CEI needs to be
   subtracted from the calculated result to become the convergence time,
   as in Equation 2.



Poretsky, et al.         Expires April 29, 2010                [Page 11]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   Next-Best Egress Interface packet loss period
       = (packets transmitted
           - packets received from Next-Best Egress Interface) / tx rate
       = Ta' - T0

                                Equation 1

      convergence time
          = Next-Best Egress Interface packet loss period - (CEI - T0)
          = Ta' - CEI

                                Equation 2

4.2.  Loss of Connectivity

   Route Loss of Connectivity Period SHOULD be measured using the Route-
   Specific Loss-Derived Method.  Since the start instant and end
   instant of the Route Loss of Connectivity Period can be different for
   each route, these can not be accurately derived by only observing
   global statistics over all routes.  An example may clarify this.

   Following a Convergence Event, route Rta is the first route for which
   packet loss starts, the Route Loss of Connectivity Period for route
   Rta starts at time Ta.  Route Rtb is the last route for which packet
   loss starts, the Route Loss of Connectivity Period for route Rtb
   starts at time Tb with Tb>Ta.

                  ^
             Fwd  |
             Rate |--------                       -----------
                  |        \                     /
                  |         \                   /
                  |          \                 /
                  |           \               /
                  |            ---------------
                  +------------------------------------------>
                           ^   ^             ^    ^      time
                          Ta   Tb           Ta'   Tb'
                                            Tb''  Ta''


            Figure 7: Example Route Loss Of Connectivity Period

   If the DUT implementation would be such that Route Rta would be the
   first route for which traffic loss ends at time Ta' with Ta'>Tb.
   Route Rtb would be the last route for which traffic loss ends at time
   Tb' with Tb'>Ta'.  By using only observing global traffic statistics
   over all routes, the minimum Route Loss of Connectivity Period would



Poretsky, et al.         Expires April 29, 2010                [Page 12]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   be measured as Ta'-Ta.  The maximum calculated Route Loss of
   Connectivity Period would be Tb'-Ta.  The real minimum and maximum
   Route Loss of Connectivity Periods are Ta'-Ta and Tb'-Tb.
   Illustrating this with the numbers Ta=0, Tb=1, Ta'=3, and Tb'=5,
   would give a LoC Period between 3 and 5 derived from the global
   traffic statistics, versus the real LoC Period between 3 and 4.

   If the DUT implementation would be such that route Rtb would be the
   first for which packet loss ends at time Tb'' and route Rta would be
   the last for which packet loss ends at time Ta'', then the minimum
   and maximum Route Loss of Connectivity Periods derived by observing
   only global traffic statistics would be Tb''-Ta, and Ta''-Ta.  The
   real minimum and maximum Route Loss of Connectivity Periods are
   Tb''-Tb and Ta''-Ta.  Illustrating this with the numbers Ta=0, Tb=1,
   Ta''=5, Tb''=3, would give a LoC Period between 3 and 5 derived from
   the global traffic statistics, versus the real LoC Period between 2
   and 5.

   The two implementation variations in the above example would result
   in the same derived minimum and maximum Route Loss of Connectivity
   Periods when only observing the global packet statistics, while the
   real Route Loss of Connectivity Periods are different.


5.  Test Considerations

5.1.  IGP Selection

   The test cases described in Section 8 MAY be used for link-state
   IGPs, such as ISIS or OSPF.  The IGP convergence time test
   methodology is identical.

5.2.  Routing Protocol Configuration

   The obtained results for IGP convergence time may vary if other
   routing protocols are enabled and routes learned via those protocols
   are installed.  IGP convergence times SHOULD be benchmarked without
   routes installed from other protocols.

5.3.  IGP Topology

   The Tester emulates a single IGP topology.  The DUT establishes IGP
   adjacencies with one or more of the emulated routers in this single
   IGP topology emulated by the Tester.  See test topology details in
   Section 3.  The emulated topology SHOULD only be advertised on the
   DUT egress interfaces.

   The number of IGP routes will impact the measured IGP route



Poretsky, et al.         Expires April 29, 2010                [Page 13]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   convergence time.  To obtain results similar to those that would be
   observed in an operational network, it is RECOMMENDED that the number
   of installed routes and nodes closely approximate that of the network
   (e.g. thousands of routes with tens or hundreds of nodes).

   The number of areas (for OSPF) and levels (for ISIS) can impact the
   benchmark results.

5.4.  Timers

   There are timers that may impact the measured IGP convergence times.
   The benchmark metrics MAY be measured at any fixed values for these
   timers.  To obtain results similar to those that would be observed in
   an operational network, it is RECOMMENDED to configure the timers
   with the values as configured in the operational network.

   Examples of timers that may impact measured IGP convergence time
   include, but are not limited to:

      Interface failure indication

      IGP hello timer

      IGP dead-interval or hold-timer

      LSA or LSP generation delay

      LSA or LSP flood packet pacing

      SPF delay

5.5.  Interface Types

   All test cases in this methodology document MAY be executed with any
   interface type.  The type of media may dictate which test cases may
   be executed.  Each interface type has a unique mechanism for
   detecting link failures and the speed at which that mechanism
   operates will influence the measurement results.  All interfaces MUST
   be the same media and Throughput [Br91][Br99] for each test case.
   All interfaces SHOULD be configured as point-to-point.

5.6.  Offered Load

   The Throughput of the device, as defined in [Br91] and benchmarked in
   [Br99] at a fixed packet size, needs to be determined over the
   preferred path and over the next-best path.  The Offered Load SHOULD
   be the minimum of the measured Throughput of the device over the
   primary path and over the backup path.  The packet size is selectable



Poretsky, et al.         Expires April 29, 2010                [Page 14]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   and MUST be recorded.  Packet size is measured in bytes and includes
   the IP header and payload.

   The destination addresses for the Offered Load MUST be distributed
   such that all routes or a statistically representative subset of all
   routes are matched and each of these routes is offered an equal share
   of the Offered Load.  It is RECOMMENDED to send traffic matching all
   routes, but a statistically representative subset of all routes can
   be used if required.

   In the Remote Interface failure testcases using topologies 2 and 4
   there is a possibility of a transient microloop between R1 and R2
   during convergence.  The TTL or Hop Limit value of the packets sent
   by the Tester may influence the benchmark measurements since it
   determines which device in the topology may send an ICMP Time
   Exceeded Message for looped packets.

   The duration of the Offered Load MUST be greater than the convergence
   time.

5.7.  Measurement Accuracy

   Since packet loss is observed to measure the Route Convergence Time,
   the time between two successive packets offered to each individual
   route is the highest possible accuracy of any packet loss based
   measurement.  When packet jitter is much less than the convergence
   time, it is a negligible source of error and therefore it will be
   ignored here.

5.8.  Measurement Statistics

   The benchmark measurements may vary for each trial, due to the
   statistical nature of timer expirations, cpu scheduling, etc.
   Evaluation of the test data must be done with an understanding of
   generally accepted testing practices regarding repeatability,
   variance and statistical significance of a small number of trials.

5.9.  Tester Capabilities

   It is RECOMMENDED that the Tester used to execute each test case has
   the following capabilities:

   1.  Ability to establish IGP adjacencies and advertise a single IGP
       topology to one or more peers.

   2.  Ability to insert a timestamp in each data packet's IP payload.





Poretsky, et al.         Expires April 29, 2010                [Page 15]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   3.  An internal time clock to control timestamping, time
       measurements, and time calculations.

   4.  Ability to distinguish traffic load received on the Preferred and
       Next-Best Interfaces [Po09t].

   5.  Ability to disable or tune specific Layer-2 and Layer-3 protocol
       functions on any interface(s).

   The Tester MAY be capable to make non-data plane convergence
   observations and use those observations for measurements.  The Tester
   MAY be capable to send and receive multiple traffic Streams [Po06].

   Also see Section 6 for method-specific capabilities.


6.  Selection of Convergence Time Benchmark Metrics and Methods

   Different convergence time benchmark methods MAY be used to measure
   convergence time benchmark metrics.  The Tester capabilities are
   important criteria to select a specific convergence time benchmark
   method.  The criteria to select a specific benchmark method include,
   but are not limited to:

   Tester capabilities:               Sampling Interval, number of
                                      Stream statistics to collect
   Measurement accuracy:              Sampling Interval, Offered Load
   Test specification:                number of routes
   DUT capabilities:                  Throughput

6.1.  Loss-Derived Method

6.1.1.  Tester capabilities

   The Offered Load SHOULD consist of a single Stream [Po06].  If
   sending multiple Streams, the measured packet loss statistics for all
   Streams MUST be added together.

   In order to verify Full Convergence completion and the Sustained
   Convergence Validation Time, the Tester MUST measure Forwarding Rate
   each Packet Sampling Interval.

   The total number of packets lost between the start of the traffic and
   the end of the Sustained Convergence Validation Time is used to
   calculate the Loss-Derived Convergence Time.






Poretsky, et al.         Expires April 29, 2010                [Page 16]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


6.1.2.  Benchmark Metrics

   The Loss-Derived Method can be used to measure the Loss-Derived
   Convergence Time, which is the average convergence time over all
   routes, and to measure the Loss-Derived Loss of Connectivity Period,
   which is the average Route Loss of Connectivity Period over all
   routes.

6.1.3.  Measurement Accuracy

   The measurement accuracy of the Loss-Derived Method is equal to the
   time between two consecutive packets to the same route.

6.2.  Rate-Derived Method

6.2.1.  Tester Capabilities

   The Offered Load SHOULD consist of a single Stream.  If sending
   multiple Streams, the measured traffic rate statistics for all
   Streams MUST be added together.

   The Tester measures Forwarding Rate each Sampling Interval.  The
   Packet Sampling Interval influences the observation of the different
   convergence time instants.  If the Packet Sampling Interval is large
   compared to the time between the convergence time instants, then the
   different time instants may not be easily identifiable from the
   Forwarding Rate observation.  The requirements for the Packet
   Sampling Interval are specified in [Po09t].  The RECOMMENDED value
   for the Packet Sampling Interval is 10 milliseconds.  The Packet
   Sampling Interval MUST be reported.

6.2.2.  Benchmark Metrics

   The Rate-Derived Method SHOULD be used to measure First Route
   Convergence Time and Full Convergence Time.  It SHOULD NOT be used to
   measure Loss of Connectivity Period (see Section 4).

6.2.3.  Measurement Accuracy

   The measurement accuracy of the Rate-Derived Method for transitions
   that occur for all routes at the same instant is equal to the Packet
   Sampling Interval and for other transitions the measurement accuracy
   is equal to the Packet Sampling Interval plus the time between two
   consecutive packets to the same destination.  The latter is the case
   since packets are sent in a particular order to all destinations in a
   stream and when part of the routes experience packet loss, it is
   unknown where in the transmit cycle packets to these routes are sent.
   This uncertainty adds to the error.



Poretsky, et al.         Expires April 29, 2010                [Page 17]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


6.3.  Route-Specific Loss-Derived Method

6.3.1.  Tester Capabilities

   The Offered Load consists of multiple Streams.  The Tester MUST
   measure packet loss for each Stream separately.

   In order to verify Full Convergence completion and the Sustained
   Convergence Validation Time, the Tester MUST measure packet loss each
   Packet Sampling Interval.  This measurement at each Packet Sampling
   Interval MAY be per Stream.

   Only the total packet loss measured per Stream at the end of the
   Sustained Convergence Validation Time is used to calculate the
   benchmark metrics with this method.

6.3.2.  Benchmark Metrics

   The Route-Specific Loss-Derived Method SHOULD be used to measure
   Route-Specific Convergence Times.  It is the RECOMMENDED method to
   measure Route Loss of Connectivity Period.

   Under the conditions explained in Section 4, First Route Convergence
   Time and Full Convergence Time as benchmarked using Rate-Derived
   Method, may be equal to the minimum resp. maximum of the Route-
   Specific Convergence Times.

6.3.3.  Measurement Accuracy

   The measurement accuracy of the Route-Specific Loss-Derived Method is
   equal to the time between two consecutive packets to the same route.


7.  Reporting Format

   For each test case, it is recommended that the reporting tables below
   are completed and all time values SHOULD be reported with resolution
   as specified in [Po09t].













Poretsky, et al.         Expires April 29, 2010                [Page 18]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


        Parameter                           Units
        ----------------------------------- -----------------------
        Test Case                           test case number
        Test Topology                       (1, 2, 3, 4, or 5)
        IGP                                 (ISIS, OSPF, other)
        Interface Type                      (GigE, POS, ATM, other)
        Packet Size offered to DUT          bytes
        Offered Load                        packets per second
        IGP Routes advertised to DUT        number of IGP routes
        Nodes in emulated network           number of nodes
        Number of Routes measured           number of routes
        Packet Sampling Interval on Tester  seconds
        Forwarding Delay Threshold          seconds

        Timer Values configured on DUT:
         Interface failure indication delay seconds
         IGP Hello Timer                    seconds
         IGP Dead-Interval or hold-time     seconds
         LSA Generation Delay               seconds
         LSA Flood Packet Pacing            seconds
         LSA Retransmission Packet Pacing   seconds
         SPF Delay                          seconds

   Test Details:

      If the Offered Load matches a subset of routes, describe how this
      subset is selected.

      Describe how the Convergence Event is applied; does it cause
      instantaneous traffic loss or not.

   Complete the table below for the initial Convergence Event and the
   reversion Convergence Event.


















Poretsky, et al.         Expires April 29, 2010                [Page 19]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


     Parameter                                  Units
     ------------------------------------------ ----------------------
     Conversion Event                           (initial or reversion)

     Traffic Forwarding Metrics:
      Total number of packets offered to DUT    number of Packets
      Total number of packets forwarded by DUT  number of Packets
      Connectivity Packet Loss                  number of Packets
      Convergence Packet Loss                   number of Packets
      Out-of-Order Packets                      number of Packets
      Duplicate Packets                         number of Packets

     Convergence Benchmarks:
      Rate-Derived Method:
       First Route Convergence Time             seconds
       Full Convergence Time                    seconds
      Loss-Derived Method:
       Loss-Derived Convergence Time            seconds
      Route-Specific Loss-Derived Method:
       Route-Specific Convergence Time[n]       array of seconds
       Minimum R-S Convergence Time             seconds
       Maximum R-S Convergence Time             seconds
       Median R-S Convergence Time              seconds
       Average R-S Convergence Time             seconds

     Loss of Connectivity Benchmarks:
      Loss-Derived Method:
       Loss-Derived Loss of Connectivity Period seconds
      Route-Specific Loss-Derived Method:
       Route LoC Period[n]                      array of seconds
       Minimum Route LoC Period                 seconds
       Maximum Route LoC Period                 seconds
       Median Route LoC Period                  seconds
       Average Route LoC Period                 seconds


8.  Test Cases

   It is RECOMMENDED that all applicable test cases be performed for
   best characterization of the DUT.  The test cases follow a generic
   procedure tailored to the specific DUT configuration and Convergence
   Event [Po09t].  This generic procedure is as follows:

   1.   Establish DUT and Tester configurations and advertise an IGP
        topology from Tester to DUT.

   2.   Send Offered Load from Tester to DUT on ingress interface.




Poretsky, et al.         Expires April 29, 2010                [Page 20]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   3.   Verify traffic is routed correctly.

   4.   Introduce Convergence Event [Po09t].

   5.   Measure First Route Convergence Time [Po09t].

   6.   Measure Full Convergence Time [Po09t].

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC Period
        [Po09t].

   9.   Wait sufficient time for queues to drain.

   10.  Restart Offered Load.

   11.  Reverse Convergence Event.

   12.  Measure First Route Convergence Time.

   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

8.1.  Interface failures

8.1.1.  Convergence Due to Local Interface Failure

   Objective

   To obtain the IGP convergence times due to a Local Interface failure
   event.

   Procedure

   1.   Advertise an IGP topology from Tester to DUT using the topology
        shown in Figure 1.

   2.   Send Offered Load from Tester to DUT on ingress interface.

   3.   Verify traffic is forwarded over Preferred Egress Interface.




Poretsky, et al.         Expires April 29, 2010                [Page 21]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   4.   Remove link on DUT's Preferred Egress Interface.  This is the
        Convergence Event.

   5.   Measure First Route Convergence Time.

   6.   Measure Full Convergence Time.

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times and Loss-Derived
        Convergence Time.

   9.   Wait sufficient time for queues to drain.

   10.  Restart Offered Load.

   11.  Restore link on DUT's Preferred Egress Interface.

   12.  Measure First Route Convergence Time.

   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   Results

   The measured IGP convergence time may be influenced by the link
   failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/
   LSP flood packet pacing, SPF delay, SPF execution time, and routing
   and forwarding tables update time [Po09a].

8.1.2.  Convergence Due to Remote Interface Failure

   Objective

   To obtain the IGP convergence time due to a Remote Interface failure
   event.

   Procedure

   1.   Advertise an IGP topology from Tester to SUT using the topology
        shown in Figure 2.





Poretsky, et al.         Expires April 29, 2010                [Page 22]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   2.   Send Offered Load from Tester to SUT on ingress interface.

   3.   Verify traffic is forwarded over Preferred Egress Interface.

   4.   Remove link on Tester's interface [Po09t] connected to SUT's
        Preferred Egress Interface.  This is the Convergence Event.

   5.   Measure First Route Convergence Time.

   6.   Measure Full Convergence Time.

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times and Loss-Derived
        Convergence Time.

   9.   Wait sufficient time for queues to drain.

   10.  Restart Offered Load.

   11.  Restore link on Tester's interface connected to DUT's Preferred
        Egress Interface.

   12.  Measure First Route Convergence Time.

   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   Results

   The measured IGP convergence time may be influenced by the link
   failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/
   LSP flood packet pacing, SPF delay, SPF execution time, and routing
   and forwarding tables update time.  This test case may produce Stale
   Forwarding [Po09t] due to a transient microloop between R1 and R2
   during convergence, which may increase the measured convergence times
   and loss of connectivity periods.

8.1.3.  Convergence Due to ECMP Member Local Interface Failure

   Objective

   To obtain the IGP convergence time due to a Local Interface link



Poretsky, et al.         Expires April 29, 2010                [Page 23]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   failure event of an ECMP Member.

   Procedure

   1.   Advertise an IGP topology from Tester to DUT using the test
        setup shown in Figure 3.

   2.   Send Offered Load from Tester to DUT on ingress interface.

   3.   Verify traffic is forwarded over the DUT's ECMP member interface
        that will be failed in the next step.

   4.   Remove link on one of the DUT's ECMP member interfaces.  This is
        the Convergence Event.

   5.   Measure First Route Convergence Time.

   6.   Measure Full Convergence Time.

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times and Loss-Derived
        Convergence Time.  At the same time measure Out-of-Order Packets
        [Po06] and Duplicate Packets [Po06].

   9.   Wait sufficient time for queues to drain.

   10.  Restart Offered Load.

   11.  Restore link on DUT's ECMP member interface.

   12.  Measure First Route Convergence Time.

   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.  At the same time measure Out-of-Order Packets [Po06]
        and Duplicate Packets [Po06].

   Results

   The measured IGP Convergence time may be influenced by link failure
   indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP
   flood packet pacing, SPF delay, SPF execution time, and routing and
   forwarding tables update time [Po09a].



Poretsky, et al.         Expires April 29, 2010                [Page 24]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


8.1.4.  Convergence Due to ECMP Member Remote Interface Failure

   Objective

   To obtain the IGP convergence time due to a Remote Interface link
   failure event for an ECMP Member.

   Procedure

   1.   Advertise an IGP topology from Tester to DUT using the test
        setup shown in Figure 4.

   2.   Send Offered Load from Tester to DUT on ingress interface.

   3.   Verify traffic is forwarded over the DUT's ECMP member interface
        that will be failed in the next step.

   4.   Remove link on Tester's interface to R2.  This is the
        Convergence Event Trigger.

   5.   Measure First Route Convergence Time.

   6.   Measure Full Convergence Time.

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times and Loss-Derived
        Convergence Time.  At the same time measure Out-of-Order Packets
        [Po06] and Duplicate Packets [Po06].

   9.   Wait sufficient time for queues to drain.

   10.  Restart Offered Load.

   11.  Restore link on Tester's interface to R2.

   12.  Measure First Route Convergence Time.

   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.  At the same time measure Out-of-Order Packets [Po06]
        and Duplicate Packets [Po06].

   Results



Poretsky, et al.         Expires April 29, 2010                [Page 25]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   The measured IGP convergence time may influenced by the link failure
   indication time, LSA/LSP delay, LSA/LSP generation time, LSA/LSP
   flood packet pacing, SPF delay, SPF execution time, and routing and
   forwarding tables update time.  This test case may produce Stale
   Forwarding [Po09t] due to a transient microloop between R1 and R2
   during convergence, which may increase the measured convergence times
   and loss of connectivity periods.

8.1.5.  Convergence Due to Parallel Link Interface Failure

   Objective

   To obtain the IGP convergence due to a local link failure event for a
   member of a parallel link.  The links can be used for data load
   balancing

   Procedure

   1.   Advertise an IGP topology from Tester to DUT using the test
        setup shown in Figure 5.

   2.   Send Offered Load from Tester to DUT on ingress interface.

   3.   Verify traffic is forwarded over the parallel link member that
        will be failed in the next step.

   4.   Remove link on one of the DUT's parallel link member interfaces.
        This is the Convergence Event.

   5.   Measure First Route Convergence Time.

   6.   Measure Full Convergence Time.

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times and Loss-Derived
        Convergence Time.  At the same time measure Out-of-Order Packets
        [Po06] and Duplicate Packets [Po06].

   9.   Wait sufficient time for queues to drain.

   10.  Restart Offered Load.

   11.  Restore link on DUT's Parallel Link member interface.

   12.  Measure First Route Convergence Time.





Poretsky, et al.         Expires April 29, 2010                [Page 26]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.  At the same time measure Out-of-Order Packets [Po06]
        and Duplicate Packets [Po06].

   Results

   The measured IGP convergence time may be influenced by the link
   failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/
   LSP flood packet pacing, SPF delay, SPF execution time, and routing
   and forwarding tables update time [Po09a].

8.2.  Other failures

8.2.1.  Convergence Due to Layer 2 Session Loss

   Objective

   To obtain the IGP convergence time due to a local layer 2 loss.

   Procedure

   1.   Advertise an IGP topology from Tester to DUT using the topology
        shown in Figure 1.

   2.   Send Offered Load from Tester to DUT on ingress interface.

   3.   Verify traffic is routed over Preferred Egress Interface.

   4.   Remove Layer 2 session from DUT's Preferred Egress Interface.
        This is the Convergence Event.

   5.   Measure First Route Convergence Time.

   6.   Measure Full Convergence Time.

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   9.   Wait sufficient time for queues to drain.




Poretsky, et al.         Expires April 29, 2010                [Page 27]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   10.  Restart Offered Load.

   11.  Restore Layer 2 session on DUT's Preferred Egress Interface.

   12.  Measure First Route Convergence Time.

   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   Results

   The measured IGP Convergence time may be influenced by the Layer 2
   failure indication time, LSA/LSP delay, LSA/LSP generation time, LSA/
   LSP flood packet pacing, SPF delay, SPF execution time, and routing
   and forwarding tables update time [Po09a].

   Discussion

   Configure IGP timers such that the IGP adjacency does not time out
   before layer 2 failure is detected.

   To measure convergence time, traffic SHOULD start dropping on the
   Preferred Egress Interface on the instant the layer 2 session is
   removed.  Alternatively the Tester SHOULD record the time the instant
   layer 2 session is removed and traffic loss SHOULD only be measured
   on the Next-Best Egress Interface.  For loss-derived benchmarks the
   time of the Start Traffic Instant SHOULD be recorded as well.  See
   Section 4.1.

8.2.2.  Convergence Due to Loss of IGP Adjacency

   Objective

   To obtain the IGP convergence time due to loss of an IGP Adjacency.

   Procedure

   1.   Advertise an IGP topology from Tester to DUT using the topology
        shown in Figure 1.

   2.   Send Offered Load from Tester to DUT on ingress interface.





Poretsky, et al.         Expires April 29, 2010                [Page 28]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   3.   Verify traffic is routed over Preferred Egress Interface.

   4.   Remove IGP adjacency from the Preferred Egress Interface while
        the layer 2 session MUST be maintained.  This is the Convergence
        Event.

   5.   Measure First Route Convergence Time.

   6.   Measure Full Convergence Time.

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   9.   Wait sufficient time for queues to drain.

   10.  Restart Offered Load.

   11.  Restore IGP session on DUT's Preferred Egress Interface.

   12.  Measure First Route Convergence Time.

   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   Results

   The measured IGP Convergence time may be influenced by the IGP Hello
   Interval, IGP Dead Interval, LSA/LSP delay, LSA/LSP generation time,
   LSA/LSP flood packet pacing, SPF delay, SPF execution time, and
   routing and forwarding tables update time [Po09a].

   Discussion

   Configure layer 2 such that layer 2 does not time out before IGP
   adjacency failure is detected.

   To measure convergence time, traffic SHOULD start dropping on the
   Preferred Egress Interface on the instant the IGP adjacency is
   removed.  Alternatively the Tester SHOULD record the time the instant
   the IGP adjacency is removed and traffic loss SHOULD only be measured



Poretsky, et al.         Expires April 29, 2010                [Page 29]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   on the Next-Best Egress Interface.  For loss-derived benchmarks the
   time of the Start Traffic Instant SHOULD be recorded as well.  See
   Section 4.1.

8.2.3.  Convergence Due to Route Withdrawal

   Objective

   To obtain the IGP convergence time due to route withdrawal.

   Procedure

   1.   Advertise an IGP topology from Tester to DUT using the topology
        shown in Figure 1.  The routes that will be withdrawn MUST be a
        set of leaf routes advertised by at least two nodes in the
        emulated topology.  The topology SHOULD be such that before the
        withdrawal the DUT prefers the leaf routes advertised by a node
        "nodeA" via the Preferred Egress Interface, and after the
        withdrawal the DUT prefers the leaf routes advertised by a node
        "nodeB" via the Next-Best Egress Interface.

   2.   Send Offered Load from Tester to DUT on Ingress Interface.

   3.   Verify traffic is routed over Preferred Egress Interface.

   4.   The Tester withdraws the set of IGP leaf routes from nodeA.
        This is the Convergence Event.  The withdrawal update message
        SHOULD be a single unfragmented packet.  If the routes cannot be
        withdrawn by a single packet, the messages SHOULD be sent using
        the same pacing characteristics as the DUT.  The Tester MAY
        record the time it sends the withdrawal message(s).

   5.   Measure First Route Convergence Time.

   6.   Measure Full Convergence Time.

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   9.   Wait sufficient time for queues to drain.

   10.  Restart Offered Load.

   11.  Re-advertise the set of withdrawn IGP leaf routes from nodeA
        emulated by the Tester.  The update message SHOULD be a single



Poretsky, et al.         Expires April 29, 2010                [Page 30]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


        unfragmented packet.  If the routes cannot be advertised by a
        single packet, the messages SHOULD be sent using the same pacing
        characteristics as the DUT.  The Tester MAY record the time it
        sends the update message(s).

   12.  Measure First Route Convergence Time.

   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   Results

   The measured IGP convergence time is influenced by SPF or route
   calculation delay, SPF or route calculation execution time, and
   routing and forwarding tables update time [Po09a].

   Discussion

   To measure convergence time, traffic SHOULD start dropping on the
   Preferred Egress Interface on the instant the routes are withdrawn by
   the Tester.  Alternatively the Tester SHOULD record the time the
   instant the routes are withdrawn and traffic loss SHOULD only be
   measured on the Next-Best Egress Interface.  For loss-derived
   benchmarks the time of the Start Traffic Instant SHOULD be recorded
   as well.  See Section 4.1.

8.3.  Administrative changes

8.3.1.  Convergence Due to Local Adminstrative Shutdown

   Objective

   To obtain the IGP convergence time due to taking the DUT's Local
   Interface administratively out of service.

   Procedure

   1.   Advertise an IGP topology from Tester to DUT using the topology
        shown in Figure 1.

   2.   Send Offered Load from Tester to DUT on ingress interface.





Poretsky, et al.         Expires April 29, 2010                [Page 31]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   3.   Verify traffic is routed over Preferred Egress Interface.

   4.   Take the DUT's Preferred Egress Interface administratively out
        of service.  This is the Convergence Event.

   5.   Measure First Route Convergence Time.

   6.   Measure Full Convergence Time.

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   9.   Wait sufficient time for queues to drain.

   10.  Restart Offered Load.

   11.  Restore Preferred Egress Interface by administratively enabling
        the interface.

   12.  Measure First Route Convergence Time.

   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   16.  It is possible that no measured packet loss will be observed for
        this test case.

   Results

   The measured IGP Convergence time may be influenced by LSA/LSP delay,
   LSA/LSP generation time, LSA/LSP flood packet pacing, SPF delay, SPF
   execution time, and routing and forwarding tables update time
   [Po09a].

8.3.2.  Convergence Due to Cost Change

   Objective

   To obtain the IGP convergence time due to route cost change.




Poretsky, et al.         Expires April 29, 2010                [Page 32]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   Procedure

   1.   Advertise an IGP topology from Tester to DUT using the topology
        shown in Figure 1.

   2.   Send Offered Load from Tester to DUT on ingress interface.

   3.   Verify traffic is routed over Preferred Egress Interface.

   4.   The Tester, emulating the neighbor node, increases the cost for
        all IGP routes at DUT's Preferred Egress Interface so that the
        Next-Best Egress Interface becomes preferred path.  The update
        message advertising the higher cost MUST be a single
        unfragmented packet.  This is the Convergence Event.  The Tester
        MAY record the time it sends the update message advertising the
        higher cost on the Preferred Egress Interface.

   5.   Measure First Route Convergence Time.

   6.   Measure Full Convergence Time.

   7.   Stop Offered Load.

   8.   Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   9.   Wait sufficient time for queues to drain.

   10.  Restart Offered Load.

   11.  The Tester, emulating the neighbor node, decreases the cost for
        all IGP routes at DUT's Preferred Egress Interface so that the
        Preferred Egress Interface becomes preferred path.  The update
        message advertising the lower cost MUST be a single unfragmented
        packet.

   12.  Measure First Route Convergence Time.

   13.  Measure Full Convergence Time.

   14.  Stop Offered Load.

   15.  Measure Route-Specific Convergence Times, Loss-Derived
        Convergence Time, Route LoC Periods, and Loss-Derived LoC
        Period.

   Results



Poretsky, et al.         Expires April 29, 2010                [Page 33]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


   The measured IGP Convergence time may be influenced by SPF delay, SPF
   execution time, and routing and forwarding tables update time
   [Po09a].

   Discussion

   To measure convergence time, traffic SHOULD start dropping on the
   Preferred Egress Interface on the instant the cost is changed by the
   Tester.  Alternatively the Tester SHOULD record the time the instant
   the cost is changed and traffic loss SHOULD only be measured on the
   Next-Best Egress Interface.  For loss-derived benchmarks the time of
   the Start Traffic Instant SHOULD be recorded as well.  See Section
   4.1.


9.  Security Considerations

   Benchmarking activities as described in this memo are limited to
   technology characterization using controlled stimuli in a laboratory
   environment, with dedicated address space and the constraints
   specified in the sections above.

   The benchmarking network topology will be an independent test setup
   and MUST NOT be connected to devices that may forward the test
   traffic into a production network, or misroute traffic to the test
   management network.

   Further, benchmarking is performed on a "black-box" basis, relying
   solely on measurements observable external to the DUT/SUT.

   Special capabilities SHOULD NOT exist in the DUT/SUT specifically for
   benchmarking purposes.  Any implications for network security arising
   from the DUT/SUT SHOULD be identical in the lab and in production
   networks.


10.  IANA Considerations

   This document requires no IANA considerations.


11.  Acknowledgements

   Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward,
   Peter De Vriendt and the BMWG for their contributions to this work.






Poretsky, et al.         Expires April 29, 2010                [Page 34]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


12.  Normative References

   [Br91]   Bradner, S., "Benchmarking terminology for network
            interconnection devices", RFC 1242, July 1991.

   [Br97]   Bradner, S., "Key words for use in RFCs to Indicate
            Requirement Levels", BCP 14, RFC 2119, March 1997.

   [Br99]   Bradner, S. and J. McQuaid, "Benchmarking Methodology for
            Network Interconnect Devices", RFC 2544, March 1999.

   [Ca90]   Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual
            environments", RFC 1195, December 1990.

   [Co08]   Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for
            IPv6", RFC 5340, July 2008.

   [Ho08]   Hopps, C., "Routing IPv6 with IS-IS", RFC 5308,
            October 2008.

   [Ko02]   Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample
            Metrics", RFC 3357, August 2002.

   [Ma98]   Mandeville, R., "Benchmarking Terminology for LAN Switching
            Devices", RFC 2285, February 1998.

   [Mo98]   Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998.

   [Po06]   Poretsky, S., Perser, J., Erramilli, S., and S. Khurana,
            "Terminology for Benchmarking Network-layer Traffic Control
            Mechanisms", RFC 4689, October 2006.

   [Po09a]  Poretsky, S., "Considerations for Benchmarking Link-State
            IGP Data Plane Route Convergence",
            draft-ietf-bmwg-igp-dataplane-conv-app-17 (work in
            progress), March 2009.

   [Po09t]  Poretsky, S. and B. Imhoff, "Terminology for Benchmarking
            Link-State IGP Data Plane Route Convergence",
            draft-ietf-bmwg-igp-dataplane-conv-term-18 (work in
            progress), July 2009.










Poretsky, et al.         Expires April 29, 2010                [Page 35]


Internet-Draft    IGP Convergence Benchmark Methodology     October 2009


Authors' Addresses

   Scott Poretsky
   Allot Communications
   67 South Bedford Street, Suite 400
   Burlington, MA  01803
   USA

   Phone: + 1 508 309 2179
   Email: sporetsky@allot.com


   Brent Imhoff
   Juniper Networks
   1194 North Mathilda Ave
   Sunnyvale, CA  94089
   USA

   Phone: + 1 314 378 2571
   Email: bimhoff@planetspork.com


   Kris Michielsen
   Cisco Systems
   6A De Kleetlaan
   Diegem, BRABANT  1831
   Belgium

   Email: kmichiel@cisco.com






















Poretsky, et al.         Expires April 29, 2010                [Page 36]