Terminology for Benchmarking LinkState IGP DataPlane Route Convergence
draftietfbmwgigpdataplaneconvterm23
The information below is for an old version of the document that is already published as an RFC.
Document  Type 
This is an older version of an InternetDraft that was ultimately published as RFC 6412.



Authors  Brent Imhoff , Scott Poretsky , Kris Michielsen  
Last updated  20151014 (Latest revision 20110216)  
RFC stream  Internet Engineering Task Force (IETF)  
Intended RFC status  Informational  
Formats  
Reviews  
Additional resources  Mailing list discussion  
Stream  WG state  WG Document  
Document shepherd  (None)  
IESG  IESG state  Became RFC 6412 (Informational)  
Action Holders 
(None)


Consensus boilerplate  Unknown  
Telechat date  (None)  
Responsible AD  Ron Bonica  
IESG note  
Send notices to  (None) 
draftietfbmwgigpdataplaneconvterm23
Network Working Group S. Poretsky InternetDraft Allot Communications Intended status: Informational B. Imhoff Expires: August 13, 2011 Juniper Networks K. Michielsen Cisco Systems February 16, 2011 Terminology for Benchmarking LinkState IGP Data Plane Route Convergence draftietfbmwgigpdataplaneconvterm23 Abstract This document describes the terminology for benchmarking linkstate Interior Gateway Protocol (IGP) route convergence. The terminology is to be used for benchmarking IGP convergence time through externally observable (black box) data plane measurements. The terminology can be applied to any linkstate IGP, such as Intermediate System to Intermediate System (ISIS) and Open Shortest Path First (OSPF). Status of this Memo This InternetDraft is submitted in full conformance with the provisions of BCP 78 and BCP 79. InternetDrafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as InternetDrafts. The list of current Internet Drafts is at http://datatracker.ietf.org/drafts/current/. InternetDrafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use InternetDrafts as reference material or to cite them other than as "work in progress." This InternetDraft will expire on August 13, 2011. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/licenseinfo) in effect on the date of publication of this document. Please review these documents Poretsky, et al. Expires August 13, 2011 [Page 1] InternetDraft IGP Convergence Benchmark Terminology February 2011 carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. Poretsky, et al. Expires August 13, 2011 [Page 2] InternetDraft IGP Convergence Benchmark Terminology February 2011 Table of Contents 1. Introduction and Scope . . . . . . . . . . . . . . . . . . . . 4 2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 4 3. Term Definitions . . . . . . . . . . . . . . . . . . . . . . . 4 3.1. Convergence Types . . . . . . . . . . . . . . . . . . . . 4 3.1.1. Route Convergence . . . . . . . . . . . . . . . . . . 5 3.1.2. Full Convergence . . . . . . . . . . . . . . . . . . . 5 3.2. Instants . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.2.1. Traffic Start Instant . . . . . . . . . . . . . . . . 6 3.2.2. Convergence Event Instant . . . . . . . . . . . . . . 6 3.2.3. Convergence Recovery Instant . . . . . . . . . . . . . 7 3.2.4. First Route Convergence Instant . . . . . . . . . . . 7 3.3. Transitions . . . . . . . . . . . . . . . . . . . . . . . 8 3.3.1. Convergence Event Transition . . . . . . . . . . . . . 8 3.3.2. Convergence Recovery Transition . . . . . . . . . . . 9 3.4. Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 9 3.4.1. Local Interface . . . . . . . . . . . . . . . . . . . 9 3.4.2. Remote Interface . . . . . . . . . . . . . . . . . . . 9 3.4.3. Preferred Egress Interface . . . . . . . . . . . . . . 10 3.4.4. NextBest Egress Interface . . . . . . . . . . . . . . 10 3.5. Benchmarking Methods . . . . . . . . . . . . . . . . . . . 11 3.5.1. RateDerived Method . . . . . . . . . . . . . . . . . 11 3.5.2. LossDerived Method . . . . . . . . . . . . . . . . . 13 3.5.3. RouteSpecific LossDerived Method . . . . . . . . . . 14 3.6. Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . 15 3.6.1. Full Convergence Time . . . . . . . . . . . . . . . . 15 3.6.2. First Route Convergence Time . . . . . . . . . . . . . 16 3.6.3. RouteSpecific Convergence Time . . . . . . . . . . . 17 3.6.4. LossDerived Convergence Time . . . . . . . . . . . . 18 3.6.5. Route Loss of Connectivity Period . . . . . . . . . . 19 3.6.6. LossDerived Loss of Connectivity Period . . . . . . . 20 3.7. Measurement Terms . . . . . . . . . . . . . . . . . . . . 21 3.7.1. Convergence Event . . . . . . . . . . . . . . . . . . 21 3.7.2. Convergence Packet Loss . . . . . . . . . . . . . . . 21 3.7.3. Connectivity Packet Loss . . . . . . . . . . . . . . . 22 3.7.4. Packet Sampling Interval . . . . . . . . . . . . . . . 22 3.7.5. Sustained Convergence Validation Time . . . . . . . . 23 3.7.6. Forwarding Delay Threshold . . . . . . . . . . . . . . 24 3.8. Miscellaneous Terms . . . . . . . . . . . . . . . . . . . 24 3.8.1. Impaired Packet . . . . . . . . . . . . . . . . . . . 24 4. Security Considerations . . . . . . . . . . . . . . . . . . . 24 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25 7. Normative References . . . . . . . . . . . . . . . . . . . . . 25 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26 Poretsky, et al. Expires August 13, 2011 [Page 3] InternetDraft IGP Convergence Benchmark Terminology February 2011 1. Introduction and Scope This document is a companion to [Po11m] which the methodology to be used for benchmarking linkstate Interior Gateway Protocol (IGP) Convergence by observing the data plane. The purpose of this document is to introduce new terms required to complete execution of the LinkState IGP Data Plane Route Convergence methodology [Po11m]. IGP convergence time is measured by observing the dataplane through the Device Under Test (DUT) at the Tester. The methodology and terminology to be used for benchmarking IGP Convergence can be applied to IPv4 and IPv6 traffic and linkstate IGPs such as Intermediate System to Intermediate System (ISIS) [Ca90][Ho08], Open Shortest Path First (OSPF) [Mo98][Co08], and others. 2. Existing Definitions This document uses existing terminology defined in other IETF documents. Examples include, but are not limited to: Throughput [Ref.[Br91], section 3.17] Offered Load [Ref.[Ma98], section 3.5.2] Forwarding Rate [Ref.[Ma98], section 3.6.1] Device Under Test (DUT) [Ref.[Ma98], section 3.1.1] System Under Test (SUT) [Ref.[Ma98], section 3.1.2] OutofOrder Packet [Ref.[Po06], section 3.3.4] Duplicate Packet [Ref.[Po06], section 3.3.5] Stream [Ref.[Po06], section 3.3.2] Forwarding Delay [Ref.[Po06], section 3.2.4] IP Packet Delay Variation (IPDV) [Ref.[De02], section 1.2] Loss Period [Ref.[Ko02], section 4] The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, RFC 2119 [Br97]. RFC 2119 defines the use of these key words to help make the intent of standards track documents as clear as possible. While this document uses these keywords, this document is not a standards track document. 3. Term Definitions 3.1. Convergence Types Poretsky, et al. Expires August 13, 2011 [Page 4] InternetDraft IGP Convergence Benchmark Terminology February 2011 3.1.1. Route Convergence Definition: The process of updating all components of the router, including the Routing Information Base (RIB) and Forwarding Information Base (FIB), along with software and hardware tables, with the most recent route change(s) such that forwarding for a route entry is successful on the NextBest Egress Interface [Section 3.4.4]. Discussion: In general IGP convergence does not necessarily result in a change in forwarding. But the test cases in [Po11m] are specified such that the IGP convergence results in a change of egress interface for the measurement dataplane traffic. Due to this property of the test case specifications, Route Convergence can be observed externally by the rerouting of the measurement dataplane traffic to the Nextbest Egress Interface [Section 3.4.4]. Measurement Units: N/A See Also: NextBest Egress Interface, Full Convergence 3.1.2. Full Convergence Definition: Route Convergence for all routes in the Forwarding Information Base (FIB). Discussion: In general IGP convergence does not necessarily result in a change in forwarding. But the test cases in [Po11m] are specified such that the IGP convergence results in a change of egress interface for the measurement dataplane traffic. Due to this property of the test cases specifications, Full Convergence can be observed externally by the rerouting of the measurement dataplane traffic to the Nextbest Egress Interface [Section 3.4.4]. Measurement Units: N/A See Also: NextBest Egress Interface, Route Convergence Poretsky, et al. Expires August 13, 2011 [Page 5] InternetDraft IGP Convergence Benchmark Terminology February 2011 3.2. Instants 3.2.1. Traffic Start Instant Definition: The time instant the Tester sends out the first data packet to the Device Under Test (DUT). Discussion: If using the LossDerived Method [Section 3.5.2] or the Route Specific LossDerived Method [Section 3.5.3] to benchmark IGP convergence time, and the applied Convergence Event [Section 3.7.1] does not cause instantaneous traffic loss for all routes at the Convergence Event Instant [Section 3.2.2] then the Tester SHOULD collect a timestamp on the Traffic Start Instant in order to measure the period of time between the Traffic Start Instant and Convergence Event Instant. Measurement Units: seconds (and fractions), reported with resolution sufficient to distinguish between different instants See Also: LossDerived Method, RouteSpecific LossDerived Method, Convergence Event, Convergence Event Instant 3.2.2. Convergence Event Instant Definition: The time instant that a Convergence Event [Section 3.7.1] occurs. Discussion: If the Convergence Event [Section 3.7.1] causes instantaneous traffic loss on the Preferred Egress Interface [Section 3.4.3], the Convergence Event Instant is observable from the data plane as the instant that no more packets are received on the Preferred Egress Interface. The Tester SHOULD collect a timestamp on the Convergence Event Instant if it the Convergence Event does not cause instantaneous traffic loss on the Preferred Egress Interface [Section 3.4.3]. Poretsky, et al. Expires August 13, 2011 [Page 6] InternetDraft IGP Convergence Benchmark Terminology February 2011 Measurement Units: seconds (and fractions), reported with resolution sufficient to distinguish between different instants See Also: Convergence Event, Preferred Egress Interface 3.2.3. Convergence Recovery Instant Definition: The time instant that Full Convergence [Section 3.1.2] has completed. Discussion: The Full Convergence completed state MUST be maintained for an interval of duration equal to the Sustained Convergence Validation Time [Section 3.7.5] in order to validate the Convergence Recovery Instant. The Convergence Recovery Instant is observable from the data plane as the instant the Device Under Test (DUT) forwards traffic to all destinations over the NextBest Egress Interface [Section 3.4.4] without impairments. Measurement Units: seconds (and fractions), reported with resolution sufficient to distinguish between different instants See Also: Sustained Convergence Validation Time, Full Convergence, NextBest Egress Interface 3.2.4. First Route Convergence Instant Definition: The time instant the first route entry completes Route Convergence [Section 3.1.1] Discussion: Any route may be the first to complete Route Convergence. The First Route Convergence Instant is observable from the data plane as the Poretsky, et al. Expires August 13, 2011 [Page 7] InternetDraft IGP Convergence Benchmark Terminology February 2011 instant that the first packet that is not an Impaired Packet [Section 3.8.1] is received from the NextBest Egress Interface [Section 3.4.4] or, for the test cases with Equal Cost MultiPath (ECMP) or Parallel Links, the instant that the Forwarding Rate on the NextBest Egress Interface [Section 3.4.4] starts to increase. Measurement Units: seconds (and fractions), reported with resolution sufficient to distinguish between different instants See Also: Route Convergence, Impaired Packet, NextBest Egress Interface 3.3. Transitions 3.3.1. Convergence Event Transition Definition: A time interval following a Convergence Event [Section 3.7.1] in which Forwarding Rate on the Preferred Egress Interface [Section 3.4.3] gradually reduces to zero. Discussion: The Forwarding Rate during a Convergence Event Transition may or may not decrease linearly. The Forwarding Rate observed on the Device Under Test (DUT) egress interface(s) may or may not decrease to zero. The Offered Load, the number of routes, and the Packet Sampling Interval [Section 3.7.4] influence the observations of the Convergence Event Transition using the RateDerived Method [Section 3.5.1]. Measurement Units: seconds (and fractions) See Also: Convergence Event, Preferred Egress Interface, Packet Sampling Interva, RateDerived Method Poretsky, et al. Expires August 13, 2011 [Page 8] InternetDraft IGP Convergence Benchmark Terminology February 2011 3.3.2. Convergence Recovery Transition Definition: A time interval following the First Route Convergence Instant [Section 3.4.4] in which Forwarding Rate on the Device Under Test (DUT) egress interface(s) gradually increases to equal the Offered Load. Discussion: The Forwarding Rate observed during a Convergence Recovery Transition may or may not increase linearly. The Offered Load, the number of routes, and the Packet Sampling Interval [Section 3.7.4] influence the observations of the Convergence Recovery Transition using the RateDerived Method [Section 3.5.1]. Measurement Units: seconds (and fractions) See Also: First Route Convergence Instant, Packet Sampling Interva, Rate Derived Method 3.4. Interfaces 3.4.1. Local Interface Definition: An interface on the Device Under Test (DUT). Discussion: A failure of a Local Interface indicates that the failure occurred directly on the Device Under Test (DUT). Measurement Units: N/A See Also: Remote Interface 3.4.2. Remote Interface Definition: An interface on a neighboring router that is not directly connected Poretsky, et al. Expires August 13, 2011 [Page 9] InternetDraft IGP Convergence Benchmark Terminology February 2011 to any interface on the Device Under Test (DUT). Discussion: A failure of a Remote Interface indicates that the failure occurred on a neighbor router's interface that is not directly connected to the Device Under Test (DUT). Measurement Units: N/A See Also: Local Interface 3.4.3. Preferred Egress Interface Definition: The outbound interface from the Device Under Test (DUT) for traffic routed to the preferred nexthop. Discussion: The Preferred Egress Interface is the egress interface prior to a Convergence Event [Section 3.7.1]. Measurement Units: N/A See Also: Convergence Event, NextBest Egress Interface 3.4.4. NextBest Egress Interface Definition: The outbound interface or set of outbound interfaces in an Equal Cost Multipath (ECMP) set or parallel link set of the Device Under Test (DUT) for traffic routed to the secondbest nexthop. Discussion: The NextBest Egress Interface becomes the egress interface after a Convergence Event [Section 3.4.4]. For the test cases in [Po11m] using test topologies with an ECMP set or parallel link set, the term Preferred Egress Interface refers to all members of the link set. Measurement Units: N/A See Also: Convergence Event, Preferred Egress Interface Poretsky, et al. Expires August 13, 2011 [Page 10] InternetDraft IGP Convergence Benchmark Terminology February 2011 3.5. Benchmarking Methods 3.5.1. RateDerived Method Definition: The method to calculate convergence time benchmarks from observing Forwarding Rate each Packet Sampling Interval [Section 3.7.4]. Discussion: Figure 1 shows an example of the Forwarding Rate change in time during convergence as observed when using the RateDerived Method. ^ Traffic Convergence Fwd  Start Recovery Rate  Instant Instant  Offered ^ ^  Load > \ /  \ /< Convergence  \ Packet / Recovery  Convergence >\ Loss / Transition  Event \ /  Transition \/ < Max Packet Loss  +> ^ ^ time Convergence First Route Event Instant Convergence Instant Figure 1: RateDerived Convergence Graph To enable collecting statistics of OutofOrder Packets per flow (See [Th00], Section 3) the Offered Load SHOULD consist of multiple Streams [Po06] and each Stream SHOULD consist of a single flow . If sending multiple Streams, the measured traffic statistics for all Streams MUST be added together. The destination addresses for the Offered Load MUST be distributed such that all routes or a statistically representative subset of all routes are matched and each of these routes is offered an equal share of the Offered Load. It is RECOMMENDED to send traffic to all routes, but a statistically representative subset of all routes can be used if required. At least one packet per route for all routes matched in the Offered Poretsky, et al. Expires August 13, 2011 [Page 11] InternetDraft IGP Convergence Benchmark Terminology February 2011 Load MUST be offered to the DUT within each Packet Sampling Interval. For maximum accuracy the value for the Packet Sampling Interval SHOULD be as small as possible, but the presence of IP Packet Delay Variation (IPDV) [De02] may enforce using a larger Packet Sampling Interval. The Offered Load, IPDV, the number of routes, and the Packet Sampling Interval influence the observations for the RateDerived Method. It may be difficult to identify the different convergence time instants in the RateDerived Convergence Graph. For example, it is possible that a Convergence Event causes the Forwarding Rate to drop to zero, while this may not be observed in the Forwarding Rate measurements if the Packet Sampling Interval is too large. IPDV causes fluctuations in the number of received packets during each Packet Sampling Interval. To account for the presence of IPDV in determining if a convergence instant has been reached, Forwarding Delay SHOULD be observed during each Packet Sampling Interval. The minimum and maximum number of packets expected in a Packet Sampling Interval in presence of IPDV can be calculated with Equation 1. number of packets expected in a Packet Sampling Interval in presence of IP Packet Delay Variation = expected number of packets without IP Packet Delay Variation +/( (maxDelay  minDelay) * Offered Load) with minDelay and maxDelay the minimum resp. maximum Forwarding Delay of packets received during the Packet Sampling Interval Equation 1 To determine if a convergence instant has been reached the number of packets received in a Packet Sampling Interval is compared with the range of expected number of packets calculated in Equation 1. If packets are going over multiple ECMP members and one or more of the members has failed then the number of received packets during each Packet Sampling Interval may vary, even excluding presence of IPDV. To prevent fluctuation of the number of received packets during each Packet Sampling Interval for this reason, the Packet Sampling Interval duration SHOULD be a whole multiple of the time between two consecutive packets sent to the same destination. Metrics measured at the Packet Sampling Interval MUST include Forwarding Rate and Impaired Packet count. To measure convergence time benchmarks for Convergence Events [Section 3.7.1] that do not cause instantaneous traffic loss for all routes at the Convergence Event Instant, the Tester SHOULD collect a Poretsky, et al. Expires August 13, 2011 [Page 12] InternetDraft IGP Convergence Benchmark Terminology February 2011 timestamp of the Convergence Event Instant [Section 3.2.2] and the Tester SHOULD observe Forwarding Rate separately on the NextBest Egress Interface. Since the RateDerived Method does not distinguish between individual traffic destinations, it SHOULD NOT be used for any route specific measurements. Therefor RateDerived Method SHOULD NOT be used to benchmark Route Loss of Connectivity Period [Section 3.6.5]. Measurement Units: N/A See Also: Packet Sampling Interval, Convergence Event, Convergence Event Instant, NextBest Egress Interface, Route Loss of Connectivity Period 3.5.2. LossDerived Method Definition: The method to calculate the LossDerived Convergence Time [Section 3.6.4] and LossDerived Loss of Connectivity Period [Section 3.6.6] benchmarks from the amount of Impaired Packets [Section 3.8.1]. Discussion: To enable collecting statistics of OutofOrder Packets per flow (See [Th00], Section 3) the Offered Load SHOULD consist of multiple Streams [Po06] and each Stream SHOULD consist of a single flow . If sending multiple Streams, the measured traffic statistics for all Streams MUST be added together. The destination addresses for the Offered Load MUST be distributed such that all routes or a statistically representative subset of all routes are matched and each of these routes is offered an equal share of the Offered Load. It is RECOMMENDED to send traffic to all routes, but a statistically representative subset of all routes can be used if required. LossDerived Method SHOULD always be combined with RateDerived Method in order to observe Full Convergence completion. The total amount of Convergence Packet Loss is collected after Full Convergence completion. To measure convergence time and loss of connectivity benchmarks for Convergence Events that cause instantaneous traffic loss for all Poretsky, et al. Expires August 13, 2011 [Page 13] InternetDraft IGP Convergence Benchmark Terminology February 2011 routes at the Convergence Event Instant, the Tester SHOULD observe Impaired Packet count on all DUT egress interfaces (see Connectivity Packet Loss [Section 3.7.3]). To measure convergence time benchmarks for Convergence Events that do not cause instantaneous traffic loss for all routes at the Convergence Event Instant, the Tester SHOULD collect timestamps of the Start Traffic Instant and of the Convergence Event Instant, and the Tester SHOULD observe Impaired Packet count separately on the NextBest Egress Interface (See Convergence Packet Loss [Section 3.7.2]). Since LossDerived Method does not distinguish between traffic destinations and the Impaired Packet statistics are only collected after Full Convergence completion, this method can only be used to measure average values over all routes. For these reasons Loss Derived Method can only be used to benchmark LossDerived Convergence Time [Section 3.6.4] and LossDerived Loss of Connectivity Period [Section 3.6.6]. Note that the LossDerived Method measures an average over all routes, including the routes that may not be impacted by the Convergence Event, such as routes via nonimpacted members of ECMP or parallel links. Measurement Units: N/A See Also: LossDerived Convergence Time, LossDerived Loss of Connectivity Period, Connectivity Packet Loss, Convergence Packet Loss 3.5.3. RouteSpecific LossDerived Method Definition: The method to calculate the RouteSpecific Convergence Time [Section 3.6.3] benchmark from the amount of Impaired Packets [Section 3.8.1] during convergence for a specific route entry. Discussion: To benchmark RouteSpecific Convergence Time, the Tester provides an Offered Load that consists of multiple Streams [Po06]. Each Stream has a single destination address matching a different route entry, for all routes or a statistically representative subset of all routes. Each Stream SHOULD consist of a single flow (See [Th00], Section 3). Convergence Packet Loss is measured for each Stream Poretsky, et al. Expires August 13, 2011 [Page 14] InternetDraft IGP Convergence Benchmark Terminology February 2011 separately. RouteSpecific LossDerived Method SHOULD always be combined with RateDerived Method in order to observe Full Convergence completion. The total amount of Convergence Packet Loss [Section 3.7.2] for each Stream is collected after Full Convergence completion. RouteSpecific LossDerived Method is the RECOMMENDED method to measure convergence time benchmarks. To measure convergence time and loss of connectivity benchmarks for Convergence Events that cause instantaneous traffic loss for all routes at the Convergence Event Instant, the Tester SHOULD observe Impaired Packet count on all DUT egress interfaces (see Connectivity Packet Loss [Section 3.7.3]). To measure convergence time benchmarks for Convergence Events that do not cause instantaneous traffic loss for all routes at the Convergence Event Instant, the Tester SHOULD collect timestamps of the Start Traffic Instant and of the Convergence Event Instant, and the Tester SHOULD observe packet loss separately on the NextBest Egress Interface (See Convergence Packet Loss [Section 3.7.2]). Since RouteSpecific LossDerived Method uses traffic streams to individual routes, it observes Impaired Packet count as it would be experienced by a network user. For this reason RouteSpecific Loss Derived Method is RECOMMENDED to measure RouteSpecific Convergence Time benchmarks and Route Loss of Connectivity Period benchmarks. Measurement Units: N/A See Also: RouteSpecific Convergence Time, Route Loss of Connectivity Period, Connectivity Packet Loss, Convergence Packet Loss 3.6. Benchmarks 3.6.1. Full Convergence Time Definition: The time duration of the period between the Convergence Event Instant and the Convergence Recovery Instant as observed using the Rate Derived Method. Discussion: Poretsky, et al. Expires August 13, 2011 [Page 15] InternetDraft IGP Convergence Benchmark Terminology February 2011 Using the RateDerived Method, Full Convergence Time can be calculated as the time difference between the Convergence Event Instant and the Convergence Recovery Instant, as shown in Equation 2. Full Convergence Time = Convergence Recovery Instant  Convergence Event Instant Equation 2 The Convergence Event Instant can be derived from the Forwarding Rate observation or from a timestamp collected by the Tester. For the test cases described in [Po11m], it is expected that Full Convergence Time equals the maximum RouteSpecific Convergence Time when benchmarking all routes in FIB using the RouteSpecific Loss Derived Method. It is not possible to measure Full Convergence Time using the Loss Derived Method. Measurement Units: seconds (and fractions) See Also: Full Convergence, RateDerived Method, RouteSpecific LossDerived Method, Convergence Event Instant, Convergence Recovery Instant 3.6.2. First Route Convergence Time Definition: The duration of the period between the Convergence Event Instant and the First Route Convergence Instant as observed using the Rate Derived Method. Discussion: Using the RateDerived Method, First Route Convergence Time can be calculated as the time difference between the Convergence Event Instant and the First Route Convergence Instant, as shown with Equation 3. First Route Convergence Time = First Route Convergence Instant  Convergence Event Instant Equation 3 The Convergence Event Instant can be derived from the Forwarding Rate Poretsky, et al. Expires August 13, 2011 [Page 16] InternetDraft IGP Convergence Benchmark Terminology February 2011 observation or from a timestamp collected by the Tester. For the test cases described in [Po11m], it is expected that First Route Convergence Time equals the minimum RouteSpecific Convergence Time when benchmarking all routes in FIB using the RouteSpecific LossDerived Method. It is not possible to measure First Route Convergence Time using the LossDerived Method. Measurement Units: seconds (and fractions) See Also: RateDerived Method, RouteSpecific LossDerived Method, Convergence Event Instant, First Route Convergence Instant 3.6.3. RouteSpecific Convergence Time Definition: The amount of time it takes for Route Convergence to be completed for a specific route, as calculated from the amount of Impaired Packets [Section 3.8.1] during convergence for a single route entry. Discussion: RouteSpecific Convergence Time can only be measured using the Route Specific LossDerived Method. If the applied Convergence Event causes instantaneous traffic loss for all routes at the Convergence Event Instant, Connectivity Packet Loss should be observed. Connectivity Packet Loss is the combined Impaired Packet count observed on Preferred Egress Interface and NextBest Egress Interface. When benchmarking RouteSpecific Convergence Time, Connectivity Packet Loss is measured and Equation 4 is applied for each measured route. The calculation is equal to Equation 8 in Section 3.6.5. RouteSpecific Convergence Time = Connectivity Packet Loss for specific route/Offered Load per route Equation 4 If the applied Convergence Event does not cause instantaneous traffic loss for all routes at the Convergence Event Instant, then the Tester SHOULD collect timestamps of the Traffic Start Instant and of the Convergence Event Instant, and the Tester SHOULD observe Convergence Poretsky, et al. Expires August 13, 2011 [Page 17] InternetDraft IGP Convergence Benchmark Terminology February 2011 Packet Loss separately on the NextBest Egress Interface. When benchmarking RouteSpecific Convergence Time, Convergence Packet Loss is measured and Equation 5 is applied for each measured route. RouteSpecific Convergence Time = Convergence Packet Loss for specific route/Offered Load per route  (Convergence Event Instant  Traffic Start Instant) Equation 5 The RouteSpecific Convergence Time benchmarks enable minimum, maximum, average, and median convergence time measurements to be reported by comparing the results for the different route entries. It also enables benchmarking of convergence time when configuring a priority value for route entry(ies). Since multiple RouteSpecific Convergence Times can be measured it is possible to have an array of results. The format for reporting RouteSpecific Convergence Time is provided in [Po11m]. Measurement Units: seconds (and fractions) See Also: RouteSpecific LossDerived Method, Convergence Event, Convergence Event Instant, Convergence Packet Loss, Connectivity Packet Loss, Route Convergence 3.6.4. LossDerived Convergence Time Definition: The average Route Convergence time for all routes in the Forwarding Information Base (FIB), as calculated from the amount of Impaired Packets [Section 3.8.1] during convergence. Discussion: LossDerived Convergence Time is measured using the LossDerived Method. If the applied Convergence Event causes instantaneous traffic loss for all routes at the Convergence Event Instant, Connectivity Packet Loss [Section 3.7.3] should be observed. Connectivity Packet Loss is the combined Impaired Packet count observed on Preferred Egress Interface and NextBest Egress Interface. When benchmarking Loss Derived Convergence Time, Connectivity Packet Loss is measured and Equation 6 is applied. Poretsky, et al. Expires August 13, 2011 [Page 18] InternetDraft IGP Convergence Benchmark Terminology February 2011 LossDerived Convergence Time = Connectivity Packet Loss/Offered Load Equation 6 If the applied Convergence Event does not cause instantaneous traffic loss for all routes at the Convergence Event Instant, then the Tester SHOULD collect timestamps of the Start Traffic Instant and of the Convergence Event Instant and the Tester SHOULD observe Convergence Packet Loss [Section 3.7.2] separately on the NextBest Egress Interface. When benchmarking LossDerived Convergence Time, Convergence Packet Loss is measured and Equation 7 is applied. LossDerived Convergence Time = Convergence Packet Loss/Offered Load  (Convergence Event Instant  Traffic Start Instant) Equation 7 Measurement Units: seconds (and fractions) See Also: Convergence Packet Loss, Connectivity Packet Loss, Route Convergence, LossDerived Method 3.6.5. Route Loss of Connectivity Period Definition: The time duration of packet impairments for a specific route entry following a Convergence Event until Full Convergence completion, as observed using the RouteSpecific LossDerived Method. Discussion: In general the Route Loss of Connectivity Period is not equal to the RouteSpecific Convergence Time. If the DUT continues to forward traffic to the Preferred Egress Interface after the Convergence Event is applied then the Route Loss of Connectivity Period will be smaller than the RouteSpecific Convergence Time. This is also specifically the case after reversing a failure event. The Route Loss of Connectivity Period may be equal to the Route Specific Convergence Time if, as a characteristic of the Convergence Event, traffic for all routes starts dropping instantaneously on the Convergence Event Instant. See discussion in [Po11m]. Poretsky, et al. Expires August 13, 2011 [Page 19] InternetDraft IGP Convergence Benchmark Terminology February 2011 For the test cases described in [Po11m] the Route Loss of Connectivity Period is expected to be a single Loss Period [Ko02]. When benchmarking Route Loss of Connectivity Period, Connectivity Packet Loss is measured for each route and Equation 8 is applied for each measured route entry. The calculation is equal to Equation 4 in Section 3.6.3. Route Loss of Connectivity Period = Connectivity Packet Loss for specific route/Offered Load per route Equation 8 Route Loss of Connectivity Period SHOULD be measured using Route Specific LossDerived Method. Measurement Units: seconds (and fractions) See Also: RouteSpecific Convergence Time, RouteSpecific LossDerived Method, Connectivity Packet Loss 3.6.6. LossDerived Loss of Connectivity Period Definition: The average time duration of packet impairments for all routes following a Convergence Event until Full Convergence completion, as observed using the LossDerived Method. Discussion: In general the LossDerived Loss of Connectivity Period is not equal to the LossDerived Convergence Time. If the DUT continues to forward traffic to the Preferred Egress Interface after the Convergence Event is applied then the LossDerived Loss of Connectivity Period will be smaller than the LossDerived Convergence Time. This is also specifically the case after reversing a failure event. The LossDerived Loss of Connectivity Period may be equal to the LossDerived Convergence Time if, as a characteristic of the Convergence Event, traffic for all routes starts dropping instantaneously on the Convergence Event Instant. See discussion in [Po11m]. For the test cases described in [Po11m] each route's Route Loss of Poretsky, et al. Expires August 13, 2011 [Page 20] InternetDraft IGP Convergence Benchmark Terminology February 2011 Connectivity Period is expected to be a single Loss Period [Ko02]. When benchmarking LossDerived Loss of Connectivity Period, Connectivity Packet Loss is measured for all routes and Equation 9 is applied. The calculation is equal to Equation 6 in Section 3.6.4. LossDerived Loss of Connectivity Period = Connectivity Packet Loss for all routes/Offered Load Equation 9 LossDerived Loss of Connectivity Period SHOULD be measured using LossDerived Method. Measurement Units: seconds (and fractions) See Also: LossDerived Convergence Time, LossDerived Method, Connectivity Packet Loss 3.7. Measurement Terms 3.7.1. Convergence Event Definition: The occurrence of an event in the network that will result in a change in the egress interface of the Device Under Test (DUT) for routed packets. Discussion: All test cases in [Po11m] are defined such that a Convergence Event results in a change of egress interface of the DUT. Local or remote triggers that cause a route calculation which does not result in a change in forwarding are not considered. Measurement Units: N/A See Also: Convergence Event Instant 3.7.2. Convergence Packet Loss Definition: The number of Impaired Packets [Section 3.8.1] as observed on the NextBest Egress Interface of the DUT during convergence. Poretsky, et al. Expires August 13, 2011 [Page 21] InternetDraft IGP Convergence Benchmark Terminology February 2011 Discussion: An Impaired Packet is considered as a lost packet. Measurement Units: number of packets See Also: Connectivity Packet Loss 3.7.3. Connectivity Packet Loss Definition: The number of Impaired Packets observed on all DUT egress interfaces during convergence. Discussion: An Impaired Packet is considered as a lost packet. Connectivity Packet Loss is equal to Convergence Packet Loss if the Convergence Event causes instantaneous traffic loss for all egress interfaces of the DUT except for the NextBest Egress Interface. Measurement Units: number of packets See Also: Convergence Packet Loss 3.7.4. Packet Sampling Interval Definition: The interval at which the Tester (test equipment) polls to make measurements for arriving packets. Discussion: At least one packet per route for all routes matched in the Offered Load MUST be offered to the DUT within the Packet Sampling Interval. Metrics measured at the Packet Sampling Interval MUST include Forwarding Rate and received packets. Packet Sampling Interval can influence the convergence graph as observed with the RateDerived Method. This is particularly true when implementations complete Full Convergence in less time than the Packet Sampling Interval. The Convergence Event Instant and First Poretsky, et al. Expires August 13, 2011 [Page 22] InternetDraft IGP Convergence Benchmark Terminology February 2011 Route Convergence Instant may not be easily identifiable and the RateDerived Method may produce a larger than actual convergence time. Using a small Packet Sampling Interval in the presence of IPDV [De02] may cause fluctuations of the Forwarding Rate observation and can prevent correct observation of the different convergence time instants. The value of the Packet Sampling Interval only contributes to the measurement accuracy of the RateDerived Method. For maximum accuracy the value for the Packet Sampling Interval SHOULD be as small as possible, but the presence of IPDV may enforce using a larger Packet Sampling Interval. Measurement Units: seconds (and fractions) See Also: RateDerived Method 3.7.5. Sustained Convergence Validation Time Definition: The amount of time for which the completion of Full Convergence is maintained without additional Impaired Packets being observed. Discussion: The purpose of the Sustained Convergence Validation Time is to produce convergence benchmarks protected against fluctuation in Forwarding Rate after the completion of Full Convergence is observed. The RECOMMENDED Sustained Convergence Validation Time to be used is the time to send 5 consecutive packets to each destination with a minimum of 5 seconds. The Benchmarking Methodology Working Group (BMWG) selected 5 seconds based upon [Br99] which recommends waiting 2 seconds for residual frames to arrive (this is the Forwarding Delay Threshold for the last packet sent) and 5 seconds for DUT restabilization. Measurement Units: seconds (and fractions) See Also: Full Convergence, Convergence Recovery Instant Poretsky, et al. Expires August 13, 2011 [Page 23] InternetDraft IGP Convergence Benchmark Terminology February 2011 3.7.6. Forwarding Delay Threshold Definition: The maximum waiting time threshold used to distinguish between packets with very long delay and lost packets that will never arrive. Discussion: Applying a Forwarding Delay Threshold allows to consider packets with a too large Forwarding Delay as being lost, as is required for some applications (e.g. voice, video, etc.). The Forwarding Delay Threshold is a parameter of the methodology, and it MUST be reported. [Br99] recommends waiting 2 seconds for residual frames to arrive. Measurement Units: seconds (and fractions) See Also: Convergence Packet Loss, Connectivity Packet Loss 3.8. Miscellaneous Terms 3.8.1. Impaired Packet Definition: A packet that experienced at least one of the following impairments: loss, excessive Forwarding Delay, corruption, duplication, reordering. Discussion: A lost packet, a packet with a Forwarding Delay exceeding the Forwarding Delay Threshold, a corrupted packet, a Duplicate Packet [Po06], and an OutofOrder Packet [Po06] are Impaired Packets. Packet ordering is observed for each individual flow (See [Th00], Section 3) of the Offered Load. Measurement Units: N/A See Also: Forwarding Delay Threshold 4. Security Considerations Benchmarking activities as described in this memo are limited to Poretsky, et al. Expires August 13, 2011 [Page 24] InternetDraft IGP Convergence Benchmark Terminology February 2011 technology characterization using controlled stimuli in a laboratory environment, with dedicated address space and the constraints specified in the sections above. The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network, or misroute traffic to the test management network. Further, benchmarking is performed on a "blackbox" basis, relying solely on measurements observable external to the DUT/SUT. Special capabilities SHOULD NOT exist in the DUT/SUT specifically for benchmarking purposes. Any implications for network security arising from the DUT/SUT SHOULD be identical in the lab and in production networks. 5. IANA Considerations This document requires no IANA considerations. 6. Acknowledgements Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, Peter De Vriendt, Anuj Dewagan, Adrian Farrel, Stewart Bryant, Francis Dupont, and the Benchmarking Methodology Working Group for their contributions to this work. 7. Normative References [Br91] Bradner, S., "Benchmarking terminology for network interconnection devices", RFC 1242, July 1991. [Br97] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, March 1999. [Ca90] Callon, R., "Use of OSI ISIS for routing in TCP/IP and dual environments", RFC 1195, December 1990. [Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for IPv6", RFC 5340, July 2008. Poretsky, et al. Expires August 13, 2011 [Page 25] InternetDraft IGP Convergence Benchmark Terminology February 2011 [De02] Demichelis, C. and P. Chimento, "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)", RFC 3393, November 2002. [Ho08] Hopps, C., "Routing IPv6 with ISIS", RFC 5308, October 2008. [Ko02] Koodli, R. and R. Ravikanth, "Oneway Loss Pattern Sample Metrics", RFC 3357, August 2002. [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching Devices", RFC 2285, February 1998. [Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998. [Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, "Terminology for Benchmarking Networklayer Traffic Control Mechanisms", RFC 4689, October 2006. [Po11m] Poretsky, S., Imhoff, B., and K. Michielsen, "Benchmarking Methodology for LinkState IGP Data Plane Route Convergence", draftietfbmwgigpdataplaneconvmeth23 (work in progress), January 2011. [Th00] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and Multicast NextHop Selection", RFC 2991, November 2000. Authors' Addresses Scott Poretsky Allot Communications 67 South Bedford Street, Suite 400 Burlington, MA 01803 USA Phone: + 1 508 309 2179 Email: sporetsky@allot.com Poretsky, et al. Expires August 13, 2011 [Page 26] InternetDraft IGP Convergence Benchmark Terminology February 2011 Brent Imhoff Juniper Networks 1194 North Mathilda Ave Sunnyvale, CA 94089 USA Phone: + 1 314 378 2571 Email: bimhoff@planetspork.com Kris Michielsen Cisco Systems 6A De Kleetlaan Diegem, BRABANT 1831 Belgium Email: kmichiel@cisco.com Poretsky, et al. Expires August 13, 2011 [Page 27]