Considerations When Using Basic OSPF Convergence Benchmarks
RFC 4063

[Ballot discuss]
should there be evaluation of performance of illegal prefix
        filtering?  Note that in some contexts, this is an important
        security mechanism

[Ballot discuss]
draft-ietf-bmwg-ospfconv-intraarea-07.txt:

> 7.1. Time required to process an LSA
>
>
>      o    Using reference topology 1 (Emulated Topology), begin with
>            all links up and a full adjacency established between the DUT
>            and the generator.

The text should note the following:

  The generator does not have direct knowledge of the state of the adjacency on
  the DUT. The fact the adjacency may be in Full on the generator does not mean
  that the DUT is ready. It may still (and is likely to) be requesting LSAs
  from the generator. This process, involving processing of requested LSAs, will
  affect the results of the test. The generator should either wait until it sees
  the DUT's router-LSA listing the adjacency with the generator or introduce a
  configurable delay before starting the test.

>            The amount of time required for an OSPF implementation to
>            process the new LSA can be computed by subtracting
>            dupLSAprocTime from newLSAprocTime.

Note that this time may or may not include the time required to perform
flooding-related operations, depending on when the implementation sends the
ack--before it floods the LSA further or after, or anywhere in between.
In other words, this measurement may not mean the same thing in all
implementations.


> 7.2. Flooding Time

...

>            Two measurements can be taken from this test:
>
>          o    The time between the last LSA is received on the collector
>              from the generator and the time the last LSA is received
>              on the collector from the DUT.
>
>          o    The time between the last LSA is received on the collector
>              from the generator and the time the first LSA is received
>              on the collector from the DUT.

Not sure if the times mentioned in the second bullet are correct. At least it is
not obvious that this time difference would be non-negative and it would be
useful to know.


> 7.3. Shortest Path First Computation Time
>
...
>          o    Change the link cost between the generator and the emu-
>              lated network it is advertising.

To match the level of granularity of other parts of the text, should
this bullet also instruct the generator to send the new LSA describing
the changed part of topology to the DUT?

>          o    Immediately inject another LSA which is a duplicate of
>              some other LSA the generator has previously injected
>              (preferrably a stub network someplace within the emulated
>              network).

The doc should probably say that the generator should make sure that outbound
LSA packing is not performed for the duplicate LSAs and they are always sent in
a separate Link-state Update packet. Otherwise, if the LSA carrying the topo
change and the "stopwatch" duplicate LSA are in the same packet, the SPF will
be started _after_ the duplicate LSA is acked.

> 8.3. Forming adjacencies with Information Already in the Database
...
>
>          o    Note the time of the acknowledgement from the DUT for the
>              last LSA transmitted on the generator.
>
>            The time between the hello received from the DUT by the gen-
>            erator and the acknowledgement for the last LSA transmitted
>            by the generator should be taken as the total amount of time
>            required for the OSPF process on the DUT to build a FULL
>            neighbor adjacency with the set of LSAs injected. In this
>            test, the DUT is already aware of the entire network topol-
>            ogy, so the time required should only include the processing
>            of each LSA from the generator and transmitting an
>            acknowledgement.

This isn't true, actually. Because DUT already has the LSAs from the collector,
it will not even request these LSAs from the generator, and hence won't receive
then and won't send Acks. What will happen instead is after exchanging DBDs,
both DUT and generator will originate new router-LSAs where they will list the
new adjacency and then will async-flood these LSAs to each other. These LSAs
will indeed cause an Ack back that can be used to time the end of the process.
Alternatively, one could look for the end of the DBD process, i.e., the last
DBD packet.

> 8.4. Designated Router Election Time on A Broadcast Network
>
...
>
>            The time between the last hello received on R2 and the first
>            network LSA generated by the DUT should be taken as the
>            amount of time required for the DUT to complete a designated
>            router election computation. Note this test includes the dead
>            interval timer at the DUT, so this time may be factored out,
>            or the hello and dead intervals reduced to make these timers
>            impact the overall test times less.

If R1 could send a "good-bye" Hello packet, which some implementations
do when the OSPF process is being stopped, this would be even better.

> 8.5. Initial Convergence Time on a Broadcast Network, Test 1
>
...
>          o    The differential between the first hello and the first
>              network LSA is the time required by the DUT to converge on
>              this new topology.

The terminology doc says convergence includes route calculation.
In this case, origination of the first network LSA may very likely
happen before.

> 8.7. Link Down with Layer 2 Detection
...
>            The difference in the time between the initial link failure
>            and the receipt of the LSA on the generator across link 2
>            should be taken as the time required for an OSPF implementa-
>            tion to recognize and process a link failure.

In fact this also includes LSA flooding time (though to one neighbor only).
May this should be mentioned.

[Ballot comment]
These documents were reviewed by Brian Carpenter, Gen-ART.

They have not been checked against the ID-NITS list.

No table of contents, no copyright boilerplate. MUST NOT used
without definition. Minor errors in English. No Security Considerations
section.

Note on "applicability":

Despite the word "applicability" in the title, this is not an Applicability
Statement in the IETF sense. The IESG needs to be sure it doesn't have a problem
with that. It's really a set of health warnings about the following document.
It does therefore need to be published at the same time.

The -intraarea- document treats IPv6 as a "poor cousin". It should say that it's equally applicable to IPv4 and IPv6 (unless it is not).

The reason this is a Comment and not a Discuss is that others hold the token for getting the security sections right. The rest should be fixed at the same time.

[Ballot discuss]
draft-ietf-bmwg-ospfconv-term-07 has no Security Considerations section.

  The draft-ietf-bmwg-ospfconv-applicability Abstract needs to be rewritten.
  If the references are removed, then there is almost no meaning left.

  draft-ietf-bmwg-ospfconv-applicability-04 has no Security Considerations
  section.

[Ballot discuss]
no security considerations in two documents

        should there be evaluation of performance of illegal prefix
        filtering?  Note that in some contexts, this is an important
        security mechanism

From: Acee Lindem
Subject: OSPF Convergence BMWG Documents
Date: Thu, 02 Oct 2003 12:46:55 -0400
To: Vishwas Manral , Russ White
    , Aman Shaikh
Cc: Bill Fenner , Rohit Dube
    , Alex Zinin


    1. Abstract shouldn't references.

    2. Section 3 "control" rather than "contorl".

    3. Section 4  "set of parameters" rather than "set parameters".
        "ABR" rather than "abr".

    4. Section 6.4 - Should there be a reference for GateD?

    5. Section 8 - I found this section to suffer the the "Too deep a
topic in too
          little space" syndrome.  Specific comments:
     
          - Don't use "algo" as short for algorithm.
          - O(E) isn't defined before used. I'm assuming E stands for Edge.
          - Won't unreachable nodes that are directly adjacent have an
impact? I guess
              they are accounting the E calculation.
          -  "versus" rather than "verses".

    6. Section 9 - Reference P2P LAN draft when saying a  LAN can
simply be configured
          as P2P.  Also "at least" rather than "atleast".

Date: Thu, 25 Sep 2003 12:58:01 -0400
To: Randy Bush , Bert Wijnen
From: Al Morton
Subject:
Cc: kdubray@juniper.net

Randy/Bert,

The BWMG I-Ds on OSPF Convergence Benchmarking Terminology:
"OSPF Benchmarking Terminology and Concepts"
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-ospfconv-term-06.txt

Methodology:
"Benchmarking Basic OPSF Single Router Control Plane Convergence"
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-ospfconv-intraarea-06.txt

Applicability:
"Benchmarking Applicability for Basic OSPF Convergence"
http://www.ietf.org/internet-drafts/draft-ietf-bmwg-ospfconv-applicability-03.txt

are being forwarded to you for review as candidates for [Informational] RFCs.

These drafts have been actively reviewed and modified based on
multiple WG Last Calls, so we believe they are sufficiently
developed for the next step of the process.

Kevin and Al