Basic BGP Convergence Benchmarking Methodology for Data-Plane Convergence
draft-ietf-bmwg-bgp-basic-convergence-05

[Ballot comment]
In Sec 4.4, the basic settings for Maximum TCP Window Size and MTU are not given.  If there isn't a recommended value, saying so would be good.

[Ballot comment]
As notes by Scott Bradner in his OPS directorate review.
Some comments/questions on the contents of the draft:


1.1
  "FIB (Data plane) convergence is defined as the completion of all FIB
  changes so that all forwarded traffic now takes the new proposed
  route. "

should route be singular or plural - i.e. is the assumption that the
routing table converges to a single next hop? (at least for the test traffic)
if so, does the draft specifically say that (or does rfc 4098 and I missed it)
note: figure 1 shows multiple peering links - sec 4.1 seems to argue for
multiple peers

  "Data plane convergence is different than control plane
  convergence within a node."

might want to say how they are different


since reporting requiremenst are covered in section 6 should
they also be mentioned here? (if so, how about in section 4.2)

secton 4.4  & 4.8
maybe replace TCP MD5 with TCP Authentication Option (2 places)
or at least mention it

section 4.4 basic test settings - maybe say why these values were chosen


section 4.7  agree as to the importance fo rrepeating trials - is
there a recognized source that discusses "generally accepted testing
practices regarding repeatability ..."?

section 5
what about Graceful Restart (RFC 4724) - would that impact the
clean start desire?

section 5.1.1
      "D.  Start the traffic from the Emulator tx towards the DUT
          targeted at a routes specified in route mixture (ex. routeA)"

change "a routes" to "a route" or "the routes"

E & F - as noted earlier in the document - these times should be very
close to the same - is it actually worth the additional complexity
to collect the time when the update is received?
also 5.1.2 H & I,  etc

section 5.1.2 mentions NTP but section 5.1.1 does not - is there a reason?


section 5.2.1 - since the shutdown event is not timed - does this test
provide a useful measurement? (or should the time be recorded and
its just not mentioned?)

section 5.3 - F - implies that the time is recorded but not actually say
say that it is

general comment - review all steps of all tests to be sure that
NTP is called for when it is needed  and that event times are
specifically called for when they are needed and use consistent
langage in each case

the overall requiremenst - e.g. NTP could also just be noted
before the test descriptions and not inlcuded in each one if
it is needed in all of them - same with advice about
nukbers of routes (do tests with different numbers or routes
up to the full Internet table)

section 6 - should this also include the number of AS Paths?

[Ballot comment]
A couple of minor, non-blocking comments that I hope you'll consider:

-- Section 2 --
As RFC 6163 is used to define necessary terminology, I think it's a normative reference.

-- Section 3 --
I found the first paragraph here to be confusing: one thing can't be grouped into multiple categories, and "regardless" seems not the right word.  Also, the sentence (ending in ":") that introduces the list doesn't have anything to do with the list it introduces.

May I propose this instead, and let you fix it if I don't have it quite right?:

OLD
  The WSON RWA information model in this document is grouped into four
  categories regardless of whether they stem from a switching
  subsystem or from a line subsystem. A switching subsystem refers to
  WSON nodes such as ROADM or Optical Add/Drop Multiplexer (OADM) and
  a line subsystem refers to devices such as WDM or Optical Amplifier:
NEW
  The WSON RWA information model in this document comprises four
  categories of information. The categories are independent of
  whether the information comes from a switching subsystem or from a
  line subsystem -- a switching subsystem refers to WSON nodes such as
  ROADM or Optical Add/Drop Multiplexer (OADM), and a line subsystem
  refers to devices such as WDM or Optical Amplifier. The categories
  are these:
END

[Ballot comment]
Thanks for addressing Terry's Routing Directorate review.

---

I liked the understatement of
  BGP is ... used by several service providers as the
  default Inter AS routing protocol.

Several == "more than two but not many"

Perhaps you could s/several/many/

(Original text included below, preserved for posterity)
----------------------------------------------------------------------

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up.

Changes are expected over time. This version is dated 24 February 2012.

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?

This RFC is intended to be Informational, as is consistent what BMWG outputs.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary
=-=-=-=-=-=-=-=-=-=
This document describes a methodology to benchmark BGP convergence, utilizing
terminology from RFC 4098. In particular, this draft describes a methodology for
benchmarking the data plane FIB convergence performance of BGP, applying to both
IPv4 and IPv6 topologies with 3 or 4 nodes.


Working Group Summary
=-=-=-=-=-=-=-=-=-=-=-=-=
The -00 version of the draft was submitted in July 2013, and with decent feedback and
discussion from the WG, the authors were able to submit a -01 document in March 2014, that moved into WGLC. At that time, a review from the Routing Directorate provided valuable feedback, which resulted in the document returning to the WG, and the authors revising the document based on this feedback, and WG feedback. -02 of the draft was introduced in June 2014, and moved to WGLC in September 2014. There've been no major issues or concerns raised, no heated debates or serious negative stances taken against this draft.


Document Quality
=-=-=-=-=-=-=-=-=
This document is in decent shape, being readable, reviewed both internally and external to the working group, and has had attentive authors answering and addressing incoming feedback in a timely manner, leading to a stronger, easily consumed document.

Personnel
=-=-=-=-=-=

Sarah Banks is the document shepherd.
Joel Jaeggli is the responsible AD.


(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.

As the document shepherd, I've read this draft twice over; previous to moving this document to WGLC and becoming the shepherd, I'd reviewed and provided comments on this draft through it's course in the Working Group. This document is ready for publication, being well vetted and well reviewed, and comments addressed within.



(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed? 

No concerns with the depth or breadth of the reviewed that have been performed.


(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.

The chairs/authors felt the draft could do with a review from outside of BMWG, and as such, the Routing Area performed a review, and submitted minor comments/concerns, which the authors addressed, prior to moving the draft into WGLC. This review is in addition to reviews within the WG itself.


(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.

This is a fairly "safe" draft - it's well written, it's well understood, and with comments raised during reviews addressed, the document is in a pretty decent state and ready for publication.


(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

No IPR claimed.


(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.

No IPR claimed.


(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 

Solid consensus exists behind this document. There were/are no dissents that the DS is aware of.


(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)

No.


(11) Identify any ID nits the Document Shepherd has found in this
document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.

No major nits, but some warnings:

  Miscellaneous warnings:
  ----------------------------------------------------------------------------

  == The document seems to contain a disclaimer for pre-RFC5378 work, but was
    first submitted on or after 10 November 2008.  The disclaimer is usually
    necessary only for documents that revise or obsolete older RFCs, and that
    take significant amounts of text from those RFCs.  If you can contact all
    authors of the source material and they are willing to grant the BCP78
    rights to the IETF Trust, you can and should remove the disclaimer.
    Otherwise, the disclaimer is needed and you can ignore this comment.
    (See the Legal Provisions document at
    http://trustee.ietf.org/license-info for more information.)

  -- The document date (October 26, 2014) is 17 days in the past.  Is this
    intentional?


  Checking references for intended status: Informational
  ----------------------------------------------------------------------------

  == Missing Reference: 'MPLSProt' is mentioned on line 441, but not defined

  == Missing Reference: 'IGPData' is mentioned on line 949, but not defined

  == Missing Reference: 'BGPSec' is mentioned on line 1286, but not defined

  == Unused Reference: 'I-D.ietf-sidr-bgpsec-protocol' is defined on line
    1416, but no explicit reference was found in the text

  == Unused Reference: 'RFC4271' is defined on line 1431, but no explicit
    reference was found in the text

  == Unused Reference: 'RFC6412' is defined on line 1434, but no explicit
    reference was found in the text

  == Outdated reference: A later version (-10) exists of
    draft-ietf-sidr-bgpsec-protocol-09



(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.

None needed.


(13) Have all references within this document been identified as
either normative or informative?

See nits above.


(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?

No.


(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in
the Last Call procedure.

No.


(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.

No.


(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).

No IANA considerations exist.


(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.

None/NA


(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.

No code chunks exist - nothing to check, short of checking nits through IETF Tools.


-------------------------------------------------------------------------------------------------------

Original text here:
Technical Summary

This document describes a methodology to benchmark BGP convergence, utilizing
terminology from RFC 4098. In particular, this draft describes a methodology for
benchmarking the data plane FIB convergence performance of BGP, applying to both
IPv4 and IPv6 topologies with 3 or 4 nodes.


Working Group Summary

The -00 version of the draft was submitted in July 2013, and with decent feedback and
discussion from the WG, the authors were able to submit a -01 document in March 2014,
that moved into WGLC. At that time, a review from the Routing Directorate provided
valuable feedback, which resulted in the document returning to the WG, and the authors
revising the document based on this feedback, and WG feedback. -02 of the draft was
introduced in June 2014, and moved to WGLC in September 2014. There've been no
major issues or concerns raised, no heated debates or serious negative stances taken
against this draft.


Document Quality

This document is in decent shape, being readable, reviewed both internally and external to
the working group, and has had attentive authors answering and addressing incoming
feedback in a timely manner, leading to a stronger, easily consumed document.


Personnel

Sarah Banks is the Document Shepherd. (sbanks@encrypted.net)
Joel Jaeggli is the Responsible AD. (joelja@bogus.com)

IESG/Authors/WG Chairs:

IANA has reviewed draft-ietf-bmwg-bgp-basic-convergence-04, which is currently in Last Call, and has the following comments:

We understand that,  upon approval of this document, there are no IANA Actions that need completion.

While it is helpful for the IANA Considerations section of the document to remain in place upon publication, if the authors prefer to remove it, IANA doesn't object.

If this assessment is not accurate, please respond as soon as possible.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Basic BGP Convergence Benchmarking Methodology for Data Plane Convergence) to Informational RFC


The IESG has received a request from the Benchmarking Methodology WG
(bmwg) to consider the following document:
- 'Basic BGP Convergence Benchmarking Methodology for Data Plane
  Convergence'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2014-11-10. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  BGP is widely deployed and used by several service providers as the
  default Inter AS routing protocol.  It is of utmost importance to
  ensure that when a BGP peer or a downstream link of a BGP peer fails,
  the alternate paths are rapidly used and routes via these alternate
  paths are installed.  This document provides the basic BGP
  Benchmarking Methodology using existing BGP Convergence Terminology,
  RFC 4098.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-bmwg-bgp-basic-convergence/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-bmwg-bgp-basic-convergence/ballot/


No IPR declarations have been submitted directly on this I-D.

Technical Summary

This document describes a methodology to benchmark BGP convergence, utilizing
terminology from RFC 4098. In particular, this draft describes a methodology for
benchmarking the data plane FIB convergence performance of BGP, applying to both
IPv4 and IPv6 topologies with 3 or 4 nodes.


Working Group Summary

The -00 version of the draft was submitted in July 2013, and with decent feedback and
discussion from the WG, the authors were able to submit a -01 document in March 2014,
that moved into WGLC. At that time, a review from the Routing Directorate provided
valuable feedback, which resulted in the document returning to the WG, and the authors
revising the document based on this feedback, and WG feedback. -02 of the draft was
introduced in June 2014, and moved to WGLC in September 2014. There've been no
major issues or concerns raised, no heated debates or serious negative stances taken
against this draft.


Document Quality

This document is in decent shape, being readable, reviewed both internally and external to
the working group, and has had attentive authors answering and addressing incoming
feedback in a timely manner, leading to a stronger, easily consumed document.


Personnel

Sarah Banks is the Document Shepherd. (sbanks@encrypted.net)
Joel Jaeggli is the Responsible AD. (joelja@bogus.com)