Notification Message Support for BGP Graceful Restart
draft-ietf-idr-bgp-gr-notification-16

[Ballot comment]
Thank you for this document, it specifies a practical mechanism for increasing overall BGP state stability.

A nit - interworking with RFC7606 error handling may be mentioned in the case of error resulting in disabling the offending AFI.

[Ballot comment]
Please consider using the boilerplate from RFC 8174 instead of 2119. There are multiple lower case instances of normative keywords; the 8174 boilerplate is designed to handle that.

[Ballot comment]
Hello,

primarily, I strongly support Benjamin's COMMENT.
Section 3.1 is confusing and one may not understand the exact format of the NOTIFICATION to send in the context of this draft.

minor comment:
Isn't the use of "protocol" in "BGP protocol" redundant?

[Ballot discuss]
I'm sure that this has already been discussed somewhere, and that I'll be able to quickly clear my DISCUSS once pointed at it, but:
"To put an upper bound on the amount of time a router retains the
stale routes, an implementation MUST support a (configurable)
timer, called the "stale timer", that imposes this upper bound.
A suggested default value for the stale timer is 180 seconds.
An implementation MAY provide the option to disable the timer
(i.e., to provide an infinite retention time) but MUST NOT do so
by default."

The "infinite retention time" part of this makes me deeply uncomfortable -- I can see a good reason for the stale timer, and the default "feels" fine to me, but having an infinite retention time (or, really anything over 10 to 15 minutes) feels like a really dangerous idea, and that it will come back and bite.

I'm hoping that I'm missing something obvious, but can you please explain under what conditions an infinite retention policy makes sense? It seems like there would be multiple opportunities for blackholing traffic with this.

[Ballot comment]
Thanks for your work on this document. I have only one very small question to ask.

§4:

>  When a BGP speaker resets its session due to a HOLDTIME expiry, it
>  should generate the relevant BGP NOTIFICATION message as mentioned in

Is this intended to be "should" or "SHOULD"?

[Ballot comment]
Is there a pointer to a formal definition for "full session reset" that you could provide at the end of this text?

  At a high level, this document can be summed up as follows.  When a
  BGP session is reset, both speakers operate as "Receiving Speakers"
  according to [RFC4724], meaning they retain each other's routes.
  This is also true for HOLDTIME expiration.  The functionality can be
  defeated using a "Hard Reset" subcode for the BGP NOTIFICATION Cease
  Error code.  If a Hard Reset is used, a full session reset is
  performed.

[Ballot comment]
The table in Section 5.1 suggests that nesting a Hard Reset in a
Hard Reset is a supported operation.  When would it make sense to do
so/should this be forbidden?

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Services Operator has completed its review of draft-ietf-idr-bgp-gr-notification-15. If any part of this review is inaccurate, please let us know.

The IANA Services Operator understands that, upon approval of this document, there are three actions which we must complete.

First, in the BGP Cease NOTIFICATION message subcodes subregistry of the BGP Error Subcodes registry on the Border Gateway Protocol (BGP) Parameters registry page located at:

https://www.iana.org/assignments/bgp-parameters/

the temporary registration for:

Value: 9
Name: Hard Reset

will be made permanent and have its reference changed to [ RFC-to-be ].

Second, a new registry is to be created called the BGP Graceful Restart Flags registry. The new registry will be located on the Border Gateway Protocol (BGP) Parameters registry page located at:

https://www.iana.org/assignments/bgp-parameters/

The new registry will be managed via Standards Action as defined in RFC 8126. There are initial registrations in the new registry as follows:

+--------------+---------------+------------+---------------+
| Bit Position | Name | Short Name | Reference |
+--------------+---------------+------------+---------------+
| 0 | Restart State | R | [RFC4724] |
| 1 | Notification | N | [ RFC-to-be ] |
| 2, 3 | unassigned | | |
+--------------+---------------+------------+---------------+

Third, a new registry is to be created called the BGP Graceful Restart Flags for Address Family registry. The new registry will be located on the Border Gateway Protocol (BGP) Parameters registry page located at:

https://www.iana.org/assignments/bgp-parameters/

The new registry will be managed via Standards Action as defined in RFC 8126. There are initial registrations in the new registry as follows:

+--------------+------------------+------------+-----------+
| Bit Position | Name | Short Name | Reference |
+--------------+------------------+------------+-----------+
| 0 | Forwarding State | F | [RFC4724] |
| 1-7 | unassigned | | |
+--------------+------------------+------------+-----------+

The IANA Services Operator understands that these are the only actions required to be completed upon approval of this document.

Note:  The actions requested in this document will not be completed until the document has been approved for publication as an RFC. This message is meant only to confirm the list of actions that will be performed.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist

The following Last Call announcement was sent out (ends 2018-04-24):

From: The IESG
To: IETF-Announce
CC: idr@ietf.org, jie.dong@huawei.com, idr-chairs@ietf.org, aretana.ietf@gmail.com, draft-ietf-idr-bgp-gr-notification@ietf.org, Jie Dong
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (Notification Message support for BGP Graceful Restart) to Proposed Standard


The IESG has received a request from the Inter-Domain Routing WG (idr) to
consider the following document: - 'Notification Message support for BGP
Graceful Restart'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2018-04-24. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the beginning of
the Subject line to allow automated sorting.

Abstract


  The BGP Graceful Restart mechanism defined in RFC 4724 limits the
  usage of BGP Graceful Restart to BGP protocol messages other than a
  BGP NOTIFICATION message.  This document updates RFC 4724 by defining
  an extension that permits the Graceful Restart procedures to be
  performed when the BGP speaker receives a BGP NOTIFICATION Message or
  the Hold Time expires.  This document also defines a new BGP
  NOTIFICATION Cease Error subcode whose effect is to request a full
  session restart instead of a Graceful Restart.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-idr-bgp-gr-notification/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-ietf-idr-bgp-gr-notification/ballot/


No IPR declarations have been submitted directly on this I-D.

=== AD Review of draft-ietf-idr-bgp-gr-notification-13 ===
https://mailarchive.ietf.org/arch/msg/idr/g-ie-ETLWDCFXLl8h6AYv2EdcsY/?qid=9e78a1c5000204d9eb7b9dbd2ade6124

Dear authors:

I just finished reading this document — I have some comments, please see the list below.

I understand the intent of this document: instead of resetting a BGP session when a NOTIFICATION is received, use Graceful Restart; if the session is going to *really* be reset, then use the new Hard Reset sub-code.  That makes sense to me…but, is that the only code/sub-code for which it makes sense to do a hard reset?  The NOTIFICATION has always had the “stigma” of being something bad, so much that we (idr/IETF) have even worked on ways to reduce its use (rfc7606, for example).  I want to ask the WG to consider whether other code/sub-code NOTIFICATION combinations should also result in a hard reset.  I think there are several cases, for example:

(1) rfc4486 (Subcodes for BGP Cease Notification Message) defines “Administrative Shutdown” ("a BGP speaker decides to administratively shut down its peering with a neighbor”).  It seems to me that the sender of this NOTIFICATION would not want to "follow the rules for the Receiving Speaker” (as specified in Section 4).

(2) rfc4486 also defines "Administrative Reset” ("a BGP speaker decides to administratively reset the peering with a neighbor”); no more details are provided, but that sounds like a hard reset to me.

(3) …there’s probably others...

Having said all that, I note that Section 3.1. (Sending a Hard Reset) specifies the “encapsulation” (for lack of a better word) of the real reason for the Hard Reset.  If the consensus is to go forward with that, and not call out other exceptions, then I think that the text in 3.1 should expand more on the encapsulation operation and the rationale for doing it this way, and the document should also address other recent work that recommends the use of Administrative Shutdown, for example draft-ietf-grow-bgp-session-culling (a BCP currently in the RFC Editor’s Queue).

[After I wrote the text above…] I found that some of the points have been discussed on the list already — please include some of that discussion/analysis in the document.


I’ll wait until the issue above and ones marked Major (below) are addressed before starting the IETF LC.

Thanks!

Alvaro.



Major:

M1. Unfortunately, rfc4724 failed to setup a registry for the Restat Flags (or the Flags for Address Family), which means that anyone is able to use the bits in there (assuming the receiver looks at them, of course).  Given that there are only a few bits, and to prevent conflicts, I would really like to see a registry set up.  This document is already tagged to Update rfc4724, so it seems like a good place to establish the registries.  [If for some reason you rather not include that information here, then we can take care of it elsewhere.  IOW, this request is not a requirement.]


M2. Section 4.1. (Rules for the Receiving Speaker) has me a little confused.  Are the proposed changes contingent to setting the N bit?  The text starts by saying: "As part of this extension, routes from the peer previously marked as stale MUST NOT be deleted, until and unless the optional timer…expires…”…does that mean that the timer is no longer optional?  Then you also say: “...if the Graceful Notification ("N") bit is not set in the newly received Graceful Restart Capability, no new actions are triggered on the Receiving Speaker -- in particular, a clear "N” bit does not trigger deletion of stale routes.”  If I understand rfc4724 correctly, stale routes could be deleted — the text indicates changes in the behavior even if the N bit is not set, right?  If you are Updating this section of rfc4724, what would make it crystal clear is an “OLD/NEW” notation of the text (as in, this is the OLD text…and this is the NEW text…).


M3. Security Considerations:  Maybe not a security issue, but something to think about.  Section 4.1 says that “routes...previously marked as stale MUST NOT be deleted, until and unless the optional timer...expires, or unless a Hard Reset is performed.  This supersedes the “consecutive restarts” requirement…”.  Not deleting the stale routes and not making the timer mandatory could result in stale routes that live forever if an attacker manages to create consecutive restarts (by simply sending NOTIFICATIONS before EoR) — stale routes are ok in the short term, but may point in the wrong direction eventually.  Is this an issue?  I think that it would be mitigated if the timer was made mandatory (with a nice default).



Minor:

P1. Section 4: “...receive and send BGP NOTIFICATION messages in Graceful mode...” What is “Graceful mode”?



Nits:

N1. In Section 2, please indicate that the first figure corresponds to the GR Capability from rfc4724.

N2. s/subcode is defined known as/subcode is defined as

N3. s/Graceful Notification flag/Graceful Notification bit  (For consistency)

N4. The operation is obviously per-AF; maybe it’s worth saying that somewhere just for completeness.

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up.

Changes are expected over time. This version is dated 24 February 2012.

(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?  Why
is this the proper type of RFC?  Is this type of RFC indicated in the
title page header?
Proposed Standard.
It defines protocol extensions to BGP.
The type of RFC is indicated in the title page header.

(2) The IESG approval announcement includes a Document Announcement
Write-Up. Please provide such a Document Announcement Write-Up. Recent
examples can be found in the "Action" announcements for approved
documents. The approval announcement contains the following sections:

Technical Summary

  The current BGP Graceful Restart mechanism limits the usage of BGP
  Graceful Restart to BGP protocol messages other than a BGP
  NOTIFICATION message.  This document defines an extension to the BGP
  Graceful Restart that permits the Graceful Restart procedures to be
  performed when the BGP speaker receives a BGP NOTIFICATION Message or
  the Hold Time expires.  This document also defines a new BGP
  NOTIFICATION Cease Error subcode whose effect is to request a full
  session restart instead of a Graceful Restart.

Working Group Summary

  IDR has reviewed and discussed this document over 3 years (2014-2017). 
  WG has reached consensus after several revisions.
  There was some discussion about the early allocation of a new subcode
  from the "BGP Cease NOTIFICATION message subcodes" registry for this
  document. The IANA early allocation procedure has been followed, and
  a temporary code has been allocated by IANA.

Document Quality

  There are two implementations of this document reported.

  RtgDir QA review on -07 version:
  https://www.ietf.org/mail-archive/web/idr/current/msg15812.html
 
  RtgDir review on -07 version:
  https://www.ietf.org/mail-archive/web/idr/current/msg16157.html

Personnel

  Document Shepherd: Jie Dong
  Responsible Area Director: Alvaro Retana

(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready
for publication, please explain why the document is being forwarded to
the IESG.
The document shepherd performed the review on:
1) Nits
2) Technical review 
3) Implementation report
4) IPR check
The document shepherd think the current version (-12) is ready for publication.

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?
No.

(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization? If so, describe the review that
took place.
Nothing beyond the normal checks.

(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the
IESG should be aware of? For example, perhaps he or she is uncomfortable
with certain parts of the document, or has concerns whether there really
is a need for it. In any event, if the WG has discussed those issues and
has indicated that it still wishes to advance the document, detail those
concerns here.
None.

(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed. If not, explain why.

Keyur Patel: no IPR
https://mailarchive.ietf.org/arch/msg/idr/nwR_8D5Oj1esd53WqVffxR5y1M8

Rax Fernando: no IPR
https://mailarchive.ietf.org/arch/msg/idr/ETc73_nC3JBKPjZMCLqd7_bFAI8

John Scudder: no IPR
https://mailarchive.ietf.org/arch/msg/idr/7u2idOpxSFKLHdld99JuaeFnxPs

Jeff Haas:  no IPR
https://mailarchive.ietf.org/arch/msg/idr/wjJFUp3PIzqxHzQ2v0Tu8u828WY

(8) Has an IPR disclosure been filed that references this document?
If so, summarize any WG discussion and conclusion regarding the IPR
disclosures.
No IPR Disclosure.

(9) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with others
being silent, or does the WG as a whole understand and agree with it? 
Solid consensus according to the review and discussion on the list.

(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director. (It should be in a
separate email because this questionnaire is publicly available.)
None.

(11) Identify any ID nits the Document Shepherd has found in this
document. (See https://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist). Boilerplate checks are not enough; this check needs to be
thorough.
No nits.

(12) Describe how the document meets any required formal review
criteria, such as the MIB Doctor, media type, and URI type reviews.
Not relevant for MIB Doctor, Media type, or URI.

(13) Have all references within this document been identified as
either normative or informative?
Yes.

(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state? If such normative
references exist, what is the plan for their completion?
No. All normative references are RFCs.

(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in
the Last Call procedure.
None.

(16) Will publication of this document change the status of any
existing RFCs? Are those RFCs listed on the title page header, listed
in the abstract, and discussed in the introduction? If the RFCs are not
listed in the Abstract and Introduction, explain why, and point to the
part of the document where the relationship of this document to the
other RFCs is discussed. If this information is not in the document,
explain why the WG considers it unnecessary.
RFC 4724 is updated by this document.
Yes RFC 4724 is listed on the title page header, abstract, and discussed in
the introduction.

(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document. Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly
identified. Confirm that newly created IANA registries include a
detailed specification of the initial contents for the registry, that
allocations procedures for future registrations are defined, and a
reasonable name for the new registry has been suggested (see RFC 5226).

IANA has performed the early allocation of subcode 9, named "Hard Reset",
in the "BGP Cease NOTIFICATION message subcodes" registry”.
Upon publication of this document as an RFC, IANA is requested to make
this allocation permanent.

(18) List any new IANA registries that require Expert Review for future
allocations. Provide any public guidance that the IESG would find
useful in selecting the IANA Experts for these new registries.
No new registries.

(19) Describe reviews and automated checks performed by the Document
Shepherd to validate sections of the document written in a formal
language, such as XML code, BNF rules, MIB definitions, etc.
No review of automated checks required.