Guidelines for Considering Operations and Management of New Protocols and Protocol Extensions
draft-ietf-opsawg-operations-and-management-09

[Ballot comment]
I agree with Cullen that the draft should make it more clear why it's
Informational instead of BCP (e.g., while there's lot of useful
information in it, we don't really have IETF wide consensus that these
guidelines are equally appropriate for all IETF work).

I also agree with Robert that much of the document is about
managing routers/networks, and probably less applicable to applications.

[Ballot discuss]
I like this document and think there is great information in it.

Section 1.1 and section 4 need a bit of word smithing after the document was moved from BCP to informational. I don't think they can say that a "a WG should" do X or the "Manageability Considerations section should". I think the text can be change dot be more like "It is useful if documents describe ...." Does this sounds like a reasonable change?

[Ballot comment]
The last paragraph of 2.2 could be generalized - speed is not the only thing that time changes.

The guidance in the last paragraph before section 3.1 starts should also point to the cases where focusing on instumenting servers might mask problems - instrumentation in clients may be the only thing that can provide information to the operator.

In section 3.1 "Information models should come from the protocol WGs" -- not all protocols come from WGs.

"some planned companion documents" is a phrase that will not age well once this is in RFC-stone. Can it be replaced with something a little less time-sensitive?

Consider changing the "must not" on the second line of page 9 to "MUST NOT" (or not).

[Ballot discuss]
This document contains a wealth of good advice and I am glad we are producing it. However, I have a few questions to work through before recommending publication:

The document is still strongly influenced by its genesis in routing. It is easy to see how to apply its advice when it is natural to talk about an application or a deployment in terms of the protocol, particularly when a configuration action on a network element maps directly into a configuration action in the protocol.

It is less obvious how to apply much of the advice when the protocol is a component or framework - when there are layers between the outward interface of the managed elements and the knobs in the protocols (TLS, SASL, S/MIME, PIDF, PIDF-LO, RTP, ...). Would it be possible to add guidance on how to apply the recommendations in those kinds of documents? Would it make sense to add/make more explicit a discussion on reveiwing managability of the interfaces that components expose to the to-be-managed applications that are going to use them?

Similarly, some of the "link", "node", and "packet" vocabulary becomes awkward when you start applying this to protocols that are higher in the stack. Can the language be abstracted to make it clearer what consideration discussion you're trying to stimulate in those cases?

I can be talked into waiting for future documents to address those concerns - the guidance here for the protocols that the recommendations can easily be applied to is worth publishing as soon as we can. But we need to address the above questions before we start to consider _requiring_ these considerations in all submissions.

[Ballot comment]
The Gen-ART Review by Miguel Garcia on 21-Sep-2009 suggests that
  acronyms be expanded on first occurrence and add a references to them
  (if there are documents). This includes: SNMP, SYSLOG, COPS, XML,
  RADIUS, DIAMETER, NETCONF, IPFIX, DMTF, TMF, RMON, and NMS.

[Ballot comment]
I found the document to be well-written and informative.

After reading the Abstract, I expected the document itself to describe, explain and give recommendations about considerations for protocol designers and document writers.  There are, indeed, many such considerations.  I have two concerns:

1. Some of the considerations include suggestions or are phrased in the form of a question without giving any recommendations about how to evaluate alternatives.  For example, section 3.3 includes a paragraph about propagation of fault information and asks a question about async notification or polling.  I think it would be helpful to add a few words about whether async notifications/polling is an either/or decision or other issues that the question is intended to point out.  Section 3.3.3 describes root cause analysis without giving any advice about how the consideration of root cause analysis might affect protocol design or operation.

2. Some of the considerations gove advice about operations that are not necessarily directly related to protocol design or documentation.  For example, section 3.3.4 gives advice about fault isolation; is that advice intended to suggest that the protocol designer should include features in the protocol to make fault isolation easy?  Or is it a suggestion to the network administrator about how to debug a network problem?

[Ballot comment]
Section 7
s/Farrell/Farrel/  (if you don't mind :-)

The Appendix is not directly referenced in the body of the text, but is actually a rather useful tool. You might include a pointer in Section 1.2 where you say...
> This document discusses the importance of considering operations and
> management by setting forth a list of guidelines and a checklist of
> questions to consider

(1.a) Who is the Document Shepherd for this document?

Scott Bradner

Has the
Document Shepherd personally reviewed this version of the
document and, in particular, does he or she believe this
version is ready for forwarding to the IESG for publication?
yes

(1.b) Has the document had adequate review both from key WG members
and from key non-WG members?

yes

Does the Document Shepherd have
any concerns about the depth or breadth of the reviews that
have been performed?

no

(1.c) Does the Document Shepherd have concerns that the document
needs more review from a particular or broader perspective,
e.g., security, operational complexity, someone familiar with
AAA, internationalization or XML?

no

(1.d) Does the Document Shepherd have any specific concerns or
issues with this document that the Responsible Area Director
and/or the IESG should be aware of?

no

For example, perhaps he
or she is uncomfortable with certain parts of the document, or
has concerns whether there really is a need for it. In any
event, if the WG has discussed those issues and has indicated
that it still wishes to advance the document, detail those
concerns here.

n/a

Has an IPR disclosure related to this document
been filed?

no

If so, please include a reference to the
disclosure and summarize the WG discussion and conclusion on
this issue.

n/a

(1.e) How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with
others being silent, or does the WG as a whole understand and
agree with it?

generally good support in the WG for publishing this document (both in the
face to face meetings and mailing list) - at least one dissenting voice but
consensus is to publish

(1.f) Has anyone threatened an appeal or otherwise indicated extreme
discontent?

no

If so, please summarise the areas of conflict in
separate email messages to the Responsible Area Director. (It
should be in a separate email because this questionnaire is
entered into the ID Tracker.)

n/a

(1.g) Has the Document Shepherd personally verified that the
document satisfies all ID nits? (See
http://www.ietf.org/ID-Checklist.html and
http://tools.ietf.org/tools/idnits/).

yes

Boilerplate checks are
not enough; this check needs to be thorough. Has the document
met all formal review criteria it needs to, such as the MIB
Doctor, media type and URI type reviews?

n/a

(1.h) Has the document split its references into normative and
informative?

yes (all references are informative)

Are there normative references to documents that
are not ready for advancement or are otherwise in an unclear
state?

no
If such normative references exist, what is the
strategy for their completion? Are there normative references
that are downward references, as described in [RFC3967]? If
so, list these downward references to support the Area
Director in the Last Call procedure for them [RFC3967].

n/a


(1.i) Has the Document Shepherd verified that the document IANA
consideration section exists and is consistent with the body
of the document?

yes

If the document specifies protocol
extensions, are reservations requested in appropriate IANA
registries? Are the IANA registries clearly identified? If
the document creates a new registry, does it define the
proposed initial contents of the registry and an allocation
procedure for future registrations? Does it suggest a
reasonable name for the new registry? See [RFC5226]. If the
document describes an Expert Review process has Shepherd
conferred with the Responsible Area Director so that the IESG
can appoint the needed Expert during the IESG Evaluation?

n/a

(1.j) Has the Document Shepherd verified that sections of the
document that are written in a formal language, such as XML
code, BNF rules, MIB definitions, etc., validate correctly in
an automated checker?

n/a

(1.k) The IESG approval announcement includes a Document
Announcement Write-Up. Please provide such a Document
Announcement Write-Up? Recent examples can be found in the
"Action" announcements for approved documents. The approval
announcement contains the following sections:

Technical Summary
The purpose of this document is to provide guidance to authors and
reviewers of documents defining new protocols or protocol extensions about
the aspects of operations and management that should be considered.

Working Group Summary
There was consensus in the working group to publish this document.

Document Quality
The document was well reviewed by the opsawg and by Scott Bradner.