Reducing the Standards Track to Two Maturity Levels
draft-housley-two-maturity-levels-09

[Ballot comment]
Thank you for addressing my earlier DISCUSS and COMMENT issues.

It looks to me as though there is still a bit of redundancy in section
2.2.  Do these two paragraphs refer to the same IETF-wide Last Call?

  The criteria for advancing from Proposed Standard to Internet
  Standard are confirmed by the IESG in an IETF-wide Last Call of at
  least four weeks.  The request for reclassification is sent to the
  IESG along with an explanation of how the criteria have been met.
  The criteria are:

  [...]

  After review and consideration of significant errata, the IESG will
  perform an IETF-wide Last Call of at least four weeks on the
  requested reclassification.  If there is consensus for
  reclassification, the RFC will be reclassified without publication of
  a new RFC.

[Ballot comment]
I'm changing to Yes on this based on the recent mails to
ietf-discuss. I continue to think there is rough consensus for
2 levels but am motivated to move to a Yes by the arguments
of those against moving to 2 levels which appear to me
indistinguishable from obfuscation. (Which could be
deliberate or a side-effect of some honestly held opinion,
its not really possible to say.)

This is such a boring topic that a funny quote seems
worthwhile:

    “A Frenchman once tried to obfuscate me from behind.
    Although I protested out of sheer principle, I must
    confess that I thoroughly enjoyed it.”
    ~ Oscar Wilde on Obfuscation

I think the always-reliable Oscar's quote is as relevant as
the objections to 2 levels that I've seen so far.

My original comment was:

I still don't know from reading this if STD numbers will be
allocated for "Internet Standard" RFCs.  I don't mind if
they are or not, but confusion about it doesn't seem good.
For example, what'll happen to [1]?

  [1] http://www.rfc-editor.org/rfcxx00.html#STDbySTD

[Ballot comment]
I still don't know from reading this if STD numbers will be
allocated for "Internet Standard" RFCs.  I don't mind if
they are or not, but confusion about it doesn't seem good.
For example, what'll happen to [1]?

  [1] http://www.rfc-editor.org/rfcxx00.html#STDbySTD

State changed to In Last Call from Last Call Requested.

The following Last Call Announcement was sent out:

From: The IESG
To: IETF-Announce
Reply-To: ietf@ietf.org
Subject: Last Call:  (Reducing the Standards Track to Two Maturity Levels) to BCP


The IESG has received a request from an individual submitter to consider
the following document:
- 'Reducing the Standards Track to Two Maturity Levels'
  as a BCP

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2011-08-24. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  This document proposes several changes to the Internet Engineering
  Task Force (IETF) Standards Process defined in RFC 2026, primarily a
  reduction from three standards track maturity levels to two.

  {{ RFC Editor: please change "proposes several changes to the" to
  "updates the". }}




The file can be obtained via
http://datatracker.ietf.org/doc/draft-housley-two-maturity-levels/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-housley-two-maturity-levels/


No IPR declarations have been submitted directly on this I-D.

[Ballot comment]
nit - Introduction

"This document proposes several changes " should presumably be "This document makes several changes "

I am concerned about the following:

"Note that the distinct requirement from RFC 2026 [1] for reports of
interoperability testing among two or more independent
implementations is intentionally subsumed by the requirement for
actual deployment and use of independent and interoperable
implementations."

We really need to consider defining "independent". NTP is a good
illustration of the problem - there are lots of boxes out here that
run NTP which some may perceive as independent implementations.
However they all seem to run the code written by Dave Mills' project
arguably making them a single implementation. Since we touching
on the subject we should clarify what we mean by "independent"
give the existence of open source.

[Ballot discuss]
I am concerned about the following:

Note that the distinct requirement from RFC 2026 [1] for reports of
interoperability testing among two or more independent
implementations is intentionally subsumed by the requirement for
actual deployment and use of independent and interoperable
implementations.

[Ballot discuss]
I support the intention and the specifics in this document, and intend
to vote "yes" after we have discussed a couple of points.  In general,
based on personal experience and observation, I believe that the
changes to the standards track will improve those processes.  Perhaps
now we can advance RFCs 2131, 2132 and 3315 to Internet Standard...

However, I am not prepared to accept section 2.1 without further
discussion.  My concern with section 2.1 is that there is no explicit
explanation of the ways in which "publishing a Proposed Standard [is]
much harder than the stated requirements in RFC 2026" and how "to
restore practice to the requirements for Proposed Standard from RFC
2026"."  For example, is the intention of this document for the IESG
to self-regulate its review of specifications to return to the "stated
requirements in RFC 2026?"

In section 2.2, the specific mechanics for advancing to Internet
Standard need to be clarified:

OLD:

  The criteria for advancing from Proposed Standard to Internet
  Standard are confirmed by the IESG in an IETF-wide Last Call of at
  least two weeks:

NEW:

  A specification must satisfy the following criteria before
  advancing to Internet Standard:

      [...]

  The IESG confirms that the specification meets the criteria.  The
  IESG uses an IETF-wide Last Call of at least two seeks to gather
  comments from the IETF community about advancement of the
  specification.

[Ballot discuss]
I agree fully with Peter's DISCUSS. He summarized much better than I could have.

I do believe that there probably *is* agreement on the requirement changes to move from Proposed to whatever is the next level and that these changes do address a problem by lowering the burden of moving out of Proposed. I would like to see those adopted, independent of agreement on the other two portions (i.e., returning to a lower bar for Proposed, or removing one level of the standards track).

[Ballot discuss]
I would love to ballot "Yes" on this document.  However, having reviewed
the feedback provided during IETF Last Call, I am not comfortable with
saying that we have rough consensus to proceed with publication.  I hope
this DISCUSS will provide some helpful input to a conversation about the
path forward.

My reading of the Last Call feedback is that there is quite a bit of
confusion about what problem(s) we need to solve in this space.  Some of 
the possibilities are:

1. Make it easier to advance to Proposed Standard, e.g., by returning to 
the RFC 2026 philosophy of what "Proposed" means.

2. Simplify and clarify the process of progressing from Proposed
Standard to Draft Standard, e.g. by telling folks that implementation
reports don't need to be a huge undertaking.

3. Advance more specifications to Full Standard, e.g. by removing
obstacles or providing incentives for advancement.

4. Improve our "branding" and better satisfy our "customers", e.g. by
removing stages in the standards process that few people have used.

5. Align our documentation with the reality of the processes that we
actually follow.

6. Start measuring the right things, e.g. by getting rid of
interoperability reports and instead gauging widespread deployment.

7. Encourage working groups and document editors to incorporate
implementation and deployment feedback into their specs throughout the
process, e.g. by making it easier to iterate at Proposed Standard.

It seems that until we have clarity in the community about which
problem(s) we're trying to solve, we'll never reach consensus about
which solutions we need.  My reluctant conclusion is that another
round of problem-defining is required before we proceed to
problem-solving.

[Ballot discuss]
I would love to ballot "Yes" on this document.  However, having reviewed
the feedback provided during IETF Last Call, I am not comfortable with
saying that we have rough consensus to proceed with publication.  I hope
this DISCUSS will provide some helpful input to a conversation about the
path forward.

My reading of the Last Call feedback is that there is quite a bit of
confusion about what problem(s) we need to solve in this space.  Some of 
the possibilities are:

1. Make it easier to advance to Proposed Standard, e.g., by returning to 
the RFC 2026 philosophy of what "Proposed" means.

2. Simplify and clarify the process of progressing from Proposed
Standard to Draft Standard, e.g. by telling folks that implementation
reports don't need to be a huge undertaking.

3. Advance more specifications to Full Standard, e.g. by first diagnosing
why it's so hard right now and then coming up with solutions.

4. Improve our "branding" and better satisfy our "customers", e.g. by
removing stages in the standards process that few people have used.

5. Align our documentation with the reality of the processes that we
actually follow.

6. Start measuring the right things, e.g. by getting rid of
interoperability reports and instead gauging widespread deployment.

7. Encourage working groups and document editors to incorporate
implementation and deployment feedback into their specs throughout the
process, e.g. by making it easier to iterate at Proposed Standard.

It seems that until we have clarity in the community about which
problem(s) we're trying to solve, we'll never reach consensus about
which solutions we need.  My conclusion is that we need another round
of problem-defining before we proceed to problem-solving.

[Ballot comment]
(1) "limited to the changes" - might be good to make it clear
that all text changes are up for checking. Otherwise someone
will make an argument that "I only changed informative text,
the protocol's the same" and that the IESG therefore should
shut up and go away.

(2) I don't see that this document changes anything at all in
terms of the difficulty of getting a 1st PS. That's ok, but the
text is a bit misleading on this I think. In particular, I'd delete
"The intention of the two-tier maturity ladder is to restore
practice to the requirements for Proposed Standard from
RFC 2026, and stop performing more scrutiny than intended
in IETF working groups and the IESG." That may be the
authors' intention and it might be good or bad, but this
document does nothing about it.
 
(3) I don't know from reading this if STD numbers will be
allocated for "Internet Standard" RFCs.  (It says that
STD numbers are allocated to "full Standard maturity
level" documents, and that existing practice will remain,
but after this BCP there will be no more "full Standard"
documents, so its not clear.)

[Ballot comment]
Section 2 contains the following:

  Experience with a Proposed Standard often leads to revisions that
  clarify, modify, enhance, or remove features.  Review of revisions to
  a Proposed Standard that is submitted for publication at the same
  maturity level is generally limited to the changes.

Perhaps r/the changes/these types of changes ?

RFC 5967 already updates RFC 2026, but a reference to RFC 2026 in Section 2.2 would be helpful to those writing implementation reports.

State changed to In Last Call from Last Call Requested.

The following Last Call Announcement was sent out:

From: The IESG
To: IETF-Announce
Reply-To: ietf@ietf.org
Subject: Last Call:  (Reducing the Standards Track to Two Maturity Levels) to BCP


The IESG has received a request from an individual submitter to consider
the following document:
- 'Reducing the Standards Track to Two Maturity Levels'
  as a BCP

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2011-06-02. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

The file can be obtained via
http://datatracker.ietf.org/doc/draft-housley-two-maturity-levels/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-housley-two-maturity-levels/


No IPR declarations have been submitted directly on this I-D.

State changed to AD Evaluation from AD Evaluation::AD Followup.
I have now reviewed discussion back to late 2010. I think we can move ahead, and I have asked for a last call.

I would classify the situation as follows. There's been discussion and there's some chance that a rough consensus exists behind doing this proposal, or at the very least that its harmless and does not stand in the way of more important improvements. We'll see what the consensus will be. I certainly personally hope we can do this, not because I believe this will by itself make a big difference. But because I think its important that we simplify our process and remove cruft that has accumulated over the years.

State changed to AD Evaluation::AD Followup from AD Evaluation.
I have reviewed this version of the document, and I believe it is ready to move forward.

I still need to review the (recent) history of community discussions about this to make sure that there's general acceptance before I initiate the final IETF Last Call.

Stay tuned.

(1.a) Who is the Document Shepherd for this document? Has the
Document Shepherd personally reviewed this version of the document
and, in particular, does he or she believe this version is ready
for forwarding to the IESG for publication?

Russ Housley is the document shepherd. He has personally reviewed the
document, and he believes that it is ready for publication.

(1.b) Has the document had adequate review both from key members of
the interested community and others? Does the Document Shepherd
have any concerns about the depth or breadth of the reviews that
have been performed?

The document was discussed in the IETF plenary, and it was discussed
on the IETF mail list.

(1.c) Does the Document Shepherd have concerns that the document
needs more review from a particular or broader perspective, e.g.,
security, operational complexity, someone familiar with AAA,
internationalization or XML?

No concerns.

(1.d) Does the Document Shepherd have any specific concerns or
issues with this document that the Responsible Area Director
and/or the IESG should be aware of? For example, perhaps he or
she is uncomfortable with certain parts of the document, or has
concerns whether there really is a need for it. In any event, if
the interested community has discussed those issues and has
indicated that it still wishes to advance the document, detail
those concerns here.

No concerns.

(1.e) How solid is the consensus of the interested community behind
this document? Does it represent the strong concurrence of a few
individuals, with others being silent, or does the interested
community as a whole understand and agree with it?

There is not unanimous sopport for this document, but there seems to
be rough consensus for it.

(1.f) Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise the areas of conflict in
separate email messages to the Responsible Area Director. (It
should be in a separate email because this questionnaire is
entered into the ID Tracker.)

Some people do not believe that the change to two maturity levels
will have any impact, but these people have not threatened to appeal.

(1.g) Has the Document Shepherd personally verified that the
document satisfies all ID nits? (See the Internet-Drafts Checklist
and http://tools.ietf.org/tools/idnits/). Boilerplate checks are not
enough; this check needs to be thorough. Has the document met all
formal review criteria it needs to, such as the MIB Doctor, media
type and URI type reviews?

IDnits produces one warning about the Abstract. The document already
contains a note to the RFC Editor that should resolve the warning.

(1.h) Has the document split its references into normative and
informative? Are there normative references to documents that are
not ready for advancement or are otherwise in an unclear state?
If such normative references exist, what is the strategy for their
completion? Are there normative references that are downward
references, as described in [RFC3967]? If so, list these downward
references to support the Area Director in the Last Call procedure
for them [RFC3967].

All references are normative.

(1.i) Has the Document Shepherd verified that the document IANA
consideration section exists and is consistent with the body of
the document? If the document specifies protocol extensions, are
reservations requested in appropriate IANA registries? Are the
IANA registries clearly identified? If the document creates a new
registry, does it define the proposed initial contents of the
registry and an allocation procedure for future registrations?
Does it suggested a reasonable name for the new registry? See
[I-D.narten-iana-considerations-rfc2434bis]. If the document
describes an Expert Review process has Shepherd conferred with the
Responsible Area Director so that the IESG can appoint the needed
Expert during the IESG Evaluation?

No IANA actions are needed.

(1.j) Has the Document Shepherd verified that sections of the
document that are written in a formal language, such as XML code,
BNF rules, MIB definitions, etc., validate correctly in an
automated checker?

No formal language is used.

(1.k) The IESG approval announcement includes a Document
Announcement Write-Up. Please provide such a Document
Announcement Writeup? Recent examples can be found in the
"Action" announcements for approved documents. The approval
announcement contains the following sections:

Technical Summary

This document proposes several changes to the Internet Engineering
Task Force (IETF) Standards Process defined in RFC 2026, primarily
a reduction from three IETF standards track maturity levels to two.

Working Group Summary

This document is not the product of any IETF WG.

Document Quality

This document was discussed in IETF plenary, and it was discussed
on the IETF mail list. There is not unanimous sopport for it, but
there seems to be rough consensus for it.