Characterization of Proposed Standards
draft-kolkman-proposed-standards-clarified-06
Revision differences
Document history
Date | Rev. | By | Action |
---|---|---|---|
2014-01-29
|
06 | (System) | RFC Editor state changed to AUTH48-DONE from AUTH48 |
2014-01-27
|
06 | (System) | RFC Editor state changed to AUTH48 from RFC-EDITOR |
2014-01-27
|
06 | (System) | RFC Editor state changed to RFC-EDITOR from EDIT |
2013-12-18
|
06 | Amy Vezza | State changed to RFC Ed Queue from Approved-announcement sent |
2013-12-17
|
06 | (System) | RFC Editor state changed to EDIT |
2013-12-17
|
06 | (System) | Announcement was received by RFC Editor |
2013-12-17
|
06 | (System) | IANA Action state changed to No IC from In Progress |
2013-12-17
|
06 | (System) | IANA Action state changed to In Progress |
2013-12-17
|
06 | Amy Vezza | State changed to Approved-announcement sent from Approved-announcement to be sent::AD Followup |
2013-12-17
|
06 | Amy Vezza | IESG has approved the document |
2013-12-17
|
06 | Amy Vezza | Closed "Approve" ballot |
2013-12-17
|
06 | Amy Vezza | Ballot approval text was generated |
2013-12-17
|
06 | Amy Vezza | Ballot writeup was changed |
2013-12-16
|
06 | Jari Arkko | State changed to Approved-announcement to be sent::AD Followup from IESG Evaluation::AD Followup |
2013-11-04
|
06 | Olaf Kolkman | IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed |
2013-11-04
|
06 | Olaf Kolkman | New version available: draft-kolkman-proposed-standards-clarified-06.txt |
2013-11-03
|
05 | Barry Leiba | [Ballot comment] I think the discussion is converging, and I'm removing my DISCUSS to let Jari, as sponsoring AD, decide when things are settled and … [Ballot comment] I think the discussion is converging, and I'm removing my DISCUSS to let Jari, as sponsoring AD, decide when things are settled and it's time to post the approval. The main issue that we're still settling is about whether to: 1 leave Section 3.2 as it is, copying the "characterization" part from 2026 Section 4.1.3, or 2. have Section 3.2 refer to 2026 Section 4.1.3, without copying the text, or 3. have Section 3 fully replace 2026 Section 4.1, so all the description of the maturity levels is in one place. Jari... it's yours. |
2013-11-03
|
05 | Barry Leiba | [Ballot Position Update] Position for Barry Leiba has been changed to Yes from Discuss |
2013-10-31
|
05 | Tero Kivinen | Closed request for Last Call review by SECDIR with state 'No Response' |
2013-10-31
|
05 | (System) | Sub state has been changed to AD Followup from Revised ID Needed |
2013-10-31
|
05 | Cindy Morgan | New revision available |
2013-10-24
|
04 | Cindy Morgan | State changed to IESG Evaluation::Revised I-D Needed from IESG Evaluation |
2013-10-24
|
04 | Barry Leiba | [Ballot discuss] I'm switching to a DISCUSS position until we address the comments that Dave Crocker has brought up in his review: http://www.ietf.org/mail-archive/web/ietf/current/msg83488.html Of particular … [Ballot discuss] I'm switching to a DISCUSS position until we address the comments that Dave Crocker has brought up in his review: http://www.ietf.org/mail-archive/web/ietf/current/msg83488.html Of particular concern are these: 1. ------------------------ > Because of this change in review assumptions, IETF Proposed > Standards should be considered to be at least as mature as final > standards from other standards development organizations. An assertion like this requires some discussion detail that compares the IETF process to specific, other SDO processes, both formally and effectively. I'm not disagreeing with the conclusion, but with its being made rather casually and firmly, but without substantiation. (Part of the real challenge in doing the comparison is looking for alternative forms of quality assurance that other SDOs might do, other than the explicit reviews the IETF does.) ------------------------ 2. ------------------------ > A Proposed Standard specification is stable, has resolved known > design choices, is well-understood, has received significant > community review, and appears to enjoy enough community interest to > be considered valuable. The text has two major flaws that serve to assert a much higher practical maturity for the Proposed Standard documents we publish than is very often observably true: Well understood: For a specification with any interesting level of innovation or complexity, it is almost never 'well understood' at the time it is approved by the IETF. That requires significant implementation, deployment and use experience. The fact that many eyes might have done a static review or even that a couple of folk have written some code or even that those folk interoperated does not come close to qualifying the specification as "well understood". Enough community interest: We often publish specifications that actually have very little community support. This is evidenced by a) very low working group participation levels, and b) failure to successfully deploy and get used. ------------------------ 3. ------------------------ > A Proposed Standard will have no known technical omissions with > respect to the requirements placed upon it. Proposed Standards are > of such quality that implementations can be deployed in the > Internet. And yet we know that this often is not true. However, language like this is going to cause the IESG to naturally raise the bar even higher, for fear that a spec still is not good enough. The real goal of the language change, here, is to give assurances that the specification is "sufficiently" mature, in terms of concern for market deployment. The part that the document can't say, but has as an implicit point since it is motivating the draft, is that the maturity is at least as good as that produced by other SDOs. ------------------------ |
2013-10-24
|
04 | Barry Leiba | [Ballot Position Update] Position for Barry Leiba has been changed to Discuss from Yes |
2013-10-24
|
04 | Gonzalo Camarillo | [Ballot Position Update] New position, No Objection, has been recorded for Gonzalo Camarillo |
2013-10-24
|
04 | Peter Yee | Request for Telechat review by GENART Completed: Ready. Reviewer: Peter Yee. |
2013-10-23
|
04 | Ted Lemon | [Ballot Position Update] New position, Yes, has been recorded for Ted Lemon |
2013-10-23
|
04 | Richard Barnes | [Ballot comment] In general, this document does a really good job of clarifying things that should have been made clear long ago. """ The IETF … [Ballot comment] In general, this document does a really good job of clarifying things that should have been made clear long ago. """ The IETF review is possibly more extensive than that done in most other SDOs owing to the cross-area technical review performed by the IETF. """ This seems like an unnecessary dig at other SDOs. Maybe the following? """ The IETF review is especially extensive owing to the cross-area technical review performed by the IETF. """ |
2013-10-23
|
04 | Richard Barnes | [Ballot Position Update] New position, Yes, has been recorded for Richard Barnes |
2013-10-22
|
04 | Stephen Farrell | [Ballot comment] - "exemplified by technical review by the full IESG at the last stage of specification development" might be problematic if the recent discussion … [Ballot comment] - "exemplified by technical review by the full IESG at the last stage of specification development" might be problematic if the recent discussion about changing the role of the IESG in document review bears fruit. I'm ok that we say that since I'd not bet on the other discussion resulting in short-term change, but it might be prudent to re-word just in case. Not sure I can think of a good rewording though that'd be worth including. - Did section 4 get any reaction in IETF LC or before? If not, I bet we regret that last sentence. - Given the non-IETF audience at which this is partly aimed, it might make sense to say that all proposed standards contain an analysis of security considerations or something. - I agree Benoit's point is important to make and missing. |
2013-10-22
|
04 | Stephen Farrell | [Ballot Position Update] New position, No Objection, has been recorded for Stephen Farrell |
2013-10-22
|
04 | Stewart Bryant | [Ballot Position Update] New position, Yes, has been recorded for Stewart Bryant |
2013-10-22
|
04 | Martin Stiemerling | [Ballot Position Update] New position, Yes, has been recorded for Martin Stiemerling |
2013-10-21
|
04 | Joel Jaeggli | [Ballot Position Update] New position, Yes, has been recorded for Joel Jaeggli |
2013-10-21
|
04 | Brian Haberman | [Ballot Position Update] New position, Yes, has been recorded for Brian Haberman |
2013-10-19
|
04 | Pete Resnick | [Ballot comment] Abstract: The word "new" strikes me as odd here. Saying "new" implies that first we're changing the characterization, and then we're going to … [Ballot comment] Abstract: The word "new" strikes me as odd here. Saying "new" implies that first we're changing the characterization, and then we're going to start behaving in accordance with it. What we're doing is updating the characterization to reflect reality. The Intro gets this right. A suggested update to the Abstract; do with it as you see fit: RFC 2026 describes the review performed by the IESG on IETF Proposed Standard RFCs and states the maturity level of those documents. Since its publication, reviews have become more stringent than described by RFC 2026. This document clarifies those descriptions and updates RFC 2026 by providing a current and more accurate characterization of Proposed Standards. Section 2: the IETF strengthened its review of Proposed Standards, basically operating as if the Proposed Standard was the last chance for the IETF to ensure the quality of the technology and the clarity of the Standard Track document. I suggest adding to this sentence: "before it is deployed on the Internet". I think this may also address Benoit's comment. The IETF review is possibly more extensive than that done in most other SDOs owing to... Whether or not the above is true, I think saying it this way, even with the "possibly", is just hubris. (Given the audience for this document, it has the potential to cause grumbling.) I suggest "The IETF review is at least as extensive as that in most other SDOs owing to..." |
2013-10-19
|
04 | Pete Resnick | [Ballot Position Update] New position, Yes, has been recorded for Pete Resnick |
2013-10-19
|
04 | Adrian Farrel | [Ballot comment] Abstract Does the IESG review Proposed Standard RFCs or does it review Internet- Drafts requested for publication as Proposed Standard RFCs? Similar issue … [Ballot comment] Abstract Does the IESG review Proposed Standard RFCs or does it review Internet- Drafts requested for publication as Proposed Standard RFCs? Similar issue at the top of the Introduction. --- Section 1 "This document exclusively updates..." Could mean "This document updates only..." or "No other document updates..." Suggest you mean the former and change to it. --- I should find this document even more excellent if the Abstract and Introduction both made a summary statement of the "new" characterization so that lazy readers picked this up easily and clearly. |
2013-10-19
|
04 | Adrian Farrel | [Ballot Position Update] New position, Yes, has been recorded for Adrian Farrel |
2013-10-18
|
04 | Benoît Claise | [Ballot comment] In section 2, I believe that you're missing the argument that many Proposed Standards are actually deployed on the Internet, as stable protocols. … [Ballot comment] In section 2, I believe that you're missing the argument that many Proposed Standards are actually deployed on the Internet, as stable protocols. This proves the point that the community deemed unnecessary to upgrade to Internet Standard (for the sake of upgrading, i.e. without some specifications improvements/changes/errata integration). |
2013-10-18
|
04 | Benoît Claise | [Ballot Position Update] New position, Yes, has been recorded for Benoit Claise |
2013-10-18
|
04 | Barry Leiba | [Ballot comment] Spencer's comment looks correct to me. |
2013-10-18
|
04 | Barry Leiba | [Ballot Position Update] New position, Yes, has been recorded for Barry Leiba |
2013-10-17
|
04 | Jean Mahoney | Request for Telechat review by GENART is assigned to Peter Yee |
2013-10-17
|
04 | Jean Mahoney | Request for Telechat review by GENART is assigned to Peter Yee |
2013-10-17
|
04 | Spencer Dawkins | [Ballot comment] In 4. Further Considerations While less mature specifications will usually be published as Informational or Experimental RFCs, the IETF may, on … [Ballot comment] In 4. Further Considerations While less mature specifications will usually be published as Informational or Experimental RFCs, the IETF may, on occasion, publish a specification that still contains areas for improvement or ^^^^^^^^^^^^^ certain uncertainties about whether the best engineering choices are made. I'm almost sure this means "something like publish a proposed standard", but that's not what it says, so I'm guessing. |
2013-10-17
|
04 | Spencer Dawkins | [Ballot Position Update] New position, No Objection, has been recorded for Spencer Dawkins |
2013-10-17
|
04 | (System) | IANA Review state changed to IANA OK - No Actions Needed from Version Changed - Review Needed |
2013-10-17
|
04 | Sean Turner | [Ballot Position Update] Position for Sean Turner has been changed to Recuse from Abstain |
2013-10-17
|
04 | Sean Turner | [Ballot Position Update] New position, Abstain, has been recorded for Sean Turner |
2013-10-17
|
04 | Olaf Kolkman | IANA Review state changed to Version Changed - Review Needed from IANA OK - No Actions Needed |
2013-10-17
|
04 | Olaf Kolkman | New version available: draft-kolkman-proposed-standards-clarified-04.txt |
2013-10-17
|
03 | Jari Arkko | Ballot has been issued |
2013-10-17
|
03 | Jari Arkko | [Ballot Position Update] New position, Yes, has been recorded for Jari Arkko |
2013-10-17
|
03 | Jari Arkko | Created "Approve" ballot |
2013-10-17
|
03 | Jari Arkko | Ballot writeup was changed |
2013-10-17
|
03 | Jari Arkko | State changed to IESG Evaluation from Waiting for Writeup |
2013-10-17
|
03 | Jari Arkko | Placed on agenda for telechat - 2013-10-24 |
2013-10-16
|
03 | Peter Yee | Request for Last Call review by GENART Completed: Ready with Nits. Reviewer: Peter Yee. |
2013-10-16
|
03 | (System) | State changed to Waiting for Writeup from In Last Call (ends 2013-10-16) |
2013-09-24
|
03 | (System) | IANA Review state changed to IANA OK - No Actions Needed from IANA - Review Needed |
2013-09-24
|
03 | Amanda Baber | IESG/Authors/WG Chairs: IANA has reviewed draft-kolkman-proposed-standards-clarified-03, which is currently in Last Call, and has the following comments: We understand that this document doesn't require … IESG/Authors/WG Chairs: IANA has reviewed draft-kolkman-proposed-standards-clarified-03, which is currently in Last Call, and has the following comments: We understand that this document doesn't require any IANA actions. IANA requests that the IANA Considerations section of the document remain in place upon publication. If this assessment is not accurate, please respond as soon as possible. |
2013-09-19
|
03 | Jean Mahoney | Request for Last Call review by GENART is assigned to Peter Yee |
2013-09-19
|
03 | Jean Mahoney | Request for Last Call review by GENART is assigned to Peter Yee |
2013-09-19
|
03 | Tero Kivinen | Request for Last Call review by SECDIR is assigned to Rob Austein |
2013-09-19
|
03 | Tero Kivinen | Request for Last Call review by SECDIR is assigned to Rob Austein |
2013-09-18
|
03 | Amy Vezza | IANA Review state changed to IANA - Review Needed |
2013-09-18
|
03 | Amy Vezza | The following Last Call announcement was sent out: From: The IESG To: IETF-Announce Reply-To: ietf@ietf.org Sender: Subject: Last Call: (Characterization of Proposed Standards) to Best … The following Last Call announcement was sent out: From: The IESG To: IETF-Announce Reply-To: ietf@ietf.org Sender: Subject: Last Call: (Characterization of Proposed Standards) to Best Current Practice The IESG has received a request from an individual submitter to consider the following document: - 'Characterization of Proposed Standards' as Best Current Practice The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2013-10-16. Exceptionally, comments may be sent to iesg@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract RFC 2026 describes the review performed by the IESG on IETF Proposed Standard RFCs and states the maturity level of those documents. This document clarifies those descriptions and updates RFC 2026 by providing a new characterization Proposed Standards. The file can be obtained via http://datatracker.ietf.org/doc/draft-kolkman-proposed-standards-clarified/ IESG discussion can be tracked via http://datatracker.ietf.org/doc/draft-kolkman-proposed-standards-clarified/ballot/ No IPR declarations have been submitted directly on this I-D. |
2013-09-18
|
03 | Amy Vezza | State changed to In Last Call from Last Call Requested |
2013-09-18
|
03 | Jari Arkko | Last call was requested |
2013-09-18
|
03 | Jari Arkko | Ballot approval text was generated |
2013-09-18
|
03 | Jari Arkko | Ballot writeup was generated |
2013-09-18
|
03 | Jari Arkko | State changed to Last Call Requested from AD Evaluation::AD Followup |
2013-09-18
|
03 | Jari Arkko | Last call announcement was generated |
2013-09-18
|
03 | Jari Arkko | Last call announcement was generated |
2013-09-18
|
03 | Jari Arkko | Document shepherd changed to Jari Arkko |
2013-09-18
|
03 | (System) | Sub state has been changed to AD Followup from Revised ID Needed |
2013-09-18
|
03 | Olaf Kolkman | New version available: draft-kolkman-proposed-standards-clarified-03.txt |
2013-09-18
|
02 | Jari Arkko | State changed to AD Evaluation::Revised I-D Needed from AD Evaluation |
2013-09-18
|
02 | Jari Arkko | State changed to AD Evaluation from Publication Requested |
2013-09-18
|
02 | Jari Arkko | Working group state set to Submitted to IESG for Publication |
2013-09-18
|
02 | Jari Arkko | IETF WG state changed to Submitted to IESG for Publication |
2013-09-18
|
02 | Jari Arkko | IESG state changed to Publication Requested |
2013-09-18
|
02 | Jari Arkko | IESG state set to Publication Requested |
2013-09-17
|
02 | Olaf Kolkman | New version available: draft-kolkman-proposed-standards-clarified-02.txt |
2013-09-13
|
01 | Olaf Kolkman | New version available: draft-kolkman-proposed-standards-clarified-01.txt |
2013-09-02
|
00 | Jari Arkko | Apologies for a late reply on this (I was on vacation after the IETF). But thank you for writing this draft. My general comment is … Apologies for a late reply on this (I was on vacation after the IETF). But thank you for writing this draft. My general comment is that the draft makes what in my mind is an accurate correction to our documents, aligning the documents to the current reality. I'd be happy to take the document forward. In fact, I think we need to make this change even if we made other, more long term changes. There is at least one ongoing effort right now that has the potential to reclassify a large set of Proposed Standard RFCs that form the basis of widely used technology. These types of efforts can have a relatively big effect on the standards status of the most commonly used RFCs. Do we want to do more? Can we do more? Secondly, the other obvious action we could take is to go back to the original mode of operation, i.e., making PS RFCs truly early and somewhat untested specifications. I am personally opposed to that on the following grounds. First, it would not change the fact that a large part of Internet technology today runs on PS RFCs, and Olaf's problem with getting these RFCs recognised would continue. Second, while I think we need to keep adjusting the level of review performed by the IESG and in IETF Last Call (we sometimes overdo it), I think broad review is actually useful. But enough about my opinions. What do the rest of you think? In terms of specific text, I also wrote a few observations, below. These are purely personal comments. First, I think you assumed this but never made it explicit. While the new characterisation recognises the often final role of PS RFCs, it does not take away the possibility of publishing Internet Standard specifications. Can this be clarified? In the two decades after publication of RFC 2026 [RFC202] the IESG has evolved its review processes of Proposed Standard RFCs and thus RFC 2026 section 4.1.1 no longer accurately describes IETF Proposed Standards. I'd prefer saying "the IETF review processes Proposed Standard RFCs have evolved". And leave the details to Section 2. 2. IESG Reveiew of Proposed Standards Review In response, the IESG strengthened its review of Proposed Standards, basically operating as if the Proposed Standard was the last chance for the IESG to ensure the quality of the technology and the clarity of the standards document. That is part of it, but I think the situation is more complicated than that. The world changed around us, and suddenly Internet was big business, global, and we got more careful about impacts to it. The process has evolved, including the number of steps in the ladder. Review practices in general have changed quite a lot, we now have a fairly broad review of RFCs. Progression has also varied, mostly downwards. But as noted, it also seems very much affected by specific initiatives. Here's what I'd say: Initially it was assumed that most IETF technical specifications would progress through a series of maturity stages starting with a relatively early Proposed Standard, then progressing to Draft Standard then, finally, to Internet Standard (see RFC 2026 section 6). Over time, for a number of reasons, this progression became less common. At the same time, the review for Proposed Standard RFCs was strengthened. This strengthening was partially a response by the IESG for the above, and in part a consequence of the growth in the importance of the Internet and broader interest in reviewing new Internet technology. At the time of this writing, the IETF operates as if the Proposed Standard was the last chance for the to ensure the quality of the technology and the clarity of the standards document. The result is that IETF Proposed Standards approved over the last decade or more have had extensive review. Because of this change in review assumptions, IETF Proposed Standards should be considered to be at least as mature as final standards from other standards development organizations. In fact, the IETF review is more extensive than is done in other SDOs due to the cross-area technical review performed in the IETF. |
2013-09-02
|
00 | Jari Arkko | Assigned to General Area |
2013-09-02
|
00 | Jari Arkko | Intended Status changed to Best Current Practice |
2013-09-02
|
00 | Jari Arkko | IESG process started in state AD is watching |
2013-09-02
|
00 | Jari Arkko | Stream changed to IETF from None |
2013-09-02
|
00 | Jari Arkko | Shepherding AD changed to Jari Arkko |
2013-08-01
|
00 | Olaf Kolkman | New version available: draft-kolkman-proposed-standards-clarified-00.txt |