Network Working Group B. Carpenter Internet Draft IBM draft-carpenter-icar-sirs-00.txt D. Crocker Expires: <09-04> Brandenburg March 2004 Careful Additional Review of Documents (CARD) by Senior IETF Reviewers (SIRS) (draft-carpenter-icar-sirs-00.txt) COPYRIGHT NOTICE If published as an RFC this document will become Copyright (C) The Internet Society (2003). All Rights Reserved. ABSTRACT IETF specifications do not receive formal review until they are submitted to the IESG. Hence, significant problems with a specification often are not detected until considerable effort has been wasted and changes to fix the problems are difficult to add. The procedure described in this document is intended to solve, or palliate, a number of related problems that have been observed in the IETF process. The basic model is to create a team of Senior IETF Reviewers (SIRS), and have all documents receive a certain number of reviews by SIRs, prior to being submitted for publication. Review at a very early stage is strongly encouraged. STATUS OF THIS MEMO This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. TABLE OF CONTENTS 1. Introduction 2. SIRs 2.1. The Body of Senior Internet Reviewers 2.2. Obtaining SIR Participation 3. Carding 3.1. Reviewing in Public 3.2. Form of a Review 3.3. Iterative Carding 4. Security considerations 5. Acknowledgements 6. Informative References 7. Authors' Addresses 8. Intellectual Property 9. Full Copyright Statement EDITORS' NOTES The acronym "SIR" is retained in this draft simply for reasons of continuity. However, feedback during an experimental period has shown that the community would prefer an acronym without the connotations of the English word "sir." The mechanism proposed in Section 2.1 for selecting the panel of reviewers led to considerable debate during the experimental period. It has not been fundamentally modified in this draft, but the authors recognize that the community may prefer a very different mechanism. Change marks are included, as [[---...---]], to indicate sections of text that differ from the -00 version of this draft. Segments that are marked, but have no included text indicate that text was removed. DISCUSSION VENUE Discussion of this proposal is intended to place on the ICAR mailing list <mailto: icar@ietf.org> INTRODUCTION IETF specifications do not receive formal review until they are submitted to the IESG. Hence, significant problems with a specification often are not detected until considerable effort has been wasted and changes to fix the problems are difficult to add. [[--- ---]] The procedure described in this document is intended to solve, or palliate, a number of related problems that have been observed in the IETF process [PROBLEM]: [[--- ---]] * submission of documents to the IESG that still have significant problems (leading to delay) * failure to detect fundamental problems and Internet- wide issues at an early stage Particularly because of the second point, it is impossible to resolve these problems simply by giving additional responsibility to working groups themselves. An additional procedure is needed. The procedure specified here calls for a team of 'sirs' to 'card'. The term 'card' is used for textiles and pubs. The former usage removes detritus from textiles and prepares it for weaving. The latter vets participants at the door. The term also is an acronym for 'Careful Additional Review of Documents.' [[--- The carding procedure makes no change to the formal process of IETF document development, review, approval, and publication. It is an additional procedure intended to tackle the problems listed above. The basic model is to create a team of Senior IETF Reviewers (SIRs, who need be neither male nor knighted) chosen in a way designed to create trust, and that all documents receive reviews by SIRs, as appropriate during document development. Review at a very early stage is strongly encouraged. ---]] The model is intended to be compatible with, and complementary to, existing mechanisms such as the various Directorates within the IETF and the MIB Doctor system. The remainder of this document described how the team of SIRs is created and refreshed, how the review process works, and how it is used by document authors and working groups to achieve their objectives. 2. SIRS 2.1. The Body of Senior Internet Reviewers 2.1.1. Initial Set of SIRs ---]] [[--- The initial team is defined by objective criteria, to avoid any bias in their selection. It will consist of: * all current IAB members * all former IAB and IESG members, and former WG Chairs, who the Secretariat can contact and who are willing to serve * all current MIB Doctors * all members of existing IETF Directorates * all authors of at least three RFCs, who the Secretariat can contact and who are willing to serve * (other suggestions???) (Current IESG Area Directors are excluded from the pool.) 2.1.2. Addition and Removal of SIRs [[--- The team of SIRs is augmented as needed -- at least once a year year [schedule TBD] by a public nomination process and a voting procedure [TBD] among the existing SIRs. SIRs who do not produce at least five reviews in a given year will be retired from the team. In extremis, SIR status may be removed by a simple majority vote of the team of SIRs. ---]] 2.2. Obtaining SIR Participation For Working Group documents, Working Groups solicit the assistance of SIRs. That is, the general IETF community controls who is authorized as a SIR, but WGs control which specific SIRs provides the formal review that is needed for a given document. A primary goal of this proposal is to ensure that Working Groups benefit from broad experience in the design of Internet technology. Hence it is entirely reasonable that some SIRs reviewing a given document should be subject matter experts. However the full set of input from SIRs is substantially more useful when it includes SIRs from other areas. In particular, cross- area review makes it more likely that architectural and operational impacts outside of the subject matter will be detected. It is therefore strongly recommended that WGs seek a diverse set of SIRs to participate in evaluations, able to cover most if not all IETF Areas between them. Each WG will make its own decision about how its SIRs are selected (e.g. chosen by the WG Chairs, chosen by the document authors concerned, etc.) For individual submissions, the document author(s) will solicit SIR reviews, according to the same principles applied to Working Group documents. There is no fixed number of SIR reviews required prior to submission to the IESG or the RFC Editor. However, it is likely that drafts with at least three positive reviews from SIRs in different areas will experience much shorter IESG review cycles than drafts with fewer positive reviews. Other common sense rules will apply; for example a MIB that has not been reviewed by a MIB Doctor is unlikely to be published. In all likelihood, Drafts without reviews will get worse IESG response time than today, whereas Drafts with reviews will be processed much more rapidly, especially as the IESG's confidence in the SIR procedure increases. 3. CARDING 3.1. Reviewing in Public The current list of SIRs will be available on the IETF web site. [[--- Reviews are posted in two places. One is for public discussion, such as in the relevant working group mailing list. These should be posted in segments, to nvite focused threads of discussion. The second venue is as a complete copy in an IETF Reviews web page. A WG or document author in need of reviews should be able to request them through the web site. ---]] 3.2. Form of a Review [[--- [A more extensive and formalized template for reviews needs to be formulated, to give reviewers guidance about offering comments on such things as efficiency, operations impact, deployment/adoption issues, etc. /d] ---]] Each review must start with a summary statement chosen from or adapted from the following list: * This draft is ready for publication as a [type] RFC. * This draft is on the right track but has open issues, described in the review. * This draft has serious issues, described in the review, and needs to be rethought. * This draft has very fundamental issues, described in the review, and further work is not recommended. The length of a review will vary greatly according to circumstances, and it is acceptable for purely editorial comments to be sent privately. All substantive comments must be included in the public review. SIRs should review for all kinds of problems, from basic architectural or security issues, Internet-wide impact, technical nits, problems of form and format (such as IANA Considerations or incorrect references), and editorial issues. As a draft progresses from its initial, "-00" version towards one that is ready for submission, successive SIR reviews should progress from the general architectural level to the editorial level. The intention is that before a draft is submitted by a WG to the IESG, or by an individual to the RFC Editor, it has already benefited from a level of review equivalent to that traditionally applied by the IESG. 3.3. Iterative Carding The carding of textiles is an iterative process, and so is the carding of documents by SIRs. It is not required that every version of an Internet Draft should be submitted for SIR review. However, it is advisable to request reviews at the very beginning (to check for fundamental issues), as major technical issues are resolved, and again just before the document is submitted for IESG approval. Thus three SIR review cycles per document may be considered the minimum. Both Working Groups and individual submitters should realise that carding should start early (to detect and hopefully fix fundamental problems) and be repeated as often as needed (to avoid submitting inadequate documents to the IESG). By these means, it should be possible to avoid most cases where a document spends a long time in IESG review or, worse, is fundamentally unacceptable to the IESG. 4. SECURITY CONSIDERATIONS This document does not directly impact the operational security of the Internet. 5. ACKNOWLEDGEMENTS Your name could go here! Valuable comments and ideas have come from many sources, especially an earlier draft by Ted Hardie and many members of the IETF 'problem' working group. 6. INFORMATIVE REFERENCES [PROBLEM] IETF Problem Statement, E. Davies (ed.), draft-ietf-problem-issue-statement- 00.txt, work in progress. 7. AUTHORS' ADDRESSES Brian E. Carpenter IBM Zurich Research Laboratory Saeumerstrasse 4 8803 Rueschlikon Switzerland Email: brian@hursley.ibm.com Dave Crocker Brandenburg InternetWorking 675 Spruce Drive Sunnyvale, CA 94086 USA Tel: +1.408.246.8253 dcrocker@brandenburg.com 8. INTELLECTUAL PROPERTY PLACEHOLDER for full IETF IPR Statement if needed. 9. FULL COPYRIGHT STATEMENT Copyright (C) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.