Improving "Rough Consensus" with Running Code
draft-sheffer-running-code-01

Network Working Group                                         Y. Sheffer
Internet-Draft                                                  Porticor
Intended status: Experimental                                  A. Farrel
Expires: June 20, 2013                                           Juniper
                                                       December 17, 2012

             Improving "Rough Consensus" with Running Code
                     draft-sheffer-running-code-01

Abstract

   This document describes a simple process that allows authors of
   Internet-Drafts to record the status of known implementations.  This
   will allow reviewers and working groups to assign due consideration
   to documents that have the benefit of running code and potentially
   reward the documented protocols by treating the documents with
   implementations preferentially.

   The process in this document is offered as an experiment.  Authors of
   Internet-Drafts are encouraged to consider using the process for
   their documents, and working groups are invited to think about
   applying the process to all of their protocol specifications.

   The authors of this document intend to collate experiences with this
   experiment and to report them to the community.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on June 20, 2013.

Copyright Notice

   Copyright (c) 2012 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

Sheffer & Farrel          Expires June 20, 2013                 [Page 1]
Internet-Draft                Running Code                 December 2012

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.    Introduction  . . . . . . . . . . . . . . . . . . . . . . . . 3
   2.    The "Implementation Status" Section . . . . . . . . . . . . . 4
   3.    Alternative Formats . . . . . . . . . . . . . . . . . . . . . 4
   4.    Benefits  . . . . . . . . . . . . . . . . . . . . . . . . . . 5
   5.    Process Experiment  . . . . . . . . . . . . . . . . . . . . . 5
   5.1.  Duration  . . . . . . . . . . . . . . . . . . . . . . . . . . 5
   5.2.  Summary Report  . . . . . . . . . . . . . . . . . . . . . . . 6
   5.3.  Success Criteria  . . . . . . . . . . . . . . . . . . . . . . 6
   6.    Implementation Status . . . . . . . . . . . . . . . . . . . . 6
   7.    Security Considerations . . . . . . . . . . . . . . . . . . . 6
   8.    IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7
   9.    Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . 7
   10.   References  . . . . . . . . . . . . . . . . . . . . . . . . . 7
   10.1. Normative References  . . . . . . . . . . . . . . . . . . . . 7
   10.2. Informative References  . . . . . . . . . . . . . . . . . . . 7
         Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . 7

Sheffer & Farrel          Expires June 20, 2013                 [Page 2]
Internet-Draft                Running Code                 December 2012

1.  Introduction

   Most IETF participants are familiar with the saying, "rough consensus
   and running code" [tao], and can identify with its pragmatic
   approach.  However, there are many examples of Internet-Drafts
   containing protocol specification that have gone through to
   publication as Proposed Standard RFCs without implementation.  Some
   of them may never get implemented.

   Over time, a variety of policies have been implemented within the
   IETF to consider running code.  In the Routing Area it used to be a
   requirement that one or more implementations must exist before an
   Internet-Draft could be published as a Proposed Standard RFC
   [RFC1264].  That RFC was later obsoleted and the requirement for
   implementation was lifted, but each working group was given the
   authority to impose its own implementation requirements [RFC4794] and
   at least one working group (IDR) continues to require two independent
   implementations.

   The hypothesis behind this document is that there are benefits to the
   IETF standardization process of producing implementations of protocol
   specifications before publication as RFCs.  These benefits, which
   include determining that the specification is comprehensible and that
   there is sufficient interest to implement, are further discussed in
   Section 4.

   This document describes a simple process that allows authors of
   Internet-Drafts to record the status of known implementations.  This
   will allow reviewers and working groups to assign due consideration
   to documents that have the benefit of running code and potentially
   reward the documented protocols by treating the documents with
   implementations preferentially.

   Contrary to proposals to fast-track Internet-Drafts that have
   associated source code implementations [I-D.farrell-ft], this
   document provides a mechanism to record and publicize the existence
   of running code.  It is up to the individual working groups to use
   this information as they see fit, but one result might be the
   preferential treatment of documents resulting in them being processed
   more rapidly.

   The process in this document is offered as an experiment.  Authors of
   Internet-Drafts are encouraged to consider using the process for
   their documents, and working groups are invited to think about
   applying the process to all of their protocol specifications.

   The authors of this document intend to collate experiences with this
   experiment and to report them to the community.

Sheffer & Farrel          Expires June 20, 2013                 [Page 3]
Internet-Draft                Running Code                 December 2012

2.  The "Implementation Status" Section

   Each Internet-Draft may contain a section entitled "Implementation
   Status".  This section, if it appears, should be located just before
   the "Security Considerations" section and contain, for each existing
   implementation:

   o  The implementation's name and/or a link to a web page describing
      the implementation.
   o  A brief general description.
   o  The implementation's level of maturity: alpha, beta, production,
      or widely used.
   o  Coverage: which parts of the protocol specification are
      implemented and which versions of the Internet-Draft were
      implemented.
   o  Licensing: whether the implementation is closed source, shareware,
      or open source.

   In addition, this section can contain information about the
   interoperability of any or all of the implementations.

   Since this information is necessarily time-dependent, the authors
   should remove this section when the I-D is published as an RFC.

3.  Alternative Formats

   Sometimes it can be an advantage to publish the implementation status
   separately from the base Internet Draft, e.g. on the IETF wiki:

   o  When the Implementation Status section becomes too large to be
      conveniently managed within the document.
   o  When a working group decides to have implementors, rather than
      authors, keep the status of their implementations current.
   o  When a working group already maintains an active wiki and prefers
      to use it for this purpose.

   It is highly important that all readers of the Internet Draft should
   be made aware of this information.  Initially this can be done by
   replacing the Implementation Status section with a URL pointing to
   the wiki.  Later, the IETF Tools may support this functionality, e.g.
   by including such a link from the HTMLized draft version, similarly
   to the IPR link.

   The implementation status information must be publicly available, in
   line with general IETF policy.

Sheffer & Farrel          Expires June 20, 2013                 [Page 4]
Internet-Draft                Running Code                 December 2012

4.  Benefits

   Publishing the information about implementations provides the working
   group with several benefits:

   o  Participants are motivated to implement protocol proposals, which
      helps in discovering protocol flaws at an early stage.
   o  Other participants can use the software, to evaluate the
      usefulness of protocol features.
   o  WG members may choose to perform interoperability testing with
      known implementations, especially when they are publicly
      available.
   o  In the case of open source, people may want to study the code to
      better understand the protocol and its limitations.
   o  Working group chairs and ADs may use the information provided to
      help prioritize the progress of I-Ds in some way [I-D.farrell-ft].
   o  And lastly, some protocol features may be hard to understand, and
      for such features, the mere assurance that they can be implemented
      is beneficial.

   We do not specify here whether and to what degree working groups are
   expected to prefer proposals that have "running code" associated with
   them, over others that do not.

5.  Process Experiment

   The current proposal is proposed as an experiment.  The inclusion of
   "Implementation Status" sections in Internet-Drafts is not mandatory,
   but the authors of this document wish to encourage authors of other
   Internet-Drafts to try out this simple process to discover whether it
   is useful.  Working group chairs are invited to suggest this process
   to document editors in their working groups, and to draw the
   attention of their working group participants to "Implementation
   Status" sections where they exist.

   Following a community discussion, it was concluded that [RFC3933] is
   not an appropriate framework for this experiment, primarily because
   no change is required to any existing process.

5.1.  Duration

   Given the typical time to produce an RFC (see [stats]), we propose a
   duration of 18 months for the experiment.  Thus, 18 months after the
   date of initial posting of this document, the authors will report on
   the experiment as described in the next section.

Sheffer & Farrel          Expires June 20, 2013                 [Page 5]
Internet-Draft                Running Code                 December 2012

5.2.  Summary Report

   The authors will summarize the results of the experiment at the end
   of the period assigned to the experiment (see Section 5.1).  If
   nothing happens (no I-Ds or only a handful include an "Implementation
   Status" section), an email to the IETF list will be sufficient.  This
   would obviously constitute a failure to adopt the idea and the
   authors will abandon the experiment.

   If this idea is adopted by document authors, a summary I-D will be
   written containing the statistics of such adoption, as well as
   (necessarily subjective) reports by working group chairs and area
   directors who have used this mechanism.

   The authors may then propose more wide-scale use of the process and
   might suggest more formal adoption of the process by the IETF.

5.3.  Success Criteria

   The goal of this experiment is to improve the quality of IETF
   specifications.  This is impossible to quantify, of course.  We
   suggest that generally positive answers to the following questions
   would indicate that the experiment was successful:

   o  Did the working group make decisions that were more informed when
      comparing multiple competing solutions for the same work item?
   o  Did authors significantly modify proposed protocols based on
      implementation experience?
   o  Did disclosure of implementations encourage more interoperability
      testing than previously?
   o  Did non-authors review documents based on interactions with
      running code and/or inspection of the code itself?
   o  Did the experiment result in toy implementations, aimed at
      "gaining points" at the IETF, or were they real and useful?

6.  Implementation Status

   This is a process document and therefore does not have any meaningful
   implementation status.  "Implementation" in the context of this
   document means actual program code.

7.  Security Considerations

   This is a process document and therefore, it does not have a direct
   effect on the security of any particular IETF protocol.  Better
   reviewed protocols are likely to also be more secure.

Sheffer & Farrel          Expires June 20, 2013                 [Page 6]
Internet-Draft                Running Code                 December 2012

8.  IANA Considerations

   None.

9.  Acknowledgements

   This document was prepared using the lyx2rfc tool, and we would like
   to thank Nico Williams, its author.

10.  References

10.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

10.2.  Informative References

   [RFC1264]  Hinden, R., "Internet Engineering Task Force Internet
              Routing Protocol Standardization Criteria", RFC 1264,
              October 1991.

   [RFC3933]  Klensin, J. and S. Dawkins, "A Model for IETF Process
              Experiments", BCP 93, RFC 3933, November 2004.

   [RFC4794]  Fenner, B., "RFC 1264 Is Obsolete", RFC 4794,
              December 2006.

   [I-D.farrell-ft]
              Farrell, S., "A Fast-Track way to RFC with Running Code",
              draft-farrell-ft-02 (work in progress), December 2012.

   [stats]    Arkko, J., "Distribution of Processing Times",
              December 2012,
              <http://www.arkko.com/tools/lifecycle/wgdistr.html>.

   [tao]      Hoffman, P., Ed., ""The Tao of IETF: A Novice's Guide to
              the Internet Engineering Task Force", 2012,
              <http://www.ietf.org/tao.html>.

Sheffer & Farrel          Expires June 20, 2013                 [Page 7]
Internet-Draft                Running Code                 December 2012

Authors' Addresses

   Yaron Sheffer
   Porticor
   10 Yirmiyahu St.
   Ramat HaSharon  47298
   Israel

   Email: yaronf.ietf@gmail.com

   Adrian Farrel
   Juniper Networks

   Email: adrian@olddog.co.uk

Sheffer & Farrel          Expires June 20, 2013                 [Page 8]