RFC Publication of Errata of Standards Track Documents Considered Harmful
draft-klensin-newtrk-8540style-harmful-00

Document Type Active Internet-Draft (individual)
Last updated 2019-06-07
Stream (None)
Intended RFC status (None)
Formats plain text pdf html bibtex
Stream Stream state (No stream defined)
Consensus Boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date
Responsible AD (None)
Send notices to (None)
Network Working Group                                         J. Klensin
Internet-Draft                                              June 7, 2019
Intended status: Informational
Expires: December 9, 2019

   RFC Publication of Errata of Standards Track Documents Considered
                                Harmful
               draft-klensin-newtrk-8540style-harmful-00

Abstract

   There appear to be some recent trends in the IETF, involving both
   published documents and proposals, to use Informational Documents to
   effectively update Standards Track ones, presenting documents as
   normative while avoiding the requirements for a higher level of
   community review and consensus and relationship tracking that would
   be required, in practice, for Standards Track updates RFC 4460,
   titled "Stream Control Transmission Protocol (SCTP) Specification
   Errata and Issues" and RFC 8540, titled "Stream Control Transmission
   Protocol: Errata and Issues in RFC 4960", were published as
   Informational documents although their clear intent is to update, or
   posit alternatives to some of the provisions of, RFcs 2960 and 4960
   respectively.  This critique suggests that it is undesirable for the
   IETF to publish documents of that type and form, explains the
   reasons, and identifies several alternatives.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on December 9, 2019.

Klensin                 Expires December 9, 2019                [Page 1]
Internet-Draft        RFC Errata Summaries Harmful             June 2019

Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Identifying Issues with Informational Updates or Reflections
       on Standards Track Documents  . . . . . . . . . . . . . . . .   3
     2.1.  Use of Normative Languege . . . . . . . . . . . . . . . .   4
     2.2.  Failure to Explicitly Update Precedessor Documents  . . .   4
     2.3.  Usability: Organization by Erratum Date and Number  . . .   5
     2.4.  IETF Consensus and "Verified" Errata  . . . . . . . . . .   6
   3.  Problem Statement and Alternatives  . . . . . . . . . . . . .   6
   4.  Conclusion  . . . . . . . . . . . . . . . . . . . . . . . . .   8
   5.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   8
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   9
     8.1.  Normative References  . . . . . . . . . . . . . . . . . .   9
     8.2.  Informative References  . . . . . . . . . . . . . . . . .   9
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  11

1.  Introduction

   Recent developments strongly suggest that IETF procedures and
   criteria for accepting and publishing documents, particularly
   Informational documents that appear to modify Standards Track ones,
   are in need of review.  This document is a critique of those
   criteria, using two recent published documents as examples of styles
   and practices that should, in the opinion of the author, not be
   repeated.  It does not alter or update the content of the RFCs
   mentioned in this document as examples, nor does it address the
   substantive content of those RFCs.  The intention of this document is
   to encourage the IETF (and any relevant related bodies) to address
   any issues around using Informational RFCs to demonstrably update
   Standards track RFCs in a different way in the future.

Klensin                 Expires December 9, 2019                [Page 2]
Internet-Draft        RFC Errata Summaries Harmful             June 2019

   RFC 4460, titled "Stream Control Transmission Protocol (SCTP)
   Specification Errata and Issues" [RFC4460] and RFC 8540, titled
   "Stream Control Transmission Protocol: Errata and Issues in RFC 4960"
   [RFC8540], were Informational documents although their clear intent
   was to update RFcs 2960 [RFC2960] and 4960 [RFC4960] respectively or
   at least to posit alternatives to some of the specifications in those
   documents in a way that calls them into doubt.  This critique
   suggests that it is undesirable for the IETF to publish documents of
   that type and form, explains the reasons, and identifies several
   alternatives.

   This document does not alter or update the content of RFCs 2960 or
   4960 in any way, nor does it address the substantive content of RFCs
   4460 or 8540.  While it includes an analysis of the structure,
   organization, and some of the statements of the latter two (focusing
   on RFC 8540 as the more recent example), those documents are only
   particular examples.

2.  Identifying Issues with Informational Updates or Reflections on
    Standards Track Documents

   There have been many discussions in the IETF over the years about
   people who have confused RFCs, including Informational and
   Experimental ones, with IETF Standards.  Other discussions -- which
   some see as related and others do not -- have focused rather more on
   how readers, implementers, and others who depend on standards are
   expected to figure out exactly what documents or portions of
   documents are normatively part of a given standard (see the
   discussion of the NEWTRK effort in Section 3).  While it would be
   difficult to characterize the overall community consensus on these
   points in any simple way, three things are clear.  First, no
   significant changes have been made in this area in the last decade
   (and probably not in the more than two decades since RFC 2026
   [RFC2026] was published).  Second, there is little the IETF or RFC
   Editor can do in terms of procedures or document structure that will
   prevent any confusion that is caused deliberately by organizations
   trying to trick others into believing that their non-standards track
   RFCs (or, for that matter, Internet-Drafts) are actually IETF
   standards.  And finally, that Informational documents that appear to
   be Standards Track ones or updates to them are almost certainly an
   invitation to confusion about what, exactly, is the standard.

   Using the recently-published RFC 8540 as an example, this section
   identifies and discusses several of the issues with updates or
   apparent updates to standards track documents, especially
   Informational ones.  It is worth reiterating that, while RFC 8540 is
   the example used, there are other RFCs that have been published (as
   well as future works that might be introduced) that fall into these

Klensin                 Expires December 9, 2019                [Page 3]
Internet-Draft        RFC Errata Summaries Harmful             June 2019

   areas of concern.  Those include, in particular, RFC 4650, mentioned
   above, which apparently acted as the inspiration and justification
   for 8540.

2.1.  Use of Normative Languege

   The text of RFC 8540 uses standardized, BCP 14, IETF normative
   language [RFC2119] [RFC8174] in many places.  Although most or all
   are quotations from suggested text in errata (see below), Section 2
   of the RFC specifies use of those terms and their interpretation in
   the usual normative sense.

   Even when explicit terminology conforming to RFC 2119 is not used,
   the document strongly implies that the changes it suggests are
   normative and replace text in the earlier document.  For example,
   even the abstract says "It provides deltas to RFC4960", which is
   consistent with approved updates to that document, not just
   suggestions in the relevant errata (see Section 2.2 and Section 2.4).

   Given the long history of concerns and complaints about confusion of
   Informational and Experimental documents with standards track ones,
   use of normative language in RFCs that are not on standards track is
   almost certainly undesirable.  If such use in a particular document
   cannot be avoided, the use should almost certainly be associated with
   clear, document-specific, explanations about the status of the
   document and that terminology, rather than just relying on generic
   boilerplate that few people read carefully.

2.2.  Failure to Explicitly Update Precedessor Documents

   The documents in the RFC Series have always been considered to be
   archival: for historical and other reasons, documents, once issued,
   may be replaced or updated by other documents but are not,
   themselves, ever changed (RFC 4844 Section 4.3 [RFC4844]).  In some
   ways, that principle is in conflict with generally recognized good
   practices for standards documents where it is an advantage to have
   only one current document describing a standard and for references to
   that standard to be stable and point to the latest version.  There
   has, so far, been clear consensus in the IETF that the tradeoffs
   required for an archival series are worth it although there have been
   various efforts to mitigate the perceived conflicts with alternate
   numbering overlays such as BCP and STD numbers and the former FYI
   series.

   However, one of the long-standing consequences with the archival
   policy that RFCs, once issued, are never modified and reissued under
   the same number is that one cannot look at a document and tell
   whether it has been replaced or updated by others or not.  If one has

Klensin                 Expires December 9, 2019                [Page 4]
Internet-Draft        RFC Errata Summaries Harmful             June 2019

   access to an RFC index (and thinks to look), "updated by" and
   "obsoleted by" information is available there or one can search newer
   documents to see the relationship to earlier ones.  However, if an
   Informational RFC is issued that merely summarizes suggested changes,
   whether based on errata or not, there are no such metadata threads --
   the only way one can start with the original document and find the
   commentary involves searching for document numbers in titles, a
   procedure that is notoriously unreliable (in part because such titles
   have been discouraged in the past).  So, whatever the intention was
   for issuing such a document, it is unlikely to be helpful except to
   those who find out about the relationship by other means.

2.3.  Usability: Organization by Erratum Date and Number

   Possibly as a result of wanting to get documents published quickly as
   Informational ones, some of the documents that are the subject of
   this critique are inconsistent with the tradition of the RFC Series
   publishing documents that are comprehensible and useful to the
   reader.

   For example, RFC 8540 is organized in sequential order by errata
   number, which is equivalent in most or all cases to sequential by
   date.  While that may provide a useful historical record (or not), it
   makes it nearly impossible to understand from a protocol or
   implementation perspective unless readers carefully go through all
   pages (more than ninety in the case of 8540) and make their own
   notes.  The RFC actually identifies the most serious part of that
   problem in the last paragraph of the introduction (Section 1) which
   says that the reader "must use care to ensure that a field or item is
   not updated later on within the document."  In some cases, e.g., at
   the end of Section 3.1.2, partial cross-references appear and
   mitigate the problem somewhat, but many of the cross-references that
   could make the document easier to follow are either missing or merely
   provide a strong hint that the reader must study the entire document.

   Similarly, even with the ordering of material as described above, the
   document could have been made considerably more useful if it had
   indexes by topic and by section of the base documents (RFC 4960 in
   the case of 8540).  The IESG has asserted in the past that their
   obligations to the community for documents that they intended to
   publish in the IETF's name included requiring that indexes be added
   when doing so was needed for document clarity or usability.  Such
   indexes were not provided.  Unless the IESG has changed its position
   on its responsibilities and authority, that is, at best, a lost
   opportunity.

Klensin                 Expires December 9, 2019                [Page 5]
Internet-Draft        RFC Errata Summaries Harmful             June 2019

2.4.  IETF Consensus and "Verified" Errata

   The status boilerplate for RFC 8540 indicates that it "represents the
   consensus of the IETF community".  That is fine, but it is unclear
   what IETF consensus represents in the case of a document that appears
   to be normative.  If it is just consensus that the document should be
   published, that should be clear and should be clear in text, not just
   in boilerplate.  Certainly it does not represent consensus that all
   "verified" errata have been accepted by the IETF community as a
   correct description of the listed issues and appropriate fixes for
   them: verification of errata normally involves only authors, WG
   Chairs (if the document was produced by a still-active WG), and
   relevant Area Directors.

   There is, of course, also the very old question of whether a document
   that is approved for publication by the IESG with one "yes" vote and
   the rest of the IESG being on record as having no objection (in
   several cases, after expressing one or more), abstaining, or not
   recording a position (see Section 5).

   Contrary to the apparent claim in the documents that their contents
   simply reproduce the verified errata and the changes they specified,
   the ballot report on the RFC-to-be strongly implies that IESG members
   felt it was entitled to second-guess that text and suggest other
   solutions [LC8540-Ballot].  That suggests another difficulty with the
   approach used in RFC 8540: if the IESG members are correct and the
   proposals in the errata should be adjusted to reflect the IETF
   community's best understanding, then the document is no longer an
   (Informational) errata summary; if the errata contents are preserved,
   then the document probably does not represent IETF consensus about
   what should be done.

3.  Problem Statement and Alternatives

   The discussion above strongly suggests that publishing Informational
   documents in the RFC Series that appear to update Standards Track
   ones is not a good idea.  The WG suggested in its summary for IETF
   Last Call for what became RFC 8540 that an errata listing like that
   provided by RFCs 4460 and 8540 is helpful in producing replacements
   for the original documents [LC8540-Statement] but there is no
   evidence that the same purpose could not be served by retaining the
   same list as an Internet-Draft until the actual replacement document
   is ready to be published and then either discarding that I-D or, if
   the WG felt a need to do so, incorporating the errata listing as an
   appendix in the final document.  Keeping the errata summary as an
   Internet-Draft, rather than publishing it as an RFC would also
   eliminate some of the questions about consensus discussed in
   Section 2.4.

Klensin                 Expires December 9, 2019                [Page 6]
Internet-Draft        RFC Errata Summaries Harmful             June 2019

   In addition to the "just don't do this" approach that is at least
   implicitly suggested above, the IETF has considered several other
   approaches to the problem of Standards Track documents that have
   accumulated extensive errata or otherwise require minor or major
   upgrades.  One thing all of them seem to have in common is that all
   of these alternatives involve Standards Track documents, allowing the
   traditional "updates" and "obsoleted" mechanisms to be used, thereby
   addressing at least some of the concerns described in Section 2.2.
   After that, those proposals diverge, at least in part based on the
   specific view they have of the problem or how much of the problems
   they have wished to take on.

   One recent draft addresses minor (or "non-major") replacements to
   existing documents [ID.draft-roach-bis-documents], but it has become
   clear from discussions on the IETF mailing list (and the document
   itself) that there is disagreement about the circumstances to which
   it is applicable.  At least in its initial draft, Section 4 of that
   Internet-Draft argues strongly for avoiding incremental updates
   rather than generating and publishing replacement documents.

   A difficulty with any "just replace the earlier document" approach is
   that the IETF has often found it to be necessary, or at least
   desirable, to define protocols in several different documents
   covering different aspects or components of the whole and sometimes
   updated or extended separately.  Producing a complete and
   comprehensive revision each time would, even if desirable, simply
   take long enough to be inconsistent with the speed at which the
   Internet has historically evolved.  As those documents are extended
   or updated in turn and documents are obsoleted without everything
   that points to them being replaced (or at least clearly identified
   and their status clarified), there are significant opportunities for
   confusion about what is, or is not, included in a particular standard
   that even a "just replace" model would not solve.  In the 2003-2006
   period, the IETF created a WG, called NEWTRK [NEWTRK-charter]
   [NEWTRK-WG].  NEWTRK was, among other things, intended to address the
   question of interrelationships among standards and updates and to
   provide a framework for documents that -- in the form of
   Applicability Statements [RFC2026] or otherwise -- would be able to
   conveniently describe (or at least identify) the component parts of a
   standard, the relationships among them, implementation status where
   appropriate, and to do so without being limited by the restrictions
   associated with, e.g., RFCs with STD numbers.  While the WG produced
   multiple documents that were intended to address those issues and
   reached rough consensus on at least some of them, the effort never
   led to actual changes.  The reasons for that are probably not worth
   too much effort or speculation now (well over a decade later) but it
   may be appropriate to ask questions about whether the time is now
   right to address the more fundamental issues raised by NEWTRK.

Klensin                 Expires December 9, 2019                [Page 7]
Internet-Draft        RFC Errata Summaries Harmful             June 2019

   Even without opening up the standards process and document model as
   NEWTRK proposed, if a summary of issues in a standard (with or
   without recommendations) is needed before an update or replacement
   document can be generated, that is an entirely appropriate role for
   an Applicability Statement.

4.  Conclusion

   The analysis in this document suggests that Informational documents
   that are written with the appearance of Standards Track ones,
   especially when they appear to update one or more Standards Track
   documents, use language or references that the IETF normally reserves
   for requirements in standards track documents, and/or creates
   confusion about assertions of IETF consensus are, at best,
   undesirable.

   As particularly problematic examples, RFCs 4460 and 8540 are written
   as Informational RFCs whose text strongly suggests that they provide
   definitive updates to Standards Track documents.  That is a bad idea
   for many reasons of which the most important may be that they have
   high potential for creating confusion as to what the relevant
   standard actually is and what features and possible changes actually
   represent IETF consensus.  Their problems are compounded by an
   organizational style --and the absence of comprehensive indexes or
   cross-references -- that makes it quite hard to follow them and the
   recommendations they are making.

   The IETF has multiple alternatives to that approach.  They include
   creating the list of errata as an Internet-Draft but publishing only
   a updating or replacement document in the RFC Series, publishing
   documents of this type only if they are extensively revised to
   explain exactly what they are and to eliminate any language that can
   be construed as either normative or making assertions about IETF
   consensus on technical issues, publishing one or more Applicability
   Statements to describe the issue with the original specification, or
   moving toward standards status documents as envisioned by NEWTRK.
   All but the last are possible today; any of them would be a better
   solution than Informational documents with content and structure
   similar to RFCs 4460 and 8540.

5.  Acknowledgements

   While the initial working draft of this document was largely underway
   before the author reviewed the IESG ballot comments on RFC 8540, the
   ballot comments by Ignas Bagdonas, Ben Campbell, Alissa Cooper,
   Suresh Krishnan, Terry Manderson, Alexey Melnikov, Eric Rescorla,
   Alvaro Retana, Adam Roach, and Martin Vigoureux strongly reinforced

Klensin                 Expires December 9, 2019                [Page 8]
Internet-Draft        RFC Errata Summaries Harmful             June 2019

   the conclusion that this document was necessary and informed some of
   the text.

   Spencer Dawkins and Heather Flanigan made suggestions about an an
   intermediate draft that preceded the first posted version.

6.  IANA Considerations

   This memo includes no requests to or actions for IANA.  However, if a
   Standards Track document contains specifications for IANA and an
   Informational one were to appear to update those instructions, that
   would create ambiguities for IANA as well as the broader community.

7.  Security Considerations

   While this critique does not address security issues directly, the
   security of the Internet is almost certainly improved when the IETF
   does not introduce confusion about the status of its protocols.

8.  References

8.1.  Normative References

   [RFC8540]  Stewart, R., Tuexen, M., and M. Proshin, "Stream Control
              Transmission Protocol: Errata and Issues in RFC 4960",
              RFC 8540, DOI 10.17487/RFC8540, February 2019,
              <https://www.rfc-editor.org/info/rfc8540>.

8.2.  Informative References

   [ID.draft-roach-bis-documents]
              Roach, A., "Process for Handling Non-Major Revisions to
              Existing RFCs", May 2019,
              <https://datatracker.ietf.org/doc/
              draft-roach-bis-documents/>.

   [LC8540-Ballot]
              IETF Internet Engineering Steering Group (IESG), "Stream
              Control Transmission Protocol: Errata and Issues in RFC
              4960: IESG Writeups", 2018,
              <https://datatracker.ietf.org/doc/rfc8540/ballot/>.

   [LC8540-Statement]
              IETF Transport Area Working Group, "Stream Control
              Transmission Protocol: Errata and Issues in RFC 4960:
              Approval Announcement: Working Group Summary", 2018,
              <https://datatracker.ietf.org/doc/rfc8540/writeup/>.

Klensin                 Expires December 9, 2019                [Page 9]
Internet-Draft        RFC Errata Summaries Harmful             June 2019

   [NEWTRK-charter]
              IETF, "New IETF Standards Track Discussion", 2006,
              <https://datatracker.ietf.org/doc/charter-ietf-newtrk/>.

              Retrieved 2019-06-02

   [NEWTRK-WG]
              IETF, "New IETF Standards Track Discussion (newtrk)",
              September 2006,
              <https://datatracker.ietf.org/wg/newtrk/about/>.

              Retrieved 2019-06-02

   [RFC2026]  Bradner, S., "The Internet Standards Process -- Revision
              3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, October 1996,
              <https://www.rfc-editor.org/info/rfc2026>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC2960]  Stewart, R., Xie, Q., Morneault, K., Sharp, C.,
              Schwarzbauer, H., Taylor, T., Rytina, I., Kalla, M.,
              Zhang, L., and V. Paxson, "Stream Control Transmission
              Protocol", RFC 2960, DOI 10.17487/RFC2960, October 2000,
              <https://www.rfc-editor.org/info/rfc2960>.

   [RFC4460]  Stewart, R., Arias-Rodriguez, I., Poon, K., Caro, A., and
              M. Tuexen, "Stream Control Transmission Protocol (SCTP)
              Specification Errata and Issues", RFC 4460,
              DOI 10.17487/RFC4460, April 2006,
              <https://www.rfc-editor.org/info/rfc4460>.

   [RFC4844]  Daigle, L., Ed. and Internet Architecture Board, "The RFC
              Series and RFC Editor", RFC 4844, DOI 10.17487/RFC4844,
              July 2007, <https://www.rfc-editor.org/info/rfc4844>.

   [RFC4960]  Stewart, R., Ed., "Stream Control Transmission Protocol",
              RFC 4960, DOI 10.17487/RFC4960, September 2007,
              <https://www.rfc-editor.org/info/rfc4960>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

Klensin                 Expires December 9, 2019               [Page 10]
Internet-Draft        RFC Errata Summaries Harmful             June 2019

Author's Address

   John C Klensin
   1770 Massachusetts Ave, Ste 322
   Cambridge, MA  02140
   USA

   Phone: +1 617 245 1457
   Email: john-ietf@jck.com

Klensin                 Expires December 9, 2019               [Page 11]