Procedure for Standards Track Documents to Refer Normatively to External Documents
draft-kucherawy-bcp97bis-05
Discuss
Yes
Erik Kline
No Objection
Recuse
Murray Kucherawy
No Record
Deb Cooley
Francesca Palombini
Jim Guichard
Mahesh Jethanandani
Summary: Has 3 DISCUSSes. Needs 3 more YES or NO OBJECTION positions to pass.
John Scudder
Discuss
Discuss
(2022-10-24 for -04)
Sent
I acknowledge that Section 4.1 is largely inherited from RFC 4897, but I'm scratching my head over the importance of a document being "stable"... without ever defining what "stability" means, for example, The exception to this process is the unusual case where the source document is less stable than the target document, even if the target document is at a lower maturity level. In such cases, the above notation is omitted. If you pressed me to define document "stability" I'd first reach for whether it's a good-quality archival reference. But that can't be it, because all RFCs are good archival references, no matter their maturity level. Because of this, I don't know how to apply the standard set out by Section 4.1. A possible fix would be to remove all the sentences and clauses that mention "stable", a different one would be to define what it means.
Comment
(2022-10-24 for -04)
Sent
## COMMENT ### Section 4.1: I've read this a few times and I don't understand what it's telling me, sorry: There is no separate review of these references. For example, when the downref is to a document of a lower maturity level that is understood to be stable, the annotation can be omitted. It looks like the "for example" grafts the final sentence on to what was in RFC 4897, but I don't get how that sentence is an example of "there is no separate review", it just seems like an additional piece of specification. (But see also my discuss about "stable".) ### Section 4.2: This section says that: In the case of a downref approved in this manner, the annotations described above should be added to the reference unless the IESG, after consideration of Last Call input, concludes it is inappropriate. I've never seen one of these annotations in the wild. I went and checked the last eight references in the Downref registry (https://datatracker.ietf.org/doc/downref/) so I could see what one looked like. There aren't any. Unless I'm confused, it appears this rule is honored mainly in the breach. As such, perhaps it doesn't make sense to propagate it forward into bcp97bis. Alternately, if we are going to keep it as a requirement, then I guess we should start doing it. I am not advocating the latter approach, but it does seem like we gotta do one or the other. ### Section 5: I agree with Éric's comment that some more precise term than "freely available" is desirable. ### Section 6: The IETF has previously established a registry of downrefs to RFCs that have been previously waived by the IESG in the manner described Is there a reason there isn't a reference to the Downref registry's location? (That being, https://datatracker.ietf.org/doc/downref/). Would the reference be... normative? (Also, how about dropping one of the "previously"s? Probably the first one.) ### Section 6: The first paragraph and the second are in almost-conflict: Going forward, new registry entries should also record the reason the registry addition was made, the name of the external body owning the target reference, and the name of the Area Director making the new entry. Note that there is currently no registry of downrefs to non-IETF documents. That is to say, the first paragraph tells us that we have to include "the name of the external body" (by the way, shouldn't there be "(if any)" appended to that?). But the second tells us there's no such animal. I guess the first paragraph implies we're going to start tracking non-IETF downrefs, and the second paragraph should be deleted? ## NITS - s/very simple procedure/procedure/ - s/docuemnt/document/ - Why is RFC 3339 a "possible" example? Isn't it just... an example?
Roman Danyliw
Discuss
Discuss
(2022-10-24 for -04)
Sent
** Section 4.2 When a document is presented to the IESG for publication that contains a downref not called out by the author/editor(s) as described in the previous section, it is strongly recommended that the normal IETF Last Call procedure will be issued, with the need for the downref explicitly documented in the Last Call itself. In the first paragraph, what’s the more precise definition of the state in which a “document is presented to the IESG” that doesn’t call out the downref? Is that the WG submitting it as a PubReq to the AD? Or the full IESG for telechat review? If it’s the former (AD Review) then why can’t the document just be updated with the new style of reference with any other AD Review feedback? If it is the latter (telechat), is this text suggesting _another_ IETF LC since the document won’t reach the full IESG without going through an initial IETF LC? A duplicate IETF LC seems to be covered in the next paragraph.
Comment
(2022-10-24 for -04)
Sent
** Section 4.1 * A note is included in the reference text that indicates that the reference is to a target document of a lower maturity level, that some caution should be used since it may be less stable than the document from which it is being referenced, and, optionally, explaining why the downref is appropriate. A process aside: are we going to hold publication until we have coordinated with the RFC Editor on how such a note would look, and “establish[ed] procedures regarding the text to use, or give guidance on this text.” ** Section 4.2 This should only occur when the same document (and version) ... Such documents are added to the "Downref Registry" defined in Section 6. The first phrase comes from RFC3967. I’m wondering about the intent of the parenthetical. The DOWNREF registry only has RFCs. What is the versioning envisioned here? See related comments in Section 6. ** Section 6. Going forward, new registry entries should also record the reason the registry addition was made, the name of the external body owning the target reference, and the name of the Area Director making the new entry. Note that there is currently no registry of downrefs to non-IETF documents. I’m not sure how to take the observation in the last paragraph. Are we expanding the aperture of the downref registry? POSSIBLE NEW TEXT (to replace the current last sentence): Going forward, the applicability of this downref registry is expanded to include non-IETF documents.
Warren Kumari
Discuss
Discuss
(2022-10-26 for -04)
Not sent
I'm still very unclear *exactly* what problem this is intended to address; we can already normatively refer to lower level documents. This document seems to add a *large* amount of process for the authors, the IESG, the community/WGs, and possibly the RFC Editor as well, and I'm not sure what the gain is here. As Alvaro pointed out, we already have the "annotation" procedure in RFC4897 (published more than 15 years ago), and we don't actually use it, and nothing seems to have broken. We also already have the "misref" bit solved - it's part of normal IETF processing. RFC3967, RFC4897, RFC8067 are important documents, and I don't think that we should be obsoleting them without a really good reason - and I don't see that here.
Erik Kline
Yes
Éric Vyncke
Yes
Comment
(2022-10-24 for -04)
Sent
# Éric Vyncke, INT AD, comments for draft-kucherawy-bcp97bis-04 CC @evyncke Thank you for the work put into this document. Please find below some non-blocking COMMENT points (but replies would be appreciated even if only for my own education). Please note that Patrick Mevzek is the DNS directorate reviewer (at chairs' request) and you may want to consider his dns-dir reviews as well (mainly nits): https://datatracker.ietf.org/doc/review-kucherawy-bcp97bis-03-dnsdir-lc-mevzek-2022-10-16/ Special thanks to Erik Kline for the shepherd's detailed write-up. I hope that this review helps to improve the document, Regards, -éric ## COMMENTS ### Section 2 Suggest not to use the (too) old MD5 as an example... ### Section 4.1 s/A note is included in the reference text/An optional note is included in the reference text/ ? ### Section 5 In `it may not be freely available` should 'freely' be more descriptive ? I.e., behind a paywall ? or restricted publication to members only ? Suggest to use 'cost free' (or any more English wording) ``` Note that there is no requirement for a freely available copy of the reference beyond the publication of the draft as an RFC. ``` To be honest, I am a little puzzled with the above text: if a required/normative reference is no more available after publication, then how can the specification be implemented ? But, this depends what is meant by 'freely', if it means 'without any cost' then this is fine, if it means 'availably only to members of a 3rd party', then this is annoying to say the least. ### Section 6 How can authors find this 'downref registry'? Should it be documented/specified ? (to be honest, I have no clue as I rely on idnits...) ## Notes This review is in the ["IETF Comments" Markdown format][ICMF], You can use the [`ietf-comments` tool][ICT] to automatically convert this review into individual GitHub issues. [ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md [ICT]: https://github.com/mnot/ietf-comments
Gunter Van de Velde
No Objection
Comment
(2024-04-03 for -04)
Sent
# Gunter Van de Velde, RTG AD, comments for draft-kucherawy-bcp97bis-04 Thank you for the work put into this document. Please find below some non-blocking COMMENT points. 132 other implementations of that standard. For documents that are 133 referenced, any document that includes key information an implementer 134 needs would be normative. For example, if one needs to understand a What about following rewrite proposal to make the text read easier: "For referenced documents, any document containing essential information required by an implementer would be considered normative." 164 There are a number of circumstances in which an IETF document may 165 need to make a normative reference to a document at a lower maturity 166 level, but such a reference conflicts with Section 4.2.4 of 167 [RFC2026]. For example: I find that in RFC2026 the section "4.2.4 Historic". Not sure how the pieces fit together. mostlikely due to being too much rookie AD to understand the conflict reference 162 2. The Need for Downward References 164 There are a number of circumstances in which an IETF document may 165 need to make a normative reference to a document at a lower maturity 166 level, but such a reference conflicts with Section 4.2.4 of 167 [RFC2026]. For example: Should it be spelled out more explicit that this list is not a limited list? 200 The term "Standards-Track document", as used in this specification, 201 is assumed to include only Standards-Track documents at any maturity 202 level, plus BCPs, but not documents with any other status. Would this include the historical documents that were valid when RFC was written, but became historical after few years due to technology evolution. (i guess i may be thrown of the rails by the reference to RFC2026 section 4.2.4 from earlier in the document) 238 At the option of the author/editor, similar notes may be attached to 239 non-normative references. is this text necessary? this seems irrelevant for normative down-refs 241 The exception to this process is the unusual case where the source 242 document is less stable than the target document, even if the target 243 document is at a lower maturity level. In such cases, the above 244 notation is omitted. 3 lines up it was already mentioned that the annotation can be omitted. not sure if these lines can be combined instead? 246 When the document is prepared to submit to the IESG for approval, a 247 document shepherd writeup [RFC4858] is normally prepared. This What about following readability rewrite: "Upon preparing a document for submission to the IESG for approval, it is customary to create a document shepherd write-up, as outlined in [RFC4858]". 271 identified. If it elects not to do so, then any future downref to 272 the same target document is subject again to the procedures described 273 in this document. In making this decision, the IESG should take into I am not sure what "any future downref to the same target document" exactly means. Does this mean that once a target document is used once as normative reference, that in futire instances for other source documents it can be used without special annotation notes? 311 * the actual reference to it (e.g., a web link) may have dubious 312 location stability; I am not a big fan of the word 'dubious'. It implies it it obscure or malicious or something else negative. Using the wording 'questionable' has less negative connotation. a possible rewrite option: 'The stability of the actual reference's location, such as a web link, may be questionable.' 348 6. Downref Registry 350 The IETF has previously established a registry of downrefs to RFCs 351 that have been previously waived by the IESG in the manner described 352 in Section 4.2. The registry includes the name of the referenced RFC How to access this registry? (i as rookie AD has no idea how to access this or how it is instantiated Be well, G/
Orie Steele
No Objection
Comment
(2024-03-29 for -04)
Not sent
I support the other discusses, in particular the comments from John Scudder
Paul Wouters
(was Discuss)
No Objection
Comment
(2024-06-05)
Sent
The text surrounding my DISCUSS item has vanished from the document, which was a clever way to resolve my issue :)
Zaheduzzaman Sarker
No Objection
Comment
(2022-10-27 for -04)
Not sent
Thanks for working on this. However I am not sure what it the final outcome of this work other than adding some notes in the references. Hence, I am supporting Warren's discuss.
Murray Kucherawy
Recuse
Deb Cooley
No Record
Francesca Palombini
No Record
Jim Guichard
No Record
Mahesh Jethanandani
No Record
Alvaro Retana Former IESG member
Discuss
Discuss
[Treat as non-blocking comment]
(2022-10-25 for -04)
Sent
(1) The annotation procedure in §4.1 was originally defined in rfc4897 more than 15 years ago. I have seen plenty of downrefs, but I have yet to see this process used. This DISCUSS point challenges the continued inclusion of this procedure as a best current practice as it seems to have never been adopted. (2) §4.2: With appropriate community review, the IESG may establish procedures for when normative downref should delay a document and when downrefs should simply be noted. Absent specific guidance, authors and reviewers should use their best judgment. It is assumed that, in a significant majority of cases, noting a downref is preferable to delaying publication. The IESG has already documented this guidance in the 2006 "IESG Statement: Normative and Informative References", where it says: The distinction between normative and informative references is often important. The IETF standards process according to RFC 2026 and RFC 3967, and the RFC Editor publication process, both need to know whether a reference to a work in progress is normative. An RFC cannot be published until all of the documents that it lists as normative references have been published. In practice, this often results in the simultaneous publication of a group of interrelated RFCs. It is clear that the IESG Statement expects publication to be delayed (vs. just noting the downref).
Lars Eggert Former IESG member
No Objection
No Objection
(2022-10-27 for -04)
Sent
# GEN AD review of draft-kucherawy-bcp97bis-04 CC @larseggert Thanks to Vijay Gurbani for the General Area Review Team (Gen-ART) review (https://mailarchive.ietf.org/arch/msg/gen-art/5ZJZXPwzpBcB3uuIPCiJMLQ6gKU). ## Comments I agree with most of the other Discusses, which is why I decided to just comment on this document rather than duplicating them. Generally, I feel we need a much more lightweight process than we currently have and that we would have if we adopted this revision. ### Section 2, paragraph 0 ``` 2. The Need for Downward References ``` Might want to add the very common example of needing to normatively cite a cryptographic algorithm described in a CFRG document. ### Section 3, paragraph 2 ``` A reference involves two documents, the one in which the reference is embedded and the document referenced. Where needed for clarity, these documents are referred to as the "source document" and "target document", respectively. ``` But which is which? I assume the "source" is the one containing the reference, and the "target" is being references? ### Section 4.1, paragraph 4 ``` * A note is included in the reference text that indicates that the reference is to a target document of a lower maturity level, that some caution should be used since it may be less stable than the document from which it is being referenced, and, optionally, explaining why the downref is appropriate. ``` Is there an xml2rfc and ideally kramdown-rfc way of doing that easily? ### Section 4.1, paragraph 5 ``` There is no separate review of these references. For example, when the downref is to a document of a lower maturity level that is understood to be stable, the annotation can be omitted. ``` Understood by whom? And what does stable mean? I think this concept needs to be removed from the process. ### Section 4.1, paragraph 5 ``` At the option of the author/editor, similar notes may be attached to non-normative references. ``` That seems redundant and would make it more difficult to spot the downrefs. ### Section 4.1, paragraph 5 ``` The exception to this process is the unusual case where the source document is less stable than the target document, even if the target document is at a lower maturity level. In such cases, the above notation is omitted. ``` We don't have a defined notion of document stability. (See above, I think this concept needs to go, or it needs to be clarified.) ### Section 4.2, paragraph 1 ``` With appropriate community review, the IESG may establish procedures for when normative downref should delay a document and when downrefs should simply be noted. Absent specific guidance, authors and reviewers should use their best judgment. It is assumed that, in a significant majority of cases, noting a downref is preferable to delaying publication. ``` I think our procedure is the DOWNREF registry. Would suggest to remove this and just merge Section 6 here. ### Section 4.2, paragraph 4 ``` understanding of the relevant technical area. For example, the use of MD5 [RFC1321] and HMAC [RFC2104] is well known among cryptographers. Such documents are added to the "Downref Registry" defined in Section 6. ``` Would suggest to replace MD5 with another example. ### Section 6, paragraph 3 ``` Note that there is currently no registry of downrefs to non-IETF documents. ``` Should there be? I think yes. ## Nits All comments below are about very minor potential issues that you may choose to address in some way - or ignore - as you see fit. Some were flagged by automated tools (via https://github.com/larseggert/ietf-reviewtool), so there will likely be some false positives. There is no need to let me know what you did with these suggestions. ### Typos #### Section 4.1, paragraph 8 ``` - When the document is prepared to submit to the IESG for approval, a - --------- + When the document is to be submitted to the IESG for approval, a + +++ +++ ``` #### Section 4.1, paragraph 8 ``` - writeup should contain a description of any downrefs that appear in - ^^^^^^ - the document, and should make particular note of any downref that is - ^^^^^^ + writeup SHOULD contain a description of any downrefs that appear in + ^^^^^^ + the document, and SHOULD make particular note of any downref that is + ^^^^^^ ``` #### Section 5, paragraph 11 ``` - that captures the important parts of the intended target docuemnt. - - + that captures the important parts of the intended target document. + + ``` ### Grammar/style #### Section 1, paragraph 5 ``` ly understand or implement the subject matter in the RFC, or whose contents ^^^^^^^^^^^^^^ ``` This phrase is redundant. Consider using "subject" to avoid wordiness. #### Section 5, paragraph 7 ``` ing access to those documents is to be be specified in the shepherd writeup. ^^^^^ ``` Possible typo: you repeated a word. ## Notes This review is in the ["IETF Comments" Markdown format][ICMF], You can use the [`ietf-comments` tool][ICT] to automatically convert this review into individual GitHub issues. Review generated by the [`ietf-reviewtool`][IRT]. [ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md [ICT]: https://github.com/mnot/ietf-comments [IRT]: https://github.com/larseggert/ietf-reviewtool
Martin Duke Former IESG member
No Objection
No Objection
(2022-10-24 for -04)
Sent
- I don't understand this paragraph: "The exception to this process is the unusual case where the source document is less stable than the target document, even if the target document is at a lower maturity level. In such cases, the above notation is omitted." So if there's a Experimental document that normatively references an Informational reference, but the Info document is though to be "more stable", we just don't annotate anything in the references section at all? To me, that seems more confusing than just explicitly stating something in the text. - s/docuemnt/document
Robert Wilton Former IESG member
No Objection
No Objection
(2022-10-27 for -04)
Sent
I support Alvaro's discuss. Hi Murray, Thanks for this document, I'm always up for a single new RFC that replaces 3 others. However, I also support Alvaro's discuss. I'm not sure that the new process, or reaffirming an existing process that nobody follows, is necessary. Specifically, I'm not quite sure what problem we are aiming at solving? (1) p 3, sec 2. The Need for Downward References * There are exceptional procedural or legal reasons that force the target of the normative reference to be an informational or historical RFC or to be at a lower standards level than the referring document. Would it be worth including the example of "Architecture" or "Framework" documents that are quite plausibly defined as being informational, but are still reasonably likely to be normatively referenced? (2) p 4, sec 4.1. Authors and Editors When a Standards-Track or BCP document requires a normative reference to a document of lower maturity, the authors/editors should apply the following very simple procedure: Is there any downref from a Full Internet Standard to a Draft Standard? Is there any downref from Informational to Experimental? (3) p 5, sec 4.2. The IESG With appropriate community review, the IESG may establish procedures for when normative downref should delay a document and when downrefs should simply be noted. Absent specific guidance, authors and reviewers should use their best judgment. It is assumed that, in a significant majority of cases, noting a downref is preferable to delaying publication. I don't understand this, why would a downref delay publication of a document? What would it be waiting for? Do you mean delay by reissuing the IETF last call? (4) p 5, sec 4.2. The IESG When a document is presented to the IESG for publication that contains a downref not called out by the author/editor(s) as described in the previous section, it is strongly recommended that the normal IETF Last Call procedure will be issued, with the need for the downref explicitly documented in the Last Call itself. Any community comments on the appropriateness of downrefs will be considered by the IESG as part of its deliberations. I think that it is probably simpler for downrefs (not in the registry) to always be called out in the last call. Don't we have tooling that is meant to already spot this? (5) p 6, sec 5. Non-IETF Target Documents Authors and editors should try to avoid such references due to the challenges they present, as they affect the IETF's ability to ensure the quality of its output. However, such references are not always avoidable. Should "references" be replaced with "normative references"? Nit level comments: (6) p 3, sec 2. The Need for Downward References * A standards document may need to refer to a proprietary protocol, and the IETF normally documents proprietary protocols using informational RFCs. Did you mean "standards track document" for "standards document" here? (7) p 4, sec 3. Definitions The term "Standards-Track document", as used in this specification, is assumed to include only Standards-Track documents at any maturity level, plus BCPs, but not documents with any other status. Here you refer to "standards-track document", in other places it has been referred to without the hyphen and capitalization. Regards, Rob