Internationalized Domain Names for Applications (IDNA) Review for New Unicode Versions
RFC 8753

Received changes through RFC Editor sync (created alias RFC 8753, changed title to 'Internationalized Domain Names for Applications (IDNA) Review for New Unicode Versions', changed abstract to 'The standards for Internationalized Domain Names in Applications (IDNA) require a review of each new version of Unicode to determine whether incompatibilities with prior versions or other issues exist and, where appropriate, to allow the IETF to decide on the trade-offs between compatibility with prior IDNA versions and compatibility with Unicode going forward.  That requirement, and its relationship to tables maintained by IANA, has caused significant confusion in the past.  This document makes adjustments to the review procedure based on experience and updates IDNA, specifically RFC 5892, to reflect those changes and to clarify the various relationships involved.  It also makes other minor adjustments to align that document with experience.', changed pages to 13, changed standardization level to Proposed Standard, changed state to RFC, added RFC published event at 2020-04-15, changed IESG state to RFC Published, created updates relation between draft-klensin-idna-unicode-review and RFC 5892)

[Ballot comment]
Hi Barry, Patrik, John, all,

I don't think it is appropriate to include text in an RFC that either insults people or stereotypes them on the basis of their field of expertise. I think the signal that this sends to potential future contributors to the IETF process is that the thanks that they may get for volunteering their time and donating their expertise is to be insulted or stereotyped or both in an immutable, archival document that we publish. If this is the consensus of the IETF, as reflected in Appendix B of this document, I want nothing to do with it. I will not be engaging further on this document.

Cheers,
Alissa

[Ballot comment]
** Section 3.  Per “Unless the IESG or the Designated Expert conclude that there are special problems or unusual circumstances, these reviews will be performed only for full Unicode versions (those numbered NN.0, e.g., 12.0) and not for minor updates (e.g., 12.1).”, to confirm, the review model being suggested is that:

-- If there is a major Unicode release, the formal review gets triggered
-- The IESG/DE is also reviewing all minor versions looking for “special problems and circumstances”

How is a “clean review” (i.e., no special problems found) of a minor version signaled?  This would be an indicator to the community, that published registry is “up-to-date as far vNN.xx”.

** Editorial Nit:
Section 3.1.  The sentence uses maintain twice.

OLD
The 2018 review (for Unicode 11.0 and versions in between it and 6.0) [IDNA-Unicode12] also concluded that maintain Unicode compatibility, rather than IDNA backward compatibility, should be maintained.

PROPOSED
The 2018 review (for Unicode 11.0 and versions in between it and 6.0) [IDNA-Unicode12] also concluded that Unicode compatibility, rather than IDNA backward compatibility, should be maintained.

[Ballot comment]
Abstract

  The standards for Internationalized Domain Names in Applications
  (IDNA) require a review of each new version of Unicode to determine
  whether incompatibilities with prior version or other issues exist

nit: I think "prior versions", "a prior version", or "the prior version"
is better.

  and, where appropriate, to allow the IETF to decide on the trade-offs
  between compatibility with prior IDNA versions and compatibility with
  Unicode going forward.  That requirement, and its relationship to

nit: I'd consider "necessary" instead of "appropriate", though both are
admittedly somewhat subjective in evaluation.

Section 1

[same comments as for abstract]

Section 3.1

  preserving backward compatibility within IDNA.  The 2018 review (for
  Unicode 11.0 and versions in between it and 6.0) [IDNA-Unicode12]
  also concluded that maintain Unicode compatibility, rather than IDNA
  backward compatibility, should be maintained.  That decision was

nit: duplicated "maintain[ed]"?

Section 3.2

  The second type of review is not clearly explained in RFC 5892, but
  is intended to identify cases in which newly-added code points, or
  perhaps even newly-discovered problematic older ones, violate design
  assumptions of IDNA, identify defects in those assumptions, or are
  inconsistent (from an IDNA perspective) with Unicode commitments
  about assignment, properties, and stability of newly-added code
  points.  [...]

Absent references to additional discussion, the "perhaps even
newly-discovered problematic older ones" clause does not seem terribly
well supported by what I have read in RFC 5892.  (I do not have any good
supporting arguments for taking a different position, though.)

  Because resolution of problems identified by this part of the review
  may take some time even if that resolution is to add additional
  contextual rules or disallow one or more code points, there will be
  cases in which it will be appropriate to publish the results of the
  algorithmic review and provide IANA with corresponding tables, with
  warnings about code points whose status is uncertain until there are
  IETF consensus conclusions about how to proceed.  The affected code

This seems to presume that having IANA continue to publish tables is
required or desirable.  I see that Section 5 notes situations in which
those presumptions should be challenged, which presumably means that the
authors do not feel that the present is such a situation?

Section 4

  At the time the IDNA2008 documents were written, the assumption was
  that, if new versions of Unicode introduced incompatible changes, the
  Standard would be updated to preserve backward compatibility for
  users of IDNA.  For most purposes, this would be done by adding to

nit: in Section 2 we use the lowercase "the standard".  Additionally,
nowhere do we call out that in this document "the standard" (whether
minuscule or majuscule) refers to IDNA2008.

not-nit: it is perhaps a little bit of a stretch to go from the language
of RFC 5892 to "the assumption was", though I have no better
alternative.

  Subsequent to the publication of this document, changes in Unicode
  detected by algorithmic reviews that would break compatibility with
  derived properties associated with prior versions of Unicode or that
  preserve such compatibility within IDNA at the price of departing
  from current Unicode specifications must be documented (in documents
  expected to be published as standards track RFCs), explained to, and
  reviewed by the IETF.

This is a very long sentence, and it seems that in the current
formulation, "that would break compatibility with derived properties"
binds to "algorithmic reviews" and not "changes in Unicode".
Additionally, as was perhaps touched on in the secdir review, the
presence of "explained to" is unusual, in that documents reviewed by the
IETF are generally considered to be products of the IETF and not for the
purpose of explaining things to the IETF.  So, I suppose that I am
asking for the "explanation on request".  (I do agree that "documented"
and 'explained" are different/non-redundant, but am not certain that
both need to be called out in this document.)

                        Doing things that way is not only at variance
  with the language in RFC 5892 but is also inconsistent with the
  intent of commitments to the registry and user communities to ensure
  that IDN labels, once valid under IDNA2008, would remain valid and,
  excepting those that were invalid because they contained unassigned
  code points, those that were invalid remained invalid.

nit(?): I'm not entirely sure how much value there is in attempting to
ascribe "intent" to events in the past and getting farther so.

  This document restores and clarifies that original language and
  intent: absent extremely strong evidence on a per-code point basis
  that preserving the validity status of possible existing (or
  prohibited) labels would cause significant harm, Unicode changes that
  would affect IDNA derived properties are to be reflected in IDNA
  exceptions that preserves the status of those labels.  There is one

This seems to be attempting to impose a requirement without naming a
responsible party or timeline for completion.  As such, it seems
unlikely to be an unqualified success...
Also, nit: s/preserves/preserve/ to keep singular/plural parity

  partial exception to this principle.  If the new code point analysis
  (see Section 3.2) concludes that some code points or collections of
  code points should be further analyzed, those code points, and labels
  including them, should be considered unsafe and used only with
  extreme caution because the conclusions of the analysis may change
  their derived property values and status.

It would perhaps be prudent to reiterate that the "new code point
analysis" is permitted(?) to expose "newly-discovered problematic older
[codepoints]".

Section 8

I agree with others that just describing as "Designated Expert(s)" and
leaving the normal expert-appointing procedures in place for both
classes of expert is the best option.  (I do note that any changes here
should also be reflected in Appendix A.)

  As discussed in Section 5, and if they have not already done so, IANA
  is requested to modify the IDNA tables collection [IANA-IDNA-Tables]

"if they have not already done so" seems unlikely to age well.

  The "Note" in that registry should also be revised to be consistent
  with the above, perhaps to say:

I don't think we need to soften this with "perhaps"; we are in a
position where we can assert the contents of the Note.

      It should be stressed that these are not normative that, in
      principle, an application can do its own calculations and these

nit: there seems to be a missing word (maybe for "in that" or "and
that"?)

  As long as the intent is preserved, the specific text is at IANA's
  discretion.

[my sense is that IANA is happy to have fewer qualitative judgments to
make on its own]

[Ballot discuss]
The document says:

"proper review will require a team of experts that has both
  broad and specific skills in reviewing Unicode characters and their
  properties in relation to both the written standards and operational
  needs.  The IESG will need to appoint experts who can draw on the
  broader community to obtain the necessary skills for particular
  situations."

And then later in Section 8 it calls on the IESG to setup a designated expert team. I have a couple of questions and concerns about this:

1) I don't fully understand the expected interaction between the team and the broader community. If the team is expected to draw on the broader community, why must it be a team? That is, why can't one DE be on the hook for completing the review, as long as the instructions to the DE say that the person must consult with experts to ensure that the breadth of expertise necessary is filtered into the review?

2) My understanding is that the number of people available to be on the envisioned DE team can be counted on one hand and that number is dwindling. Do we really expect to be able to build a team with "broad and specific skills" given this reality? I'd rather we not setup a process that we are unlikely to be able to fulfill at the time when a review is needed.

3) What is the decisionmaking process for the DE team? What happens if the DEs on the team can't agree with each other? Do we just not do the review?

[Ballot comment]
Section 3.2:

Please break these sentences and use fewer clauses and parentheticals to make them more readable:

The second type of review is not clearly explained in RFC 5892, but
  is intended to identify cases in which newly-added code points, or
  perhaps even newly-discovered problematic older ones, violate design
  assumptions of IDNA, identify defects in those assumptions, or are
  inconsistent (from an IDNA perspective) with Unicode commitments
  about assignment, properties, and stability of newly-added code
  points.
 
Actions to deal with it
  are likely to be disruptive (although perhaps not to large
  communities of users) or to leave either security risks
  (opportunities for attacks and inadvertent confusion as expected
  matches do not occur) or excessive reliance on registries
  understanding and taking responsibility for what they are registering
  [RFC5894] [RegRestr].

Abstract

    The standards for Internationalized Domain Names in Applications
    (IDNA) require a review of each new version of Unicode to determine
    whether incompatibilities with prior version or other issues exist
    and, where appropriate, to allow the IETF to decide on the trade-offs
    between compatibility with prior IDNA versions and compatibility with
    Unicode going forward.  That requirement, and its relationship to
    tables maintained by IANA, has caused significant confusion in the
    past.  This document makes adjustments to the review procedure based
    on experience and updates IDNA, specifically RFC 5892, to reflect
    those changes and clarify the various relationships involved.  It
    also makes other minor adjustments to align that document with
    experience.

The main changes are described in Appendix A of the document:

    At a very high level, this document clarifies that
    two types of review were intended and separates them for clarity and
    restores the original (but so far unobserved) default for actions
    when code point derived properties change.  For this reason, this
    document effectively provides a replacement for Section 5.1 of RFC
    5892 and adds or changes some material needed to have the replacement
    be clear or make better sense.

    Changes or clarifications that may be considered important include:

    o  Separated the new Unicode version review into two explicit parts
      and provided for different review methods and, potentially,
      asynchronous outcomes.

    o  Specified a review team, not a single expert, for the code point
      review.

    o  Eliminated the de facto requirement for the (formerly single)
      Designated Expert to be the same person as the IAB's Liaison to
      the Unicode Consortium but called out the importance of
      coordination.

    o  Created an explicit provision for an "under review" entry in the
      IANA tables so that, if there is ever again a need to tell the
      community to wait until the IETF sorts things out, that will be
      about selected potentially problematic code points and not whole
      Unicode versions.

    o  In part because Unicode is now on a regular one-year cycle rather
      than producing major and minor versions as needed, to avoid
      overloading the IETF's i18n resources, and to avoid generating and
      storing IANA tables for trivial changes (e.g., the single new code
      point in Unicode 12.1), the review procedure is applied only to
      major versions of Unicode unless exceptional circumstances arise
      and are identified.

While this document is primarily an update to review procedures, it is
an update to an existing standards track document, and therefore this
one is properly put forward as Proposed Standard.

2. Review and Consensus

This document has not been formally reviewed on any IETF list, including
on the i18ndir list, though a few key directorate participants have read
and commented on the document.  It was mentioned on the iana-update list,
and some review comments came from there. That said, this is a document
regarding recommendations to the registry and registrar community that
could only be developed by experts on IDNA, of which the IETF has very
few. A 4-week IETF-wide Last Call (as required for individual
submissions) is more than enough time for the i18ndir list to remind
experts to take a final look and confirm that there is community
consensus, insofar as that ever exists for these kinds of documents.

The shepherd has done a review of the document and thinks it is
technically sound. The most controversial bit of this document is
actually section 4:

    This document restores and clarifies that original language and
    intent: absent extremely strong evidence on a per-code point basis
    that preserving the validity status of possible existing (or
    prohibited) labels would cause significant harm, Unicode changes that
    would affect IDNA derived properties are to be reflected in IDNA
    exceptions that preserves the status of those labels.

The shepherd is agnostic as to whether this is the correct move, but it
is the only pragmatic choice given the state of the world, and there was
no disagreement with that choice in any of the reviews.

3. Intellectual Property

Each author has stated that their direct, personal knowledge of any IPR
related to this document has already been disclosed, in conformance with
BCPs 78 and 79. There has been no other discussion of IPR regarding this
document, and the shepherd has no reason to believe that any applicable
IP exists.

4. Other Points

From the idnits check:

- The text describing the updates to 5892 in the Abstract seems
sufficient, so no change needed.

- The reference warnings are false positives. No downrefs
needed.

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has completed its review of draft-klensin-idna-unicode-review-02. If any part of this review is inaccurate, please let us know.

The IANA Functions Operator understands that, upon approval of this document, there is a single action which we must complete.

IANA will work with the authors and the Designated Expert required in the IANA Considerations section to ensure that the IANA Actions, specified in that section, are correctly executed.

The IANA Functions Operator understands that this is the only action required to be completed upon approval of this document.

Note:  The actions requested in this document will not be completed until the document has been approved for publication as an RFC. This message is meant only to confirm the list of actions that will be performed.

Thank you,

Sabrina Tanamal
Senior IANA Services Specialist

The following Last Call announcement was sent out (ends 2019-08-30):

From: The IESG
To: IETF-Announce
CC: draft-klensin-idna-unicode-review@ietf.org, barryleiba@gmail.com, resnick@episteme.net
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (IDNA Review for New Unicode Versions) to Proposed Standard


The IESG has received a request from an individual submitter to consider the
following document: - 'IDNA Review for New Unicode Versions'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2019-08-30. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the beginning of
the Subject line to allow automated sorting.

Abstract


  The standards for Internationalized Domain Names in Applications
  (IDNA) require a review of each new version of Unicode to determine
  whether incompatibilities with prior version or other issues exist
  and, where appropriate, to allow the IETF to decide on the trade-offs
  between compatibility with prior IDNA versions and compatibility with
  Unicode going forward.  That requirement, and its relationship to
  tables maintained by IANA, has caused significant confusion in the
  past.  This document makes adjustments to the review procedure based
  on experience and updates IDNA, specifically RFC 5892, to reflect
  those changes and clarify the various relationships involved.  It
  also makes other minor adjustments to align that document with
  experience.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-klensin-idna-unicode-review/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-klensin-idna-unicode-review/ballot/


No IPR declarations have been submitted directly on this I-D.

Abstract

    The standards for Internationalized Domain Names in Applications
    (IDNA) require a review of each new version of Unicode to determine
    whether incompatibilities with prior version or other issues exist
    and, where appropriate, to allow the IETF to decide on the trade-offs
    between compatibility with prior IDNA versions and compatibility with
    Unicode going forward.  That requirement, and its relationship to
    tables maintained by IANA, has caused significant confusion in the
    past.  This document makes adjustments to the review procedure based
    on experience and updates IDNA, specifically RFC 5892, to reflect
    those changes and clarify the various relationships involved.  It
    also makes other minor adjustments to align that document with
    experience.

The main changes are described in Appendix A of the document:

    At a very high level, this document clarifies that
    two types of review were intended and separates them for clarity and
    restores the original (but so far unobserved) default for actions
    when code point derived properties change.  For this reason, this
    document effectively provides a replacement for Section 5.1 of RFC
    5892 and adds or changes some material needed to have the replacement
    be clear or make better sense.

    Changes or clarifications that may be considered important include:

    o  Separated the new Unicode version review into two explicit parts
      and provided for different review methods and, potentially,
      asynchronous outcomes.

    o  Specified a review team, not a single expert, for the code point
      review.

    o  Eliminated the de facto requirement for the (formerly single)
      Designated Expert to be the same person as the IAB's Liaison to
      the Unicode Consortium but called out the importance of
      coordination.

    o  Created an explicit provision for an "under review" entry in the
      IANA tables so that, if there is ever again a need to tell the
      community to wait until the IETF sorts things out, that will be
      about selected potentially problematic code points and not whole
      Unicode versions.

    o  In part because Unicode is now on a regular one-year cycle rather
      than producing major and minor versions as needed, to avoid
      overloading the IETF's i18n resources, and to avoid generating and
      storing IANA tables for trivial changes (e.g., the single new code
      point in Unicode 12.1), the review procedure is applied only to
      major versions of Unicode unless exceptional circumstances arise
      and are identified.

While this document is primarily an update to review procedures, it is
an update to an existing standards track document, and therefore this
one is properly put forward as Proposed Standard.

2. Review and Consensus

This document has not been formally reviewed on any IETF list, including
on the i18ndir list, though a few key directorate participants have read
and commented on the document. That said, this is a document regarding
recommendations to the registry and registrar community that could only
be developed by experts on IDNA, of which the IETF has very few. A
4-week IETF-wide Last Call (as required for individual submissions) is
more than enough time for the i18ndir list to remind experts to take a
final look and confirm that there is community consensus, insofar as
that ever exists for these kinds of documents.

The shepherd has done a review of the document and thinks it is
technically sound. The most controversial bit of this document is
actually section 4:

    This document restores and clarifies that original language and
    intent: absent extremely strong evidence on a per-code point basis
    that preserving the validity status of possible existing (or
    prohibited) labels would cause significant harm, Unicode changes that
    would affect IDNA derived properties are to be reflected in IDNA
    exceptions that preserves the status of those labels.

The shepherd is agnostic as to whether this is the correct move, but it
is the only pragmatic choice given the state of the world.

3. Intellectual Property

Each author has stated that their direct, personal knowledge of any IPR
related to this document has already been disclosed, in conformance with
BCPs 78 and 79. There has been no other discussion of IPR regarding this
document, and the shepherd has no reason to believe that any applicable
IP exists.

4. Other Points

From the idnits check:

- The text describing the updates to 5892 in the Abstract seems
sufficient, so no change needed.

- The reference warnings are false positives. No downrefs
needed.

Abstract

    The standards for Internationalized Domain Names in Applications
    (IDNA) require a review of each new version of Unicode to determine
    whether incompatibilities with prior version or other issues exist
    and, where appropriate, to allow the IETF to decide on the trade-offs
    between compatibility with prior IDNA versions and compatibility with
    Unicode going forward.  That requirement, and its relationship to
    tables maintained by IANA, has caused significant confusion in the
    past.  This document makes adjustments to the review procedure based
    on experience and updates IDNA, specifically RFC 5892, to reflect
    those changes and clarify the various relationships involved.  It
    also makes other minor adjustments to align that document with
    experience.

The main changes are described in Appendix A of the document:

    At a very high level, this document clarifies that
    two types of review were intended and separates them for clarity and
    restores the original (but so far unobserved) default for actions
    when code point derived properties change.  For this reason, this
    document effectively provides a replacement for Section 5.1 of RFC
    5892 and adds or changes some material needed to have the replacement
    be clear or make better sense.

    Changes or clarifications that may be considered important include:

    o  Separated the new Unicode version review into two explicit parts
      and provided for different review methods and, potentially,
      asynchronous outcomes.

    o  Specified a review team, not a single expert, for the code point
      review.

    o  Eliminated the de facto requirement for the (formerly single)
      Designated Expert to be the same person as the IAB's Liaison to
      the Unicode Consortium but called out the importance of
      coordination.

    o  Created an explicit provision for an "under review" entry in the
      IANA tables so that, if there is ever again a need to tell the
      community to wait until the IETF sorts things out, that will be
      about selected potentially problematic code points and not whole
      Unicode versions.

    o  In part because Unicode is now on a regular one-year cycle rather
      than producing major and minor versions as needed, to avoid
      overloading the IETF's i18n resources, and to avoid generating and
      storing IANA tables for trivial changes (e.g., the single new code
      point in Unicode 12.1), the review procedure is applied only to
      major versions of Unicode unless exceptional circumstances arise
      and are identified.

While this document is primarily an update to review procedures, it is
an update to an existing standards track document, and therefore this
one is properly put forward as Proposed Standard.

2. Review and Consensus

This document has not been formally reviewed on any IETF list, including
on the i18ndir list, though a few key directorate participants have read
and commented on the document. That said, this is a document regarding
recommendations to the registry and registrar community that could only
be developed by experts on IDNA, of which the IETF has very few. A
4-week IETF-wide Last Call (as required for individual submissions) is
more than enough time for the i18ndir list to remind experts to take a
final look and confirm that there is community consensus, insofar as
that ever exists for these kinds of documents.

The shepherd has done a review of the document and thinks it is
technically sound. The most controversial bit of this document is
actually section 4:

    This document restores and clarifies that original language and
    intent: absent extremely strong evidence on a per-code point basis
    that preserving the validity status of possible existing (or
    prohibited) labels would cause significant harm, Unicode changes that
    would affect IDNA derived properties are to be reflected in IDNA
    exceptions that preserves the status of those labels.

The shepherd is agnostic as to whether this is the correct move, but it
is the only pragmatic choice given the state of the world.

3. Intellectual Property

Each author has stated that their direct, personal knowledge of any IPR
related to this document has already been disclosed, in conformance with
BCPs 78 and 79. There has been no other discussion of IPR regarding this
document, and the shepherd has no reason to believe that any applicable
IP exists.

4. Other Points

The idnits check has a few things:

- The text describing the updates to 5892 in the Abstract seems
sufficient, so no change needed.

- The 2119 boilerplate is not required here, and therefore its lack is
fine. The reference to 2119 should be removed.

- The pre-5378 warning does not seem appropriate.

- There is a duplicate reference: ID.draft-klensin-idna-rfc5891bis and
RegRestr.

- The remaining reference warnings are false positives. No downrefs
needed.