IDNA2008 and Unicode 11.0.0
draft-faltstrom-unicode11-08

[Ballot discuss]
Thank you for your work on this document. I have one point that seems easy to fix, but important enough to Discuss briefly.

This topic is dense, but I'm pretty sure that it's possible to make it a bit more accessible for people who don't know the topic and its history in detail. My suggestion would be moving a bunch of 3.1 above almost everything after the first paragraph in Section 1, because people who don't know what (for example) PVALID is, will likely either bounce off the document in Section 1 and go home, or guess and continue, and they may guess wrong. If you start with the critical references, you can act disappointed if people don't chase them.

[Ballot discuss]
Thank you for your work on this document. I have a couple of things that seem easy to fix, but important enough to Discuss briefly.

This topic is dense, but I'm pretty sure that it's possible to make it a bit more accessible for people who don't know the topic and its history in detail. My suggestion would be moving a bunch of 3.1 above almost everything after the first paragraph in Section 1, because people who don't know what (for example) PVALID is, will likely either bounce off the document in Section 1 and go home, or guess and continue, and they may guess wrong. If you start with the critical references, you can act disappointed if people don't chase them.

[Ballot comment]
For the editor:

Because PVALID, CONTEXTJ, and CONTEXTO seem critical to a high-level understanding of this document, and especially section 4, perhaps it's worth adding either specific references on first use or Terminology section entries for them. If you do, UNASSIGNED, DISALLOWED, and TOTAL would be easy enough to include.

Some of the references are pretty clear ("Stringprep [RFC3454]"), some are less informative for folks who haven't read many RFCs ("RFC 6912 [RFC6912]"), and some don't give much of a clue ("SAC-084 [SAC-084]" is pretty opaque unless you know something about SSAC). You might think about the description of these references.

The string DISSALOWED and the string DISALLOWED are used interchangeably in the document - I'm assuming that DISSALOWED is a typo.

For Alexey:

You asked the balloting ADs to include their thoughts about timing of various strategies for this document and a unicode12 version. If producing unicode12 is trivial and approval might only be delayed by 6 weeks, I'd guess that approving a unicode12 might seem less odd than producing two documents with trivial differences less than two months apart, and I bet you don't want to wait months or years to process a unicode12 version now that the Unicode Consortium has released Unicode 12, but I know you'll Do The Right Thing.

[Ballot discuss]
Based on recent discussions on I18NDir mailing list, I suggest that this document is being replaced (in place, to keep history) by . This would require another IETF LC.

[Ballot comment]
I just have some editorial comments/suggestions; I'll try to trim the
ones already noted by other ADs (especially the "editor proposes" line
that Adam noted, which I agree with).

Abstract

                                      Although IDNA2008 allows adding
  exceptions to the algorithm for backward compatibility; however, this
  document does not add any such exceptions.  This document provides
  the necessary tables to IANA to make its database consisstent with

nit: "consistent"

Section 1

  o  Assigning code points can create problems if the newly-assigned
      code points are compositions of existing code points and because
      of that the normalization relationships associated with those code
      points should have been changed.

nit: I'm having a hard time parsing this, that a new code point can
literally be the composition of existing code points; does not the mere
existence of different codepoints make them distinct?  (On the other
hand, it's not clear to me that it's correct to just say "equivalent
to", since that does not specify what metric to use for the comparison.)

Section 3.1

  o  The Unicode Code Points and Internationalized Domain Names for
      Applications (IDNA) [RFC5892], informally called "Tables", lists
      the categories and rules that identify the code points allowed in
      a label written in native character form (called a "U-label"), and
      is based on Unicode 5.2.0 [Unicode-5.2.0] code point assignments
      and additional rules unique to IDNA2008.  The Unicode-based rules
      in RFC 4892 are expected to be stable across Unicode updates and
      hence independent of Unicode versions.  RFC 5892 [RFC5892]

s/4892/5892/

Section 3.2

  There are other documents important for the understanding and
  functioning of IDNA2008, for example this.

I can't tell if "this" is supposed to be [draft-faltstrom-unicode11] or
"the following"; if the latter, I'd suggest just "for example:".

Section 3.3

  In practice, the Unicode Consortium creates a maximum set of code
  points by assigning code points in the Unicode Standard.  The

"maximum set" in what scope/for what use case?

  IDNA2008 rules use the Unicode Standard to create a further subset of
  code points and context that are permitted in DNS labels associated
  with its PVALID, CONTEXTJ, and CONTEXTO derived property values.  [...]

nit: is this supposed to be "further subset of code points that are
permitted in DNS labels as well as further context-specific
restrictions"?

Appendices

It might be good to state in the preface the motivation for including
the changes from UNASSIGNED to PVALID/DISALLOWED at intermediate
versions (I assume to have a fixed record of the derived property
without reproducing the full table at each intermediate version),
especially when the actual symbol descriptions are not always visible,
since there are a lot of ranges and the right half of the range gets
truncated.

[Ballot comment]
Thanks to everyone for the impressive amount of work that has gone into
creating this document. I have only a small number of editorial suggestions to
make.


---------------------------------------------------------------------------

I-D Nits reports:

  == The document doesn't use any RFC 2119 keywords, yet has text resembling
    RFC 2119 boilerplate text.

  == Unused Reference: 'I-D.freytag-troublesome-characters' is defined on
    line 527, but no explicit reference was found in the text

  == Unused Reference: 'N4330' is defined on line 546, but no explicit
    reference was found in the text


---------------------------------------------------------------------------

Abstract:

>  Although IDNA2008 allows adding
>  exceptions to the algorithm for backward compatibility; however, this
>  document does not add any such exceptions.

Grammar nit: either remove "Although" or remove the semicolon and the word
"however."

---------------------------------------------------------------------------

§1:

>  There was three incompatible changes in the Unicode standard after

Nit: "There were..."

>  derived property value from PVALID to DISSALOWED.  They where
>  examined in great detail and IETF concluded that the consensus is

Nit: "They were..."

>  there are changes in the derived property value is a result of the

Nit: "...as a result..."

---------------------------------------------------------------------------

§5:

>  As including an exception would require implementation
>  changes in deployed implementations of IDNA20008, the editor proposes
>  that such a BackwardCompatible rule NOT to be added to IDNA2008.

This seems like the kind of thing we'd see in a draft as opposed to a published
RFC. Perhaps rephrase as "...the IETF has concluded that such a
BackwardCompatible rule will NOT be added to IDNS2008."

[Ballot comment]
Thanks to all of those who have been working on this document. I added a management item to approve the tables on March 14, so the ballot text below from the week of the March 7 telechat is no longer relevant. Keeping it here for reference:

Nearly a year ago, the IAB issued a statement encouraging the IETF community to bring the IDNA tables into alignment with the then-current version of Unicode . Given how much time has passed, we are not quite able to achieve this goal. Unicode 12 is expected to be published on March 5 and this document brings the tables into alignment with Unicode 11. Nevertheless, to avoid further delays, I'd like to suggest that if the IESG is unable to approve this document on the March 7 telechat because of issues arising during IESG evaluation that would not affect the contents of the tables themselves, that we immediately add a management item to the telechat agenda that day to approve a direct request for IANA to update the IDNA Parameters registry of derived property values to align with what is described in this document, after the expert reviewer validates that the derived property values are calculated correctly.

I added the document to the last IESG telechat before Prague in expectations that I18N Directorate will come up with recommendations about disposition of this document.

Based on feedback from I18N Directorate and the document editor, this document is not quite ready for IESG review. So I am removing it from this week's telechat.

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has completed its review of draft-faltstrom-unicode11-07. If any part of this review is inaccurate, please let us know. Please see the question at the end of this message.

The IANA Functions Operator understands that upon approval of this document, there is a single action we must complete.

The IDNA Derived Properties registry located at

https://www.iana.org/assignments/idna-tables

is to be updated once the expert reviewer for the registry determines that the derived property values are calculated correctly.

IANA Question --> Should this document be listed as an additional reference for the most recent version of this registry and for future versions of this registry? Should it not be listed as a reference at all?

Thank you,

Amanda Baber
IANA Services Specialist

[Ballot comment]
Thanks to all of those who have been working on this document.

Nearly a year ago, the IAB issued a statement encouraging the IETF community to bring the IDNA tables into alignment with the then-current version of Unicode . Given how much time has passed, we are not quite able to achieve this goal. Unicode 12 is expected to be published on March 5 and this document brings the tables into alignment with Unicode 11. Nevertheless, to avoid further delays, I'd like to suggest that if the IESG is unable to approve this document on the March 7 telechat because of issues arising during IESG evaluation that would not affect the contents of the tables themselves, that we immediately add a management item to the telechat agenda that day to approve a direct request for IANA to update the IDNA Parameters registry of derived property values to align with what is described in this document, after the expert reviewer validates that the derived property values are calculated correctly.

The following Last Call announcement was sent out (ends 2019-03-05):

From: The IESG
To: IETF-Announce
CC: alexey.melnikov@isode.com, housley@vigilsec.com, draft-faltstrom-unicode11@ietf.org, Russ Housley
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (IDNA2008 and Unicode 11.0.0) to Proposed Standard


The IESG has received a request from an individual submitter to consider the
following document: - 'IDNA2008 and Unicode 11.0.0'
  as Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2019-03-05. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the beginning of
the Subject line to allow automated sorting.

Abstract


  This document describes the changes between Unicode 6.3.0 and Unicode
  11.0.0 in the context of IDNA2008.  It further suggests a path
  forward for the IETF to ensure IDNA2008 follows the evolution of the
  Unicode Standard.

  Some changes have been made in the Unicode Standard related to the
  algorithm IDNA2008 specifies.  IDNA2008 allows adding exceptions to
  the algorithm for backward compatibility; however, this document
  makes no such changes.  Thus this document requests that IANA update
  the tables to Unicode 11.

  The document also recomments that all DNS registries continue the
  practice of calculating a repertoire using conservatism and inclusion
  principles.

  TO BE REMOVED AT TIME OF PUBLICATION AS AN RFC:

  This document is discussed on the i18nrp@ietf.org mailing list of the
  IETF.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-faltstrom-unicode11/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-faltstrom-unicode11/ballot/


No IPR declarations have been submitted directly on this I-D.


The document contains these normative downward references.
See RFC 3967 for additional information:
    rfc6912: Principles for Unicode Code Point Inclusion in Labels in the DNS (Informational - IAB stream)
    draft-freytag-troublesome-characters: Those Troublesome Characters: A Registry of Unicode Code Points Needing Special Consideration When Used in Network Identifiers (None - )
    rfc5894: Internationalized Domain Names for Applications (IDNA): Background, Explanation, and Rationale (Informational - IETF stream)
    rfc5895: Mapping Characters for Internationalized Domain Names in Applications (IDNA) 2008 (Informational - Independent Submission Editor stream)

The following Last Call announcement was sent out (ends 2018-12-31):

From: The IESG
To: IETF-Announce
CC: alexey.melnikov@isode.com, housley@vigilsec.com, draft-faltstrom-unicode11@ietf.org, Russ Housley
Reply-To: ietf@ietf.org
Sender:
Subject: Last Call:  (IDNA2008 and Unicode 11.0.0) to Informational RFC


The IESG has received a request from an individual submitter to consider the
following document: - 'IDNA2008 and Unicode 11.0.0'
  as Informational RFC

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2018-12-31. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the beginning of
the Subject line to allow automated sorting.

Abstract


  This document describes changes between Unicode 6.3.0 and Unicode
  11.0.0 in the context of IDNA2008.  It further suggests for the IETF
  a path forward regarding ensuring IDNA2008 follows the evolution of
  the Unicode Standard.

  In a few cases changes have been made in the Unicode Standard related
  to the algorithm IDNA2008 specifies.  IDNA2008 do give the ability to
  add exceptions for backward compatibility to the algorithm but the
  conclusions provided in this document suggests no such changes.

  Thus this document requests that IANA update the tables to Unicode
  11.

  In addition, all registries should continue the practice of
  calculating a repertoire using conservatism and inclusion principles.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-faltstrom-unicode11/

IESG discussion can be tracked via
https://datatracker.ietf.org/doc/draft-faltstrom-unicode11/ballot/


No IPR declarations have been submitted directly on this I-D.

Shepherd Write-up for draft-faltstrom-unicode11-05


(1) What type of RFC is being requested (BCP, Proposed Standard,
Internet Standard, Informational, Experimental, or Historic)?
Why is this the proper type of RFC?  Is this type of RFC
indicated in the title page header?

  Informational.  Yes, the header show that status.


(2) The IESG approval announcement includes a Document Announcement
Write-Up.  Please provide such a Document Announcement Write-Up.  Recent
examples can be found in the "Action" announcements for approved
documents.  The approval announcement contains the following sections:

  Technical Summary:

  This document describes changes between Unicode 6.3.0 and
  Unicode 11.0.0 in the context of IDNA2008, and it requests
  that IANA update the tables to Unicode 11.
 
  This document also suggests a path forward for the IETF to
  ensure IDNA2008 follows the evolution of the Unicode Standard.

  Working Group Summary:

    This document is not the product of any IETF WG.

  Document Quality:

  The Internet Architecture Board issues a statement that
  requested IETF to resolve the issues related to the code
  point ARABIC LETTER BEH WITH HAMZA ABOVE (U+08A1) that was
  introduced in Unicode 7.0.0.  This document resolves this
  issue and suggests IDNA2008 standard is to follow the
  Unicode Standard and not update RFC 5892 or any other
  IDNA2008-related RFC.

  Personnel:

    Russ Housley is the document shepherd.
    Alexey Melnikov is the responsible area director.


(3) Briefly describe the review of this document that was performed by
the Document Shepherd.  If this version of the document is not ready for
publication, please explain why the document is being forwarded to the
IESG.

  The document shepherd did a thorough review of the document.
  Several people with IDNA2008 "clue" have assisted with the
  document; they are listed in the Acknowledgements section.
  Of course, IETF Last Call is needed.
 

(4) Does the document Shepherd have any concerns about the depth or
breadth of the reviews that have been performed?

  No concerns.


(5) Do portions of the document need review from a particular or from
broader perspective, e.g., security, operational complexity, AAA, DNS,
DHCP, XML, or internationalization?  If so, describe the review that took
place.

  Several people with IDNA2008 "clue" have assisted with the
  document; they are listed in the Acknowledgements section.


(6) Describe any specific concerns or issues that the Document Shepherd
has with this document that the Responsible Area Director and/or the IESG
should be aware of?  For example, perhaps he or she is uncomfortable with
certain parts of the document, or has concerns whether there really is a
need for it.  In any event, if the interested community has discussed
those issues and has indicated that it still wishes to advance the
document, detail those concerns here.

  No concerns.


(7) Has each author confirmed that any and all appropriate IPR
disclosures required for full conformance with the provisions of BCP 78
and BCP 79 have already been filed.  If not, explain why.

  The author has explicitly stated that he is unaware of any IPR
  related to this document.


(8) Has an IPR disclosure been filed that references this document?  If
so, summarize any discussion and conclusion regarding the IPR
disclosures.

  No IPR disclosures have been submitted for this Internet-Draft.


(9) How solid is the consensus of the interested community behind this
document?  Does it represent the strong concurrence of a few individuals,
with others being silent, or does the interested community as a whole
understand and agree with it?

  The Internet Architecture Board issues a statement that
  requested IETF to resolve the issues related to the code
  point ARABIC LETTER BEH WITH HAMZA ABOVE (U+08A1) that was
  introduced in Unicode 7.0.0.  This document resolves this
  issue and suggests IDNA2008 standard is to follow the
  Unicode Standard and not update RFC 5892 or any other
  IDNA2008-related RFC.


(10) Has anyone threatened an appeal or otherwise indicated extreme
discontent?  If so, please summarise the areas of conflict in separate
email messages to the Responsible Area Director.  (It should be in a
separate email because this questionnaire is publicly available.)

  No one has threatened an appeal.


(11) Identify any ID nits the Document Shepherd has found in this
document.  (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts
Checklist).  Boilerplate checks are not enough; this check needs to be
thorough.

  IDnits raises two minor things:
 
  ** Obsolete normative reference: RFC 3491 (Obsoleted by RFC 5891)

  -- Obsolete informational reference (is this intentional?): RFC 3490
    (Obsoleted by RFC 5890, RFC 5891)

  These are not mistakes; old versions are referenced to provide
  context.


(12) Describe how the document meets any required formal review criteria,
such as the MIB Doctor, media type, and URI type reviews.

  None needed.


(13) Have all references within this document been identified as either
normative or informative?

  Yes, the references are divided into normative and informative.


(14) Are there normative references to documents that are not ready for
advancement or are otherwise in an unclear state?  If such normative
references exist, what is the plan for their completion?

  None.


(15) Are there downward normative references references (see RFC 3967)?
If so, list these downward references to support the Area Director in the
Last Call procedure.

  There are no downward references.


(16) Will publication of this document change the status of any existing
RFCs?  Are those RFCs listed on the title page header, listed in the
abstract, and discussed in the introduction?  If the RFCs are not listed
in the Abstract and Introduction, explain why, and point to the part of
the document where the relationship of this document to the other RFCs is
discussed.  If this information is not in the document, explain why the
interested community considers it unnecessary.

  No status changes are requested by this document.


(17) Describe the Document Shepherd's review of the IANA considerations
section, especially with regard to its consistency with the body of the
document.  Confirm that all protocol extensions that the document makes
are associated with the appropriate reservations in IANA registries.
Confirm that any referenced IANA registries have been clearly identified.
Confirm that newly created IANA registries include a detailed
specification of the initial contents for the registry, that allocations
procedures for future registrations are defined, and a reasonable name
for the new registry has been suggested (see RFC 5226).

  IANA is requested to update the registry of derived property values
  after validation with the Appointed Expert that the derived property
  values are calculated correctly.  See the appendices in the document.


(18) List any new IANA registries that require Expert Review for future
allocations.  Provide any public guidance that the IESG would find useful
in selecting the IANA Experts for these new registries.

  The author of the document is the IANA Expert of the registry
  in question.


(19) Describe reviews and automated checks performed by to validate
sections of the document written in a formal language, such as XML code,
BNF rules, MIB definitions, etc.

  The document does not make use of a formal language such as
  ABNF, XML, or ASN.1.