Internet Draft                                         Paul Hoffman
draft-ietf-idn-compare-00.txt                            IMC & VPNC
June 14, 2000
Expires in six months



Comparison of Internationalized Domain Name Proposals

Status of this memo

This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC 2026.

Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF), its areas, and its working groups. Note that other groups
may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference material
or to cite them other than as "work in progress."

     The list of current Internet-Drafts can be accessed at
     http://www.ietf.org/ietf/1id-abstracts.txt

     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html.


Abstract

The IDN Working Group is working on proposals for internationalized
domain names that might become a standard in the IETF. Before a single
full proposal can be made, competing proposals must be compared on a
wide range of requirements and desired features. This document compares
the many parts of a comprehensive protocol that have been proposed. It
is the companion document to "Requirements of Internationalized Domain
Names" [IDN-REQ], which lays out the requirements for the
internationalized domain name protocol.


. Introduction

As the IDN Working Group has discussed the requirements for IDN,
suggestions have been made for various candidate protocols that might
meet the requirements. These proposals have been somewhat helpful in
bringing up real-world needs for the requirements.

It became clear no single proposal had wide agreement from the working
group. In fact, the authors of various proposals found themselves taking
some features from other proposals as they revised their drafts. At the
same time, working group participants were making suggestions for
incremental changes that might affect more than one proposal.

Because of this mixing and matching, it was decided that this IDN
comparisons document should compare features that might end up in the
final protocol, not full protocol suggestions themselves. The features
that had been discussed in the working group were divided by function,
and appear in this document in separate sections. For each function,
there are multiple suggestions for protocol elements that might meet the
requirements that are described in [IDN-REQ].

This document is being discussed on the "idn" mailing list. To join the
list, send a message to <majordomo@ops.ietf.org> with the words
"subscribe idn" in the body of the message. Archives of the mailing list
can also be found at ftp://ops.ietf.org/pub/lists/idn*.

1.1 Format of this document

Each section covers one feature that has been discussed as being part of
the final IDN solution. Within each section, alternate proposals are
listed with the major perceived pros and cons of the proposal. Also,
each proposal is given a label to make discussion of this document (and
of the proposals themselves) easier.

References to the numbered requirements in [IDN-REQ] are from version
-02 of that document. These numbers are expected to change and the
requirements document evolves. In this draft, the requirements are show
as "[#n-02]", where "n" is the requirement number from draft -02 of
[IDN-REQ]. This document only lists where particular proposals don't
meet particular requirmenents from [IDN-REQ], not the ones that they
fulfill.

Note that this document is supposed to reflect the discussion of all
proposed alternatives, not just the ones that fully match the
requirements in [IDN-REQ]. It will serve as a summary of the discussion
in the IDN WG for readers in the future who may want to know why certain
alternatives were not chosen for the eventual protocol.

The proposal drafts covered in this document are:

[DUERST] Character Normalization in IETF Protocols,
draft-duerst-i18n-norm-03

[HOFFMAN] Compatible Internationalized Domain Names Using Compression,
draft-hoffman-idn-cidnuc-03

[KWAN] Using the UTF-8 Character Set in the Domain Name System,
draft-skwan-utf8-dns-03

[OSCARSSON] Internationalisation of the Domain Name Service,
draft-oscarsson-i18ndns-00

[SENG] UTF-5, a transformation format of Unicode and ISO 10646,
draft-jseng-utf5-01

1.2 Editor's note for the -00 draft

This first draft is probably incomplete in many aspects. There may be
proposals that appeared on the mailing list or in conversation that I
did not include here, and there are likely to be pro and con arguments
that I have left out. Any such omission is not an indication of the lack
of merit, but instead an error on the editor's part. Also, if there are
proposals here that do not fulfill some of the requirements from
[IDN-REQ] but that fact is not reflected here, that is an omission that
should be corrected. In any case, please actively send comments and
corrections to this document to the IDN working group.


. Architecture

One of the biggest questions raised early in the IDN discussion was what
the format of internationalized name parts would be on the wire, that
is, between the user's computer and the DNS resolvers. It was agreed
that the DNS protocols certainly allow non-ASCII octets in domain name
parts and resource records, but there was also acknowledgement that many
protocols that rely on the DNS could not handle non-ASCII names due to
the design of the protocol. Section 3.1 of this document describes the
proposed encodings for the non-ASCII name parts.

Because of requirement [#2-02], there were proposals for
ASCII-compatible encodings (ACEs) of non-ASCII characters. Different
ACEs were proposed (and are discussed in Section 4 of this document),
but they all have the same goal: to allow non-ASCII characters to be
represented in host names that conform to RFC 1034 [RFC1034].

2.1 arch-1: Just send binary

[KWAN] proposes beginning to send characters outside the range allowed
in RFC 1034.

Pro: Easiest to describe. Only changes host name syntax, not any of the
related DNS protocols.

Con: Doesn't work with many exiting protocols that relies on DNS.
Violates requirement [#9-02].

2.2 arch-2: Send binary or ACE

[OSCARSSON] proposes using both binary and ACE formats on the wire.

Pro: Allows protocols that can handle binary name parts to use them
directly, while allowing protocols that cannot use binary name parts to
also handle names without conversion. Allows domain names in free text
to be displayed in binary even in systems that require ACE-formatted
names on the wire.

Con: Requires all software that uses domain names to handle both
formats. Requires processing time for conversion of ACE formats into the
format must likely used internally to the software.

2.3 arch-3: Just send ACE

[HOFFMAN] and [SENG] propose that host naming rules remain the same and
that all internationalize domain names be sent in ACE format.

Pro: No changes at all to current DNS protocols.

Con: Requires all software to recognize ACE domain names and convert
them to human-readable for display. This is true not only in domain
names used on the wire but also domain names used in free text.


. Names in binary

Both arch-1 and arch-2 include domain name parts that are represented on
the wire in a binary format. This section describes some of the features
of such names.

3.1 bin-1: Format

There are many different charsets and encodings for the scripts of the
world. The WG has discussed which binary encoding should be used on the
wire.

3.1.1 bin-1.1: UTF-8

The IETF policy on character sets [RFC2277] states that UTF-8 [RFC2279]
is the preferred charset for IETF protocols. UTF-8 encodes all
characters in the ISO 10646 repertoire.

Pro: Well-supported in other IETF protocols. Compact for most scripts.
Wide implementation in programming languages. US-ASCII characters have
the same encoding in UTF-8 as they do in US-ASCII. Because it is based
on ISO 10646, expansion of the repertoire comes from respected
international standards bodies.

Con: Asian scripts require three octets per character.

3.1.2 bin-1.2: Labelled charsets

Mailing list discussion mentioned using multiple charsets for the binary
representation. Each name part would be labelled with the charset used.

Pro: Allows users to specify names in the charsets they are most
familiar with.

Con: All resolvers would have to know all charsets. Thus, the number of
charsets would probably have to be limited and never expand. Mapping of
characters between charsets would have to be exact and not change over
time.

3.2 bin-2: Distinguishing binary from current format

Software built for current domain names might give unexpected results
when dealing with non-ASCII characters in domain names. For example, it
was reported on the mailing list that some software crashes when a
non-ASCII domain name is returned for in-addr.arpa requests. Thus, there
may be a need for IDN to prevent software that is not binary-aware from
receiving domain names with binary parts. This would only apply to an
IDN that used arch-2, not arch-1.

3.2.1 bin-2.1: Don't mark binary

[KWAN] does not specify any way of changing requests to prevent binary
name parts from being transmitted.

Pro: No changes to current DNS requests and responses.

Con: Likely to cause disruption in software that is not binary-aware.
Likely to cause systems to misread names and possibly (and incorrectly)
convert them to ASCII names by stripping off the high bit in octets;
this in turn would lead to security problems due to mistaken identities.
Returning binary host names to DNS queries is known to break some
current software.

3.2.2 bin-2.2: Mark binary with IN bit

[OSCARSSON] describes using a bit from the header of DNS queries to mark
the query as possibly containing a binary name part and indicating that
the response to the query can contain binary name parts.

Pro: This bit is currently unused and must be set to zero, so current
software won't use it accidentally. No changes to any other part of the
query or RRs.

Con: It's the last unused bit in the header and DNS folks have indicated
that they are very hesitant to give it up.

3.2.3 bin-2.3: Mark binary with new QTYPEs

Off-list discussion has mentioned using new QTYPEs to mark the query as
possibly containing a binary name part and indicating that the response
to the query can contain binary name parts. QTYPEs are two octets long,
and no QTYPEs to date use more than the lower eight bits, so one of the
bits from the upper octet could be used to indicate binary names.

Pro: These bits are currently unused and must be set to zero, so current
software won't use them accidentally. No changes to any other part of
the query or RRs. Uses a bit that isn't as prized as the IN bit.

Con: Software must pay more attention to the QTYPEs than it might have
previously.

3.2.4 bin-2.4: Mark binary with EDNS0

Off-list discussion has mentioned using EDNS0 [RFC2671] to mark the
query as possibly containing a binary name part and indicating that the
response to the query can contain binary name parts.

Pro: There is little use of EDNS0 at this point, so it is very unlikely
to have bad interactions with old software.

Con: There is little use of EDNS0 and this might make implementation
harder.


. Names in ASCII-compatible encoding (ACE)

Both arch-2 and arch-3 include domain name parts that are represented on
the wire in an ASCII-compatible encoding (ACE). This section describes
some of the features of such names.

4.1 ace-1: Format

A variety of proposals for the format of ACE have been proposed. Each
proposal has different features, such as how many characters can be
encoded within the 63 octet limit for each name part. The length
descriptions in this section assume that there is no distinguishing of
ACE from current names; this is not a likely outcome of the WG work.

The descriptions of lengths is based on script block names from
[BLOCK-NAMES].

4.1.1 ace-1.1: UTF-5

[SENG] Describes UTF-5, which is a fairly direct encoding of ISO 10646
characters using a system similar to UTF-8. Characters from Basic Latin
and Latin-1 Supplement take 2 octets; Latin Extended-A through Tibetan
take 3 octets; Myanmar through the end of BMP take 4 octets; non-BMP
characters take 5 octets. This means that names using all characters
in the Myanmar through the end of BMP are limited to 15 characters.

Pro: Extremely simple.

Con: Poor compression, particularly for Asian scripts.

4.1.2 ace-1.2: CIDNUC

[HOFFMAN] describes CIDNUC, which is a two-step algorithm that first
compresses the name part, then converts the compressed string into and
ACE. Name parts in all scripts other than Han, Yi, Hangul syllables, and
non-BMP take up ceil(1.6*(n+1)) octets; name parts in those scripts and
any name that mixes characters from different rows in ISO 10646 take up
ceil(3.2*(n+1)) octets. This means that names using Han, Yi, or Hangul
syllables are limited to 18 characters.

Pro: Best compression for most scripts, and similar compression for the
scripts where it is not the best.

Con: More complicated than UTF-5. Not well optimized for names that have
mixed scripts, such as non-Latin names that use hyphen or ASCII digits.

4.1.3 ace-1.3: Hex of UTF-8

[OSCARSSON] describes "hex of UTF-8", which is a straight-forward
hexadecimal encoding of UTF-8. Characters in Basic Latin (other than
non-US-ASCII and hyphen) take 3 octets; Latin Extended-A through Tibetan
take 5 octets; Myanmar through end of BMP take 7 octets; non-BMP
characters take 9 octets. This means that names using all characters
in the Myanmar through the end of BMP are limited to 9 characters.

Pros: Very simple to describe.

Cons: Very poor compression for all scripts.

4.1.4 ace-1.5: SACE

A message on the mailing list pointed to code for SACE, an ASCII
encoding that purports to compact to about the same size as UTF-8.

Pros: Similar compression to UTF-8.

Cons: No description of how the algorithm works.

4.2 ace-2: Distinguishing ACE from current names

Software that finds ACE name parts in free text probably should
display the name part using the actual characters, not the ACE
equivalent. Thus, software must be able to identify which ASCII name
parts are ACE and which are non-ACE ASCII parts (such as current names).
This would only apply to an IDN proposal that used arch-2, not arch-3.

4.2.1 ace-2.1: Currently legal names

Name parts that are currently legal in RFC 1034 can be tagged to
indicate the part is encoded with ACE.

4.2.1.1 ace-2.1.1: Add hopefully-unique legal tag

[HOFFMAN] proposes adding a hopefully-unique legal tag to the beginning
of the name. The proposal would also work with such a tag at the end of
the name part, but it is easier for most people to recognize at the
beginning of name parts.

Pros: Easy for software (and humans) to recognize.

Cons: There is no way to prevent people from beginning non-ACE names
with the tag. Unless the tag is very unlikely to appear in any name in
any human language, non-ACE names that begin with the tag will display
oddly or be rejected by some systems.

4.2.1.2 ace-2.1.2: Add a checksum

Off-list discussion has mentioned the possibility of creating a checksum
mechanism where the checksum would be added to the beginning (or end) of
ACE name parts.

4.2.2 ace-2.2: Currently illegal names

Instead of creating names that are currently legal, another proposal is
to create names that use the current ASCII characters but are illegal.

4.2.2.1 ace-2.2.1: Add trailing hyphen

[OSCARSSON] describes using a trailing hyphen as a signifier of an ACE
name.

Pros: It is surmised that most current software does not reject names
that are illegal in this fashion. Thus, there would be little disruption
to current systems. This mechanism takes up fewer characters than any
proposed in ace-2.1.

Cons: Some current software is will probably break with this mechanism.
It goes against some current protocols that match the rules in RFC 1034.

. Prohibited characters

There was a short but active discussion on the mailing list about which
characters from the ISO 10646 character set should never appear in host
names. To date, there are no Internet Drafts on the subject. This
section summarizes some of the suggestions.

5.1 prohib-1: Identical and near-identical characters

Some characters are visually identical or incredibly similar to other
characters, thus making it impossible to accurately enter host names
that are seen in print.

5.2 prohib-2: Separators

Horizontal and vertical spacing characters would make it unclear where a
host name begins and ends. Also, allowing periods and period-like
characters as characters within a name part would also cause similar
confusion.

5.3 prohib-3: Non-displaying and non-spacing characters

There are many characters that cannot be seen in the ISO 10646 character
set. These include control characters, non-breaking spaces, formatting
characters, and tagging characters. These characters would certainly
cause confusion if allowed in host names.

5.4 prohib-4: Private use characters

Private use characters from ISO 10646 inherently have no specified
visual form (and in fact can be used for non-displaying characters).
Thus, there could be no visual interoperability for characters in the
private use areas.

5.5 prohib-5: Punctuation

Some punctuation characters are disallowed in URLs because they are used
in URL syntax.

5.6 prohib-6: Symbols

Some mailing list discussion stated that characters that do not normally
appear in human or company names should not be allowed in host names.
This includes symbols and non-name punctuation.


. Canonicalization

The working group has a spirited discussion on the need for
canonicalization. [IDN-REQ] describes many requirements for when and what
type of canonicalization might be performed.

6.1 canon-1: Type of canonicalization

The Unicode Consortium's recommendations and definitions of
canonicalization [UTR-15] describes many forms of canonicalization that
can be performed on character strings. [DUERST] covers much of the same
ground but makes more focused requirements for canonicalization on the
Internet.

6.1.1 canon-1.1: Normalization Form C

Both [UTR-15] and [DUERST] recommend Normalization Form C, as described
in [UTR-15]. This form is a canonical decomposition, followed by
canonical composition.

6.1.2 canon-1.2: Normalization Form KC

Discussion on the mailing list recommended Normalization Form KC, This
form is a compatibility decomposition, followed by canonical
composition. Compatibility decomposition makes characters that have
compatibility equivalence the same after decomposing.

6.2 canon-2: Other canonicalization

Host names may have special canonicalization needs that can be added to
those given in canon-1.

6.2.1 canon-2.1: Case folding in ASCII

RFC 1034 specifies that there is no difference between host names that
have the same letters but the letters have different case. Thus, the
name part "example" is considered the same as "Example" and "EXamPLe".
Neither uppercase nor lowercase is specified as being canonical.

6.2.2 canon-2.2: Case folding in non-ASCII

Discussion on the mailing list has raised the issue of whether or not
non-ASCII Latin characters should have the same case-folding rules as
ASCII. Such rules would match the expectations of native speakers of
some languages, but would go counter to the expectations of native
speakers of other languages.

6.2.3 canon-2.3: Han folding

Discussion on the mailing list has raised the issue of equivalences in
some languages use of Han characters. For example, in Chinese, there are
many traditional characters that have equivalent simplified characters.
Similarly, there are some Han ideographs for which there are multiple
representations in ISO 10646. There are no well-established rules for
such folding, and some of the proposed folding would be locale-specific.

. Transitions

Early in the working group discussion, there was active debate about how
the transition from the current host name rules to IDN would be handled.
Given requirement [#1-02], this transition is quite important to
deciding which proposals might be feasible.

7.1 trans-1: Always do current plus new architecture

In this proposal, IDN will be used at the same time as the current DNS
forever. That is, IDN will be in addition to the current DNS.

7.2 trans-2: Transition period

In this proposal, IDN will be used at the same time as the current DNS
for a specified period of time, after which only IDN will exist. That
is, IDN will replace the current DNS.


. Root server considerations

DNS root servers receive all requests for top-level domains that not in
the local DNS cache. They are critical to the Internet.  Care must be
taken to ensure that root servers will not be affected by new mechanisms
introduced.

Any IDN proposal that includes a binary encoding will have an impact on
the root servers. The binary requests will affect the root servers
because the current root server software is designed to handle current
host names. Further, the root zone files which contain ccTLDs and gTLDs
would have to support binary domain names and possibly binary host names
for NS records. Because all the root servers are equivalent, they would
have to be synchronized to support the binary domain names at the same
time.

Proposals that only use ACE and use tagging with currently-legal names
would, by definition, not affect the root servers.


. Security considerations

All security considerations listed in [IDN-REQ] apply to this document.
Further, all security considerations listed in each of the IDN proposals
must be considered when comparing the proposals.

Some proposals described in this document may create new security
considerations. However, these considerations will have to be addressed
in the eventual protocol document. All the proposals described here are
still incomplete and security considerations may be added to them as
they are revised. All the proposals listed in this document use the ISO
 character set, so the proposals inherit any security
characteristics of that character set.

Many protocols and applications rely on domain names to identify the
parties involved in a network transaction. For example, a user who
connects to a web site by entering or selecting a URL expects that their
software will select the web site named in the URL. The uniqueness of
domain names are crucial to ensure identification of Internet entities.

To make round-trip translation between local charsets and ISO 10646, the
ISO 10646 specification has assigned multiple code points to individual
glyphs. Moreover, some glyphs might look similar to some users, but look
clearly different by other users. This means that it would be simple for
an attacker to mimic a domain name by using similar-looking but
different glyphs and guessing that some users will not see the
difference in their user interface.

Some IDN protocols may have denial of service attacks, such as by using
non-identified chars, exception characters, or under-specified behavior
in using some special characters.


. IANA considerations

This document does not create any new IANA registries. However, it is
possible that a character property registry may need to be set up when
the IDN protocol is created in order to list prohibited characters
(section 5) and canonicalization mappings (section 6).


. Acknowledgements

James Seng and Marc Blanchet gave many helpful suggestions on the
pre-release versions of this document.


. References

[BLOCK-NAMES] Unicode Consortium,
<ftp://ftp.unicode.org/Public/UNIDATA/Blocks.txt>.

[DUERST] Character Normalization in IETF Protocols,
draft-duerst-i18n-norm-03

[HOFFMAN] Compatible Internationalized Domain Names Using Compression,
draft-hoffman-idn-cidnuc-03

[IDN-REQ] Requirements of Internationalized Domain Names,
draft-ietf-idn-requirements-02

[KWAN] Using the UTF-8 Character Set in the Domain Name System,
draft-skwan-utf8-dns-03

[OSCARSSON] Internationalisation of the Domain Name Service,
draft-oscarsson-i18ndns-00

[RFC2277] IETF Policy on Character Sets and Languages, RFC 2277

[RFC2279] UTF-8, a transformation format of ISO 10646, RFC 2279

[RFC2671] Extension Mechanisms for DNS (EDNS0), RFC 2671

[SENG] UTF-5, a transformation format of Unicode and ISO 10646,
draft-jseng-utf5-01

[UTR15] Unicode Normalization Forms, Unicode Technical Report #15


A. Author Contact

Paul Hoffman
IMC & VPNC
 Segre Place
Santa Cruz, CA  95060
phoffman@imc.org or paul.hoffman@vpnc.org