IETF Technical Plenary Session, 
November 12, 2009
   
1.  Introduction - Olaf Kolkman   
   
(Olaf)  It's now 4:30 and I would like to start in about a minute,
so if everybody can find themselves a seat.  So, memorize the note
well.  

Ladies and gentlemen, welcome all to the technical plenary, 
IETF 76 in Hiroshima.  During this meeting, we've got some 
supporting tools for people in jabber land, and also for the people
in this room.  There's a jabber room for the plenary, 
jabber.ietf.org, and the presentations that we will use have been 
uploaded.  You can find them on the meeting material site - so you 
can read a along if you're back home.  Welcome to those back home.  

There's a little bit of an opportunistic environment for an 
experiment, we had the opportunity to find somebody to transcribe 
in English, and you can see that on the screen on the left and 
right - what is being said in this meeting.  This might help people
who do not capture the accents of some of the speakers or some of 
the people at the mic.  And we would like to know your experience 
with that - if you value this experiment.  

When you step up to the mic and ask questions, again, it's 
important to swipe your card.  I will do so now, so you can also 
see who I am.  And when you are at the mic, please keep it short 
and keep it to the point.  It's something that we heard yesterday.  
We hope we can establish a tradition.  

A short paragraph on the agenda for today.  Aaron will start with
his report.  I will follow with the IAB's report.  And then we will 
have a session on internationalization of names and other 
identifiers, that's a session led by John Klensin, Dave Thaler and
Stuart Cheshire.  After that, somewhere between 7:00 and 7:10 we 
will have an open microphone session which will hopefully not last
longer than 7:30.  It all depends on the conciseness of questions 
and amount of questions asked.  Without further ado, Aaron...
 

2.  IRTF Chair Report - Aaron Falk 

(Aaron) Hello, I'm here to give a brief report on the Internet 
Research Task Force (IRTF).  My name is Aaron Falk.  The short 
status - we had four research groups meeting this week here at the
IETF.  The Host Identity Protocol (HIP) research group and the 
Scalable Adaptive Multicast (SAM) research groups have already met.
Tomorrow the Delay Tolerant Networking (DTN) and Routing Research 
Group (RRG) will meet.  Another thing we did was to have review of
the Routing Research Group with the IAB a this morning.

We have six IRTF RFCs waiting to be published.  Publication is 
currently wedged on finalizing the TLP, the trust license, and that
looks like (as was reported last night) it will happen around the 
end of the year or early next year.  The document defining the IRTF 
RFC stream was revised and is now sitting the RFC editor's queue, 
and for those of you keeping track, it was modified so that now the 
independent stream and the IRTF registries have common rights 
language.  

Let's see.  What else?  

In work - there's been some proposed new work.  We've been talking
about for quite sometime, I guess almost two years, an RG on 
virtual networks, or network virtualization.  There's a bar BOF 
starting in about 40 minutes, so those of who are going to that 
will miss the fascinating talk that's coming up later tonight, 
here in the plenary.  But hopefully, this group is finally getting
to discussion of a charter for the research group, so that would be
good progress.  

Another topic that's come up, in groups that I've been in a couple
of times this week, has been internet of things, smart objects, and
so there's starting to be discussion about maybe there should be a
research group that's looking at an architecture for those 
technologies and how they fit into the internet architecture.  That
is just talk at this point, but I'm just giving the community heads
up.  One thing we want to make sure is that anything that happens 
in the IRTF does not slow down the smart grid activities that are 
going on.  So make sure we work around that.
  
I like to give sort of a quick snapshot of the different research 
groups and the energy levels - who is active and who is not.  The 
groups on the right, the 'active' list, are groups that are 
meeting at IETF or elsewhere, or having very a active mailing 
lists.  The colored one at the bottom is the SAM research group,
which has moved to the active column.  They met this week, and they
have actually had a few meetings at non-IETF locations with other
conferences.  

The quiescent groups, they're not totally inactive, they have 
meeting lists going along.  The Public Key Next Generation Research 
Group, PKNGRG, they had a little trouble getting started, but it 
sounds like there's some energy.  Do you have a question?

(Richard Barnes)  I was just wondering if the PKNG group will be 
meeting at IETF 79?

(Aaron)  I don't think that's been decided why yet.  I don't think
there's a planned meeting now, but if you're on the mailing list, 
you would hear about it.  Is Paul in the room?  Can you confirm 
that's true?  

(Paul Hoffman) That's correct.  

(Aaron)  Moving right along.  So, another thing that I've been 
doing with the IRTF reports is to take a couple of research groups
and give a very quick snapshot of the topic area - what the group 
is up to, some of their recent work items, to give a flavor of 
what's going on in these groups.  This is really very cursory, and
just to give a you flavor of some research stuff that's happening 
in the IETF and maybe help discover whether there is interest in 
getting more involved.  

The first group I want to talk about is Anti-Spam Research Group.  
It's an open research group that's looking at anti-spam research.  
In particular, at the open problems.  It's been hoped that there 
would be some standards work that would come out of that, but it's
not been as fruitful as was originally hoped.  But there's a wide 
range of participation from not only the standards folks that we
see here, but also researchers and other folks who are working in 
the area.  There's lots of industrial activity going on this in 
this space.  Because anti-spam is a big industry, there are lots of
other activities going on, and so it's important to understand that
the research group is not doing standards work.  There is some in 
the IETF, in DKIM.  It's not a trade group - there are several of 
those - I think the large one is MOK.  And it's not an academic
conference.  So, this has really been sort of a discussion of 
technical topics in the area, and they've worked on a couple of 
documents, but mostly the activity has been on the mailing list.  

There's a document they produced on DNS black lists and white 
lists that's waiting to be published.  And then there's another one
on black list management that has been, it's a draft that has 
been circulated for a while and it's sort of waiting to be 
finalized.  

Another topic that's been going on in this research group has been 
starting to develop a taxonomy of the different techniques for 
fighting spam, and also, of different spamming strategies.  You can 
see the URL here if want too check it on the web.  This is really 
sort of open for contributions.  I think that part of the 
motivation for this is that many people have come up with ideas, 
often the same ideas repeatedly for how to solve the spam problem.
And so, this is a little bit, it's been described in the past as 
pre-printed rejection slips for why your idea won't work, so you 
can be indexed into the correct part of the WIKI when you have an
idea and don't have to re-circulate threads on the mailing list 
over and over again.  So I think that would be good work.  This is
turned out to be, like the spam problem in general, hard to make 
progress in.  And the research group - I've heard, and I think the
chair has heard, some frustration as to why they have not done 
more.  There are a lot of folks doing research in this area and 
they're focused on publishing papers, sometimes more so than doing
collaboration in the IRTF.  Also, some of these problems are 
extremely hard.  But, one of the values of what's happening on the
research group mailing list is it's starting to go capture some of
the folklore, some of the wisdom that's passed around between 
practitioners.  But there's also misinformation that's stamped out,
and they're making an effort to capture these things in the wiki.
So the chair asked me to pass along - the mailing list is really 
intended for folks in the IETF and elsewhere, having questions 
about spam and anti-spam related technologies, that this research
group is intended to be a good discussion point for bringing those
topics out.  

Okay.  So, the other research group that I wanted to talk about is
the Scaleable Adaptive Multi-cast group.  I apologize, it's hard 
for me to read (the slide) so it's probably hard for you to read.  
The concept behind this group, if you look at the pictures, the 
bottom one is intended to be a conventional network, host at the 
edge and routers in the middle.  And the goal of the group is 
really to enable multicast services, taking hybrids of application
layer multicast, which is easy to deploy among consenting end 
systems, and take advantage of either IP multicast or link layer
multicast - any native multicast that might exist.  

This is what see in the pictures, in the bottom you have the 
conventional network, then you have the multicast tree, and then at
the top you have a hybrid multicast environment where you have 
native multicast in one region, and application multicast in 
regions that don't support it.  It takes advantage of the AMT 
protocol - this is a protocol for tunneling.  AMT is Adaptive, 
automatic multi cast tunneling.  That connects multicast enabled 
clouds over unicast networks.  And this is technology that was 
developed in the MBONE D environment, and so, it's a way of sort of
gluing these together.  So the Sam RG is trying to create framework
and protocols for integrating these various strategies for enabling 
multicast. 

There's a bunch of different communities, this work was initial led
out of the X-cast environment, where they have some protocols, and
they've got sort of one point they developed in the space.  There's
also P2P overlays and the IP multicast folks, and then applications 
include streaming and mobile networks and other kinds of 
applications.  

They've developed, I've gotten some drafts on developing a 
framework.  This is just another illustration of another version of
the same picture where you've got networks that have neighborhood 
IP multicast.  They might have link layer multicast, application 
layer multicast, and they're glued together, and they've developed 
a protocol that's got different kinds of joins, IP multicast join, 
join-by-gateway, join-by-native-link, and so this is some of the 
work that's been going on the longest in the group.  And it's 
pretty mature, I understand.  Another thing that they've been 
working on is developing namespace support so that hosts can 
directly participate in multicast services.  And along with 
middleware to make that work.  And they've been also trying to 
build a simulation environment to allow exploration of a wide range
of networks in this space.  They started with a tool called OMnet, 
and they're extending it to support IP multicast, and then 
extending that again to support different kinds of overlay 
strategies.  

Then finally, to go beyond simulation to test beds, there's some
work that's just being discussed now about building a hybrid 
multicast test bed that's started with contributions from the 
different participants in the research group.  That is, they're 
actually globally distributed with the hopes that that will grow 
for implementing and exploring some of these protocols.  

So in a nutshell that's the status of the IRTF and two of the 
research groups, and I am open for questions if anybody has any.  
Okay.  Thank you very much.  
   
(applause)  
   

3.  IAB Report - Olaf Kolkman

As far as the IAB report, I love these little crane birds folded
out of paper, so I put one on the picture, typically for Hiroshima.
And I enjoyed making them during the social.  

Anyway about the IAB.  I show this slide every time I open the 
session - basically pointing out what we're about.  It's very hard
to give a nutshell description of what the IAB is about.  But it 
has a charter, RFC 2850, and we try to describe as much as possible
on our home page.  You can find the current membership there.  
There are links to documents, and within the documents section you 
can find our minutes.  It is the goal to have minutes posted not 
more than two meetings behind.  We're very bad at meeting that 
goal.  Just before this meeting, a batch of minutes was published 
that were approved earlier in this week.  

Correspondence... when we talk to other organizations we usually 
leave a trail of correspondence and that is published on our web 
site as well.  Documents are one of outputs.  Recently we published
as an RFC 5620 the RFC Editor model.  I will be talking about the 
model implementation at the end of this presentation.  

There are two documents currently in auth48, 5694 which is the P2P 
architecture definition, taxonomies, and examples and 
applicability.  That is about to be published.  There's a final 
little thing with a header, same goes for RFC 5704 Uncoordinated 
Protocol Development Considered Harmful.  You've heard a 
presentation about that, in previous IAB reports.  Those are two 
very short.  

There is ongoing document activity.  We've been working on a 
document considering IPv6 NATs.  There was a call for comments from
the community.  Those comments have been incorporated into version 
2 of this draft and we're about to submit this to the RFC editor 
when every IAB member has a chance to sign off on it.  So that will
be going to the RFC editor shortly.  

There's another document - IAB Thoughts on Encoding and 
Internationalized Domain Names.  It's part of the inspiration for 
today's technical session.  So basically, it's a call for comments.
The technical plenary today is a working session that is based 
around this document.  

There are a bunch of documents that have draft IAB that are sitting
somewhere in various states, that didn't have much attention over 
the last few weeks, at least not visibly.  Drafts IAB headers and 
boiler plates - that document has been finished for a long time and
is sitting in the RFC editor and refers to the BIS document.  We 
found a way to get out of there by changing the reference to RFC 
3932bis itself if the situation with 3932bis does not get resolved 
pretty soon.  So we want to get that out as soon as possible.  
That document basically changes the headers of documents, and 
changes some of the boilerplates so it's more obvious if a document
is an Independent RFC or an IETF Stream RFC or IAB Stream RFC.  

We're also working, and internally we've been rehashing, this 
document that is intended to describe the IANA functions, and what
the IETF needs out of that.  An update on that is imminent, and so
is an update on the IP model (on which we've been working, and 
which will be uploaded as soon as the queue is open)  
   
A little bit of news.  We had a communication with IANA on the way 
forward with respect to signing .arpa  We received a plan of action 
shortly before the IETF, in which there's two phase approach, where
they will proceed with the temporary set up to get .arpa signed in 
fourth quarter of 2009.  Given that quarter four of 2009 is still 
only six weeks old, we expect it will be signed before the end of 
the year, so it's really imminent.  

After the design has been finished for signing the root zone, that 
same system will be used for signing .arpa  We responded positively
to that plan and we find it's very important to get .arpa signed, 
to get its key signing key published in the IANA tar? and in the 
signed root whenever that is available, and make sure there is 
secure delegations to sign sub zones, and that is now being set in 
motion.  So this is some progress on that front.  

We made a bunch of appointments.  We've re-appointed Thomas Narten
as the IETF liaison to the ICANN BOT, and related to that, Henk 
Uijterwaal for ICAN NOMCOM.  And finally, we appointed Fred Baker 
to the smart grid inter operability panel.  And a number of you 
have been to the smart grid bar BOF yesterday and know what this is
about.  

Communication to other bodies...  There is an effort underway 
within the EU to standardize, to modernize ICT standardization.  
There was a white paper published by the committee, and we've 
reviewed that and basically replied with a number of facts around 
our process.  So that we, so we at least are sure that there's no
misunderstanding of how the IETF works.  We also provided comments 
to the ICANN CEO and the ICANN board of trustees on a study that
appeared recently that was about scaling the root.  You can find 
those comments on our IAB correspondence section of the web page.  

Something that is of a more operational nature is the 
implementation of the RFC editor model.  Just as a recap of the 
state of affairs, I've been talking about this previously, we're in
a transition period.  We're moving away from ISI as a service 
provider, and into an implementation of a model that has been 
developed over the last few years.  Within this model, we've got 
four legs: the RFC Series Editor, the Independent Submissions
Editor, the Publisher, and the Production House.  The IAOC is 
responsible for selecting the RFC production center and the RFC 
publisher, and the IAB is responsible for creating an advisory 
group, appointing an advisory group for helping us with the 
selection of RSE and ISE candidates.  That has all been done.

Looking for the RFC series editor is our responsibility, and the 
Independent Submission Editor, which is also one of the functions
within the model that is our job.  

Where are we with all that?  Well, the IAOC, you heard yesterday,
has appointed, awarded the production center contract to AMS and 
also the RFC publisher contract to AMS.  And the good news here is
that Sandy and Alice are the core members of the production center, 
which means, that the continuity of publishing and getting RFCs 
online is not in danger.  This is the good news so to speak.  

As far as the Independent Submission Editor goes, that is the 
editor that assesses the technical quality of documents on the 
independent stream, we've had significant delays.  That delay has 
been because we've been focusing on trying to find RFC series 
editor.  However, we have candidates and we are currently 
interviewing and assessing those candidates and we are basically on 
track with that now.  

As far as the RFC Series Editor function goes, we had a call in 
July, not quite half a year ago.  Closed nominations, August 
fifteenth, and the nominations were provided to the ACEF, the 
committee that helps us with assessing the candidates.  They 
interviewed candidates.  They've had long deliberations, and their 
conclusion was that there was no suitable match between the 
candidates, the functions, and the expectations of the role - those 
three variables didn't quite match.  And their advice was to seek 
somebody to manage the transition, to do a step back, and make sure
that pieces are in place and go for the long term solution.  
"Manage the transition" was the advice.  

The IAB went over this advice, turned it around a couple of times, 
and finally decided that the transitional RFC Series Editor (RSE)
way forward is the best plan, the best way out so to speak.  We've
defined that job, and you should have seen the announcement with 
job description, and call for candidates earlier last week, mid 
last week.  

There is an ongoing call for candidates, but the evaluation of 
whatever we have will start November 20 or so.  In a week.  So, why
do we think we will be successful now, or have a higher odds of 
success?  Well, there are a couple of things that are different 
than we had with the situation on July 8.  First, there is less 
uncertainty about the state of the production and publication 
functions.  It is known who is going to execute those functions.
There is capable staff there, there is institutional knowledge 
which makes the job easier.  

There's also, in the job description that we are looking for, 
there's more focus on the transitional aspects.  We've called out
that the person who is going to do this needs to refine the role of
the RSE after the initial transition, so that it is more clear what
the successor will be getting into.  There is an explicit task to 
propose possible modifications to the RFC editor model in order to 
see that things work better, when we go out for the more permanent 
function.  And, because this is a transitional management job, so 
to speak, it has different type of commitment, a different type of
personality, and also a shorter time space for commitment.  So we 
hope that the pool is wider, deeper, or of different dimensions.  

One of the things that we will also not do with respect to this, 
looking for candidates, is to disclose the names of the candidates
publicly.  We think that was a mistake and we don't do that now.  

As I said, the call for nominations is now open.  We will start
evaluation November 23, and we will accept nominations as long as
nobody has been announced.  

We believe that this is inline with RFC 5620, the RFC Editor Model
and the general community consensus.  That doesn't mean that we 
kept out, or we didn't go back to the community and ask "is this 
all okay?".  Because there is time pressure - ISI will stop this 
function December 31st, and January 1 we will be starting 
implementing this new model.  We couldn't afford to lose time.  
That doesn't mean we're not listening if you have any comments or 
things should have been done differently.  

We have been talking, as Bob Hinden said yesterday, with ISI, and
Bob Braden in particular, about their willingness to extend the, 
current contract on somewhat of a consultancy basis.  So at least
we have somebody so that balls do not fall on the ground, and 
somebody who can actually transfer some institutional memory to 
whoever gets this job.  There are two small links that you can get
and read them from the slides. 
 
Finally, it is worth mentioning this time we did not have appeals, 
and that basically closes my presentation.  

(applause)
   
With that, I would like to invite John, Stuart, and Dave for a 
session on internationalization.  And I'll start up the slide set.  


4.  Internationalization in Names and Other Identifiers
   
(John Klensin) All right.  Good afternoon, everybody.  

We've come to share some general ideas about internationalization,
and where we stand, and where we're going.  The plenary's goal is 
to try to inform the community about this topic.  This is not the 
first time the IAB has tried to do this.  We continue to learn more
and to try to share that with you.  

Internationalization is badly understood.  It is understood 
moderately well by a fairly small number of experts, most of whom 
end up realizing how little we actually understand.  But it affects
a large number of protocols, a large number of people, and should 
affect virtually everything we're doing in the IETF, and elsewhere 
there's a user interface.
  
We've got a new working draft which contains some recommendations 
about choices of binds and encodings that we'll talk about.  
Current version is draft-iab-encoding-01.txt.  It is very much 
still in progress.  

And more work is needed in this area, both on this document and 
about other things, and should continue.  

Internationalization is important and timely because a lot of 
things are going a on around us.  Names.  Names can have non-ASCII 
characters in them.  Not everybody writes in Roman characters, 
especially undecorated ones.  We're seeing trends towards 
internationalized domain names.  They've been floating around the 
Internet, fairly widely deployed since about 2003.  Earlier than 
that in other than the public Internet.  Pieces of this talk we'll
show you later, and they can be as simple as a string of Latin 
characters with accents or other declaration on some of them, or as
complex as scripts which some of the people in this room can read, 
and others cannot.  The things you can't read become problems.  The 
things you can read become opportunities.
  
We see URLs floating around.  Actually IRIs - what we discover is
that a great deal of software doesn't know the difference, and a 
great many people know even less about the difference.  

We have path names in various kinds of systems that use
internationalized identifiers, and use them in ways which are fully 
interchangeable with plane ASCII things, because the underlying 
operating systems have been internationalized.  

Users want to use the Internet in their own languages.  It seems 
obvious.  It took us a long time to get there.  We're not there 
yet.  At the same time, we've been making progress.  The MIME work
which permitted non-ASCII characters in e-mail bodies in a 
standardized way was done in Internet time a very, very long time 
ago, and has been working fairly successfully since.  

In China, IDNs are being used for all government sites.  A great 
many IDNs are deployed in the .cn domain.  Users in China see 
domain names as if the top level domains are actually IDNs.  We're
told that about 35 percent of the domains in Taiwan are IDNs, and 
almost 14 percent of the domains in Korea are IDNs.  

There's demand from various parts of the world that use Arabic 
scripts and they have special problems because the script runs 
right to left under normal circumstances, and most of our 
protocols that use URLs and identifiers have been written around 
the assumption that things are written from left to right.  In 
general, right to left is not a problem.  Mixing right to left and
left to right produces various strange effects.  
   
(Stuart Cheshire) Thank you.  Jon.  

So, I'm going to go over some of the basic ideas and some of the 
terminology.  

Unicode is a set of characters which are represented using 
integers.  There's actually about a million of them, but most of 
those are not used.  Most commonly used characters fit in the first 
65,000.  Most of even the less commonly used ones fit in the first 
200,000 or so, but the unicode standard defines up to about a 
million.  

These are abstract integers that, for the most part, represent 
characters.  I say 'for the most part' because there are 
variations.  You can have an E accent as a character, or the E with 
the character accent separately.  But ignoring those details, 
roughly speaking it's a set of characters with numbers a assigned 
to them.  

Now, you can write those numbers on paper with a pen, or on a 
blackboard with chalk.  When we use them in computer systems, we 
need some way to encode them.  And it's easy for us to forget that,
but the encoding is important.  

Here are three of the common encoding: UTF-32 is 32 bits in 
memory, of the normal way of representing integer values.  That 
means there are endian issues.  UFT-16 slightly more compact 
because the majority of characters fit in 65,000.  That means most 
unicode characters can be represented by a single 32-bit word, so 
that takes half the space.  Similarly, there are endian issues with
UTF-16.  UTF- 8 uses a sequence of 8 bit bytes to encode the 
characters.  And UTF-8 has some interesting properties, so I'm 
going to talk a bit more about that.  

The IETF policy on character sets and protocols specifies that all 
protocols starting in January 1998 should be able to use UTF-8.  
Why is that?  Why do we like UTF-8?  Well, UTF-8 has a useful 
property of being ASCII compatible.  And what that means is that 
the unicode code points from zero to a 127 are the same as the 
ASCII code points.  So, decimal 65, hexadecimal 41, represents an 
upper case A, both in ASCII and unicode.
  
I'm talking about integers here.  When you represent that integer 
unicode value using UTF-8, use the same value for values up to 127.
So this may seem very obvious, but, it's an important distinction 
between the integer value and how represent it in memory.  

The property of this is that if I have an ASCII file which is clean
7-bit ASCII, I can wave a magic wand and say that's actually UTF-8, 
and it is actually valid UTF-8 and not just 'valid', but valid with
the same meaning - it represents the same string of characters.  

For files that already have other meanings for the octet values 128
and up, like Latin 8859-1, that property is not true, because 
those code values have already been given other meanings.  But for 
plain ASCII, UTF-8 is backwards compatible.  UTF-8 uses those octet 
values above a 127 to encode the high numbered code points, and 
I'll explain how that works.  

So, in blue, we have the unicode characters that are the same as 
ASCII characters.  It's just a single byte in memory.  In the 
middle, we have the green ones, and those are the octets that 
start with the top two bits being one, or the top three or the top
four.  

And when see one of those, that indicates the start of a multi- 
character, a multi-byte sequence for encoding one unicode code 
point.  In the right in purple, we have the continuation bytes 
which all have the top bits one and zero.  

The nice property of this is, by looking at any octet value in 
memory, you can tell whether it's a stand-alone character, the 
start of a sequence, or something in the middle of a multi-byte 
sequence.  This is how they look in memory.  

So, we have the ASCII characters standing alone, we have the 
two-byte sequence where we start with the 110 marker, we have the
three byte and the four byte sequence.  

UTF-8 has some nice properties.  Part of being ASCII compatible is
that UTF-8 encoding results in no zero octets in the middle of the 
string.  That's useful for standard C APIs that expect null-
terminated strings.  .  

The fact that the bytes are all self-describing makes it robust to 
errors.  If there is corruption in the data, or data is copied and 
pasted, inserted and deleted, maybe by software that doesn't 
understand UTF-8, it is possible to recognize.  If I give a 
megabyte character, and look at a byte in the middle of the file, 
then you can tell whether you've got a stand-alone character.  If 
you look at the byte and the top two bits are 10, you know you're 
in the middle of a sequence, so you have to go forward or back, but 
you don't have to go very far before you can re-synchronize with 
the byte stream, and you know how to recode the characters 
correctly.  That is in contrast to other encodings that uses skip 
characters to switch modes, where you really have to parse the data 
from the start to keep track of what mode you're in. 

Another nice property of UTF-8 is because it has this structure, 
you can tell with very high probability looking at a file whether 
it's valid UTF-8, or whether it is something else like Latin.  
One of the properties is that if I see a byte above 127 in a file, 
it can't appear by itself, because that must be part of a multi-
byte sequence.  So there have to be at least two, and the first one
has to have two or three or four top bits, and the later ones have 
to have the top bits be one zero.  So the probability of typing 
8859-1 that happens to meet that pattern goes down very quickly for
all but the shortest files.  

Another useful property is that a simple byte-wise comparison of 
two strings of UTF-8 bytes using routines results in them sorting 
in the same order as sorting the unicode code points as integers.  
This is not necessarily for humans to see in terms of what we 
consider to be alphabetical order, but for software like quick 
sort that needs an ordering of things.  This is a suitable 
comparison which results inconsistent behavior whether you're using 
unicode or UTF-8.
  
One of the criticisms that's often raised against UTF-8 is that 
while it's great for ASCII - one character is one byte - and it's 
pretty good for European languages, since most European characters 
fit in two bytes, for Asian languages they often take three or four 
bytes per character.  And this has led to a concern that it results
in big, bloated files on disk.  While that may have been a concern 
10 or 20 years ago, I think in today's world, there are different 
trade offs we have to consider.  One thing is that everybody 
inventing their own encoding, which is locally optimal in some 
particular context, may save a few bytes of memory in that context,
but it comes at a big price in interoperability.  And when I talk 
about different contexts here, I don't mean just geographically
different places around the world, or different languages, but 
context like e-mail doing one thing and web pages doing a different 
thing.  Also in the context of applications and working groups, we 
have a tendency for each community to roll their own solution that 
they feel meets their needs best, which is different to that of 
other people's, and we have a lot of friction at the boundaries 
when you convert between these different protocols that are using 
different encodings.  We'll have some more examples of that later.  

Another aspect is that on most of our disks today, most of that 
space is taken up you with images, and audio and video.  Text 
actually takes a very small amount of space.  When you view a web 
page, most of the data that's coming over the network is JPEG 
images.  If you're looking at YouTube, almost all of it is the 
video data.  Those images and video are almost always compressed, 
because it makes sense to compress them.  

Ten years ago there were web browsers that would actually gzip the 
HTML part of the file to make the download faster.  I don't believe 
anybody worries about that anymore, because the text part of the 
web page is so insignificant compared to the other media that it's 
not that important.  

Another interesting observation here is, with today's file formats
like HTML and XML, quite often the the machine readable mark up 
tags in that file, which are not there for end users to ever see,
they're there to tell your web browser how to render the text, 
those tags really, they're just bytes in memory.  They have 
no human meanings, but it's convenient that we use mnemonic text, 
so we use ASCII characters for things like title and head and body.
And even in files containing international text, a lot of that mark
up is ASCII.  And I have had discussions very much like this with 
engineering teams at apple, all the applications that apple ships 
are internationalized in multiple languages and we have inside the 
application, which you can see for yourself, if you control click 
on it and open up to see the contents, they contain files that 
contain all of the user interface text in different languages. 
And we had the debate should it be in UTF-8 or UTF-16.  And clearly
for western European languages, the UTF-8 is more compact.  But the 
argument was for Asian languages that would be wasteful.  So I did 
an experiment.  

This is the file path, you can try the experiment for yourself.  I 
had a look at that file.  And in UTF-16 it was a 117K.  In UTF-8, 
it was barely half the size.  This is the Japanese localization.  
I'm thinking, how can that be?  

I was expecting it to be about the same size or a little bigger, 
but I wasn't expecting it to be smaller.  When I looked at the file, 
it's because of this...  

The file is full of these key equals value pairs.  And all that 
text on the left is ASCII text.  And the Japanese on the right may 
be taking three or four bytes per character, but that's not the 
only thing in the file.  So, I believe that the benefits we get 
from having a consistent text encoding so we can communicate with 
each other are worth paying possible performance size overhead that 
there might be.  And as this example shows, there may not be a size 
overhead in many cases.  

So that's UTF-8.  But we know that not everything uses UTF-8.  So 
the other thing we're going to talk is punycode, which is what's 
used in international domain names.  

Now, this is not because the DNS can't handle 8 bit data.  The DNS 
protocol itself perfectly well can.  But many of the applications 
that use DNS names have been written assuming that the only valid 
DNS names contains letters, digits and hyphens.  So in order to 
accommodate those applications punycode was invented.  And whereas 
UTF-8 encodes unicode code points as octet values in the range from
zero up to hexF4, punycode restricts itself to a smaller range of 
values, listed on the slide.  And those are the byte values that 
correspond to a ASCII characters hyphen, digits and letters.  

So, what that means is that when punycode encodes a unicode string
you get out a series of bytes, which if you interpret them as being 
ASCII, look like a sequence of characters.  If you interpret them 
as being a punycode encoding of a unicode string, and do the 
appropriate decoding, and then display it using the appropriate 
fonts, they look like rich text.  

So this is a subtle point.  We have these same sequence of bytes in 
memory or on disk or on the wire in the protocol, that have two 
interpretations.  They can be interpreted as letters, digits and 
hyphens -not particularly helpful, as it kind of looks like opening
a JPEG in emacs.  You see a bunch of characters, but that doesn't 
really communicate what the meaning of the JPEG is.  Or, the 
letters and hyphens can be interpreted as punycode data which 
represents a unicode string.  Let me give you another example of 
that.  

Does this look like standard 7 bit U.S. ASCII or not?  Let me zoom 
in.  We'll do a hum.  Who would say this is 7 bit ASCII?  Can I 
have a hum?  Who would say this looks like rich unicode text?  Hum?  

Okay.  Let me zoom in a bit closer.  This is a plane ASCII file.  
In fact it only contains Xs and spaces.  You can edit this file in
vi if you want.  

So, the same data has two interpretations.  Seen from a sufficient 
distance, it looks like Chinese characters, but it can also be 
interpreted as Xs and spaces.  So the meaning of this text depends 
very much on how you choose to look at it.  But I would argue that 
editing this file in vi would not be the most efficient way of 
writing Chinese text.  

So this problem that the same byte values and memory can be 
interpreted in different ways really plagues us today.  That was 
just a few days ago I was buying a hard disk on amazon and got 
these upside down question marks.  I think that's supposed to be 
dashes.  This isn't even in complicated script systems - this is 
just the characters that any English or American reader would 
expect to use in plain text.  

I remember when I had my first computer, an Apple IIe, and it could
only do upper case.  And then my next one, the BBC micro, had lower
case.  And the next one which was a Macintosh, in about 1985, could
actually do curly quotes, and I could write degrees Fahrenheit with
a degrees symbol, and I could do E-M dashes, and I could do Greek 
alpha signs.  I could write not equals as an equal sign with a line
through it the way I did in school when I was writing with a point.
Not exclamation point.  We have done it for so long, we forget.  
Not equals is a equal sign with a slash through it.  So by 1986 we 
had gone from typewriter to some fairly nice typography where I 
could type what I wanted on my Mac.  And here we are 10 years later
and things seem to have gone backwards.  I'm not happy.  How do we 
solve this problem?  

We make the user guess from 30 different encodings - "what do you 
think this web page might be?"  This is not something that we want 
to impose on users.  This is not something that the average end 
user is even qualified to understand what they're being asked here.
  
So, international domain names don't only appear on their own.  
They appear in context.  And here are some examples.  They can 
appear in URLs, they can appear in file paths on windows.  Of all 
these different encodings, which most of the people in this room 
would probably recognize as meaning the same thing, these are the 
only ones that in my mind are really useful, if we have a goal of 
supporting international text.  

If you you asked a child to draw a Greek alpha symbol, and gave her
a pencil and paper, plain pencil and paper, she would draw an alpha 
symbol.  She would not write %cn, % something and say that's an 
alpha.  That's complete insanity.  That is not an alpha.  An alpha 
is this thing that looks like an A with a curly tail on the right 
side.  If we want to support international text, it's got to look 
like international text.  

But because we have all these protocols that don't have a native 
handling of international text, we keep thinking of ways to encode 
international text using printable ASCII characters.  And when you 
do that encoding, who decodes it?  There's an assumption that if I
encode it with percent something or ampersand something, then the 
thing on the receiving side will undo can that and put it back to 
the alpha character it was supposed to be.  Well, we got bitten by
this yesterday.  We sent out an e-mail announcing this plenary.  
This was not staged.  This is real.  And some piece of software,
somewhere decided that unicode newlines were no good.  So it was 
going to replace them with the HTML ampersand code for a unicode 
newline.  

And something on the receiving side was supposed to undo that, and
turn it back into a newline.  Well, nothing did and this is what 
you all got in your email.  

This can get really crazy.  Suppose you have a domain name which is
part of an e-mail address, which you put in a mail to URL, which is 
then appearing on a web page in HTML text.  Is the domain name 
supposed to be actual rich text as seen to the user?  Or is it 
supposed to be puny code?  Because it's an email address, email 
uses printable encoding followed by two hexadecimal characters.  
Well, in an e-mail address do we have to do that escaping?  And the 
e-mail is part of the URL.  And the whole thing is going into a web 
page, so HTML has its own escaping for representing arbitrary 
characters.  

Do we use all of these?  A lot of people say yes.  It's not clear 
which ones we wouldn't use out of that four in the nested hierarchy
of containers.  If you're looking at an HTML file in your editor, 
you are very far removed from having rich text in front of you on 
the screen.  

So we tried, we decided we'd try an experiment.  What would happen
if we didn't do all this encoding?  What would happen if we just 
sent straight 8 bit data over the network and decided to try this 
email test.  Now, the SMTP specification says it's 7 bit only, but
we asked the question, what if we disregarded that, and tried it 
anyway, to see what would happen?
  
So, I sent a test e-mail, where I replaced the E in my name with a
Greek epsilon, and I with an iota, and I sent this e-mail by hand, 
using net cat, so it wasn't my mail client doing encoding.  I just
put the raw bytes on to the wire and sent them to the SMTP server 
to see how it would handle it.  I did it two ways, using the 
punycode-encoded representation of that first label of the domain 
name, X M -- something that looks like line noise.  And I did it a 
second time, just using the UTF-8 representation of that, which I'm 
showing here as the actual unicode characters.  

So to make that really clear, this is the text that I sent using 
net cat to the SMTP server.  This is the first one, using punycode, 
so this whole email is plane 7 bit ASCII.  No surprising byte 
values in it.  The first two lines are the header, after the blank
line, the rest is the body.  I point this out because headers are
handled differently from bodies.  Header lines are processed by the 
mail system.  The body by and large is delivered to the user for 
viewing.  

The second e-mail is conceptually the same thing, except not using
punycode, using just 8 bit UTF-8.  So this is the result of the 
first test.  Not surprisingly, the puny code in the body of the 
message was displayed by all the mail clients we tried as line 
noise.  Which is not surprising, because it's just text in the body
of an e-mail message.  There's no way that the mail client really 
knows that that text is actually the representation of an 
international domain name that's been encoded.  We could have some
heuristics where it looks through the e-mail.  I would not be happy 
about that.  Type the wrong thing in e-mail and it magically 
displays as something else.  That seems like going further in the 
wrong direction.  

In the from line, where we could argue that the mail client does 
know this is a domain name because it's user name angle bracket, 
user at example.com, close angle bracket, that is a clearly 
structured syntax for st e-mail address, and the mail client knows
how to reply to it.  It could conceivably decode that text and say 
'this is puny code'.  The intended meaning of this text is not 
xxc-x, it's a rich text name with epsilons and iotas in it.  One 
client did that, which was outlook on windows.  

The second test was the raw 8 bit UTF-8 data.  And I'm very happy 
to say, in our small set of e-mail clients that we tested, a 100% 
of them displayed UTF-8 text in the body in a sensible way.  

We had some more interesting results from the from line.  Gmail did
this very interesting thing where it clearly received and 
understood the UTF 8 text perfectly well, because it displayed it 
to the user as the punycode form.  I'm not quite sure why.  
Possibly for security reasons, because there is concern with 
confusable characters, which you will hear about in great detail in
a few minutes.  There is concern with confusable characters that 
you might get spoofed emails that look like they're from somebody 
you know but are really not.  Turning it into this punycode form, 
at some level, should avoid that.  I'm not sure it really does, 
because in a world where all of my email comes from line noise, the 
chance of me noticing that the line noise is different in this 
particular email, I don't know how much of a security feature that 
really is.  But that may be the motivation.  

Eudora 6 is an old mail client, written I think before UTF was very 
common.  Those characters there are what get if you interpret the 
UTF bytes as being ISO 8859-1.  And the last three here, to be 
fair, I don't think we should blame the Outlook clients here, 
because what appears to have happened is that the mail server that 
received the mail, went through and whacked any characters that 
were above 127 and changed them to question marks.  It didn't do 
that in the body, see, but in the header it did do that pre- 
processing.  So it's unclear right now whether it was the mail 
client that did this or the mail server that messed it before the 
client even saw it.  

So, back to terminology.  Mapping is the process of converting one
string into another equivalent one.  And we'll talk a little bit 
later about what that's used for.  

Matching is the process of comparing things that are intended to be 
equivalent as far as the user is concerned, even though the unicode 
code points may be different, the bytes in memory used to represent 
those unicode code points may be different, but the user intention 
is the same.  

Sorting is a question of deciding what order things should be 
displayed to the user.  And the encoding issue has various levels 
to it.  I've talked today about how to encode unicode code points 
using UTF-8.  There is also the question that the E accent 
character can be represented by a single unicode code point for E
accent, or as the code point for E followed by the accent combining 
character.  

So more terminology.  In the IDNA space, an IDNA valid string is 
one that contains allowed unicode characters to go into 
international domain names, and those can take two forms.  The term
will commonly used in the IDN community is a U label.  An IDNA- 
valid string represented in unicode, by which they mean, in 
whatever is a sensible representation in that operating system.  It 
might be UTF-8, it might be UTF-16, but it is one of the natural 
forms of encoding unicode strings.
  
An A label is that string encoded with the punycode algorithm, and 
xn-- to call out the fact that that is not just a string of 
characters in DNS, this is something that's encoded by punycode, so
you have to decode it in order to get the meaning.  

So I'll wrap up my part of the presentation with an observation.  
When it comes to writing documents, or writing an e-mail to your 
family, having the most expressively rich writing tools available 
is very nice.  When it comes to identifiers that are going to be 
passed around, and are used to identify specific things, then 
it's not quite so clear.  Because the bigger the alphabet, the more 
ambiguity.  

Telephone numbers use ten digits.  And by and large, we can read 
those digits without getting too confused.  We can hear them over 
the telephone.  Most people who can work a telephone, can read and 
hear the ten digits without getting them too confused.  When we go
to domain names, we have a bigger alphabet.  We have 37 characters,
and we start to get a bit of confusion.  Os and zeros, Ls and ones
and Is - there's a bit of confusion, which is bad, but it's limited
to those few examples.  When we move to international domain names,
the alphabet is tens of thousands, and the number of characters 
that look similar or identical is much much greater.  So with more 
expressibility comes more scope for confusion.  And I will note, 
that while we're going in this direction, of bigger and bigger 
alphabets, the computer systems we use went in the opposite 
direction.  They went to binary.  Because when you only have one 
and zero, then there's a lot less scope for confusion in terms of 
signaling on the wire, with voltage levels.  If there's only two 
voltage levels that are valid, you're high or low.  If there are 
ten that are valid, then a smaller error might mean reading a 5 as
a 6.  So we know that when we build reliable computer systems that 
binary has this nice property.  So, I leave you with that.  And I 
ask Dave to come up and tell more.  

   
(Dave Thaler) I'm going to talk about matching first.  So earlier 
on when we talked about definitions, we said, you probably thought
that matching meant comparing two things in memory.  That is 
certainly one of the as aspects of matching.  You do a database 
entry, and I know whether to respond or not.  There's another 
problem with matching - that is the human recognition matching 
problem.  

So, let's do another eye test here.  We have two strings up here 
that could be easily confused by a human.  Can you spot the 
difference?  Hum if you can spot the difference.  

Okay.  The difference that you can spot here, is that on the left,
this is .com, and on the right this is .corn.  It seems like a 
great opportunity for some farmer's organization, doesn't it?  

This illustrates that even in plain ASCII we have confusion.  Now,
some of you who have been participating in the RFID experiment are 
aware of another type of confusion.  On this slide, these are not 
capital Is, they are lower case Ls.  More confusion with just 
ASCII.  But wait, it gets worse.  

This is the Greek alphabet.  If Ethiopia, those are not the letters
E T H I O P I A.  If you check the lower case both of those, then 
they look different.  All right.  The lower case versions of the 
Greek letters there is fairly distinctive.  So as a result, we see 
the current trend to actually deprecate these various forms and 
revert to one standard one or one conical one if you will - in 
various identifiers such as IRIs and so on.  In IDNA, in 2008, some
of these characters are treated as disallowed for these types of 
reasons.  

Second eye chart, okay.  Look up from your computer and stare at 
the screen.  Hum if you spot the difference.  They both look the 
same.  If you can spot the difference, you may need an eye test. 
 
The difference here is that all the characters on the right are in 
the Cyrillic alphabet.  There's no visual difference.  What's worse
is that in ASCII .py is the TLD for Paraguay.  On the right, those 
are the Cyrillic alphabet letters corresponding to .ru, which is
Russian. 
 
Now, anybody here who actually speaks Russian or is intimately 
familiar with internationalization, will be quick to point out 
one important fact - 'jessica' uses letters that are not in the 
Russian language.  For example, J and S do not appear in the 
Russian language.  It points out there are alphabets, and languages
that use a subset of characters in those alphabets.  In order to 
get the letters that look like 'jessica', you have to combine 
characters from two different languages, but they're both in the 
same alphabet.  

So what this points out if you are a registry, that is going to be 
accepting say, domain registrations under your zone, then they may 
want to apply additional restrictions, such as not accepting things
that look like, or that contain characters that are not in their 
language.  If you're .py for Paraguay, if there are certain 
characters that don't want to allow, you can restrict that.  This 
particular example requires combining characters from two different 
languages, and there are other examples that are purely from the 
same language.  Epoxy, and on the right, this may be, say, a 5 
letter acronym for some Russian organization.  The problem is 
that's at the human matching layer.  Is that the thing you're 
looking for?  Does that match or not match?  

John is going to talk about a couple of more examples.  So 
hopefully your eye tests have been enlightening.  


(John Klensin)  We get more interesting problems when we move 
beyond the eye tests into a piece of human perception problem, 
which is people tend to see what they kind of expect to see.  

So we have here two strings which look different, but look 
different only when they're next to each other.  And, the first one
is a restaurant, and the second one is in Latin characters and 
something different altogether.  But they look a lot alike if 
you're not sensitive to what's going on.  

In general, if you have a sufficiently creative use of fonts, and 
style sheets from a strange environment, almost anything can look 
like almost anything else.  A number of years ago I came into 
Bangkok very late at night, and I was exhausted and I was driving,
being driven to the city and I saw a huge billboard and it had 
three characters on it and a red, white, and blue background and 
from the characters I was firmly convinced it was USA.  Well, it 
was in Thai, the characters were decorated, and having seen them
outside of that script, maybe I would have understood the 
difference.  Maybe I would not have.  

That brings us to another perception test which snuck up on my last 
month, me and an audience of other people.  We were sitting in a 
room, two months ago, at an AP meeting and there was a poster in 
the back of the room with the sponsors.  And on the poster we had 
these three logos, and the first one, pretend that you're not used 
to looking at Latin characters.  You look at the first one and you
don't know whether that character is an A or star.  

And then there's a reverse eye test.  See the second and third 
lines there, and convince yourself, assuming you know nothing about 
Latin alphabets, as to whether those are the same string or same 
letters or not.  Because this is the problem that you're going to 
get into when you're seeing characters in scripts and strings that 
you're not familiar with.  And it's a problem when people are not 
used to looking at Latin characters, when the fonts get fancy.  
People keep carrying out tests in which they say 'these things are 
confusable or not confusable' when they're looking at things in 
fonts which are designed to make maximum distinctions.  When people
get artistic about their writing systems, they're not trying to 
make maximum distinctions, they're trying to be artistic.  And 
artistic-ness is another source of ambiguity for people, as to 
whether two things are the same or different.  

We have other kinds of equivalence problems.  To anyone who looks 
closely, or who is vaguely familiar with Chinese, simplified 
Chinese characters do not look like traditional Chinese characters.
But they're equivalent if it's Chinese.  If it's Japanese or Korean 
instead, one of them may be completely unintelligible, which means
they are not equivalent anymore.  

As a consequence of some coding decisions which unicode made for 
perfectly good reasons, there are characters in the Arabic script 
with two different code points but which look exactly the same.  So
the two strings seen there, which are the name of the Kingdom of 
Saudi Arabia, look identical, but would not compare equal if one 
simply compared the bytes.  

Two strings, same a appearance, different code points.  Little 
simple things like worrying about whether accents go over Es do not
get caught in the same way these things do.  This is another 
equivalence issue.  What you're looking at are digits from zero to 
to 9 in most cases and from one to 9 in a few.  Are they equivalent?  

Well, for some purposes, if we write numbers in two different 
scripts are the same.  For other purposes, they're not.  

We've seen an interesting situation with Arabic in that input 
method mechanisms in parts of the world accept what the user thinks 
of as indic Arabic characters going in encode European digits.  
When they decode they treat the situation as a localization matter, 
so users see Arabic digits going and and coming out.  But if we 
compare them to a system in which the actual indic Arabic digits 
are stored, we get not equal.  

We've also got in unicode some western indic Arabic digits and some 
eastern Arabic indic digits.  They look the same above three, but 
below three they look different, and all of the code points are 
different.  Are are they equal or not equal?  

And if you think that they're digits and if you think, as we've 
said several times this week in various working groups and over the
last several years, that user facing information ought to be 
internationalized, remember that we show IP addresses in URLs which
users look at and sometimes type, so now assume you see some of 
these Arabic digits, two or three of them, followed by a period.  

And then you see another two or three Arabic digits followed by a
period.  And then see another one, two or three Arabic digits 
followed by a period and then see another one, two or three Arabic
digits.  Is that an IPv4 address?  Or domain name?  And if it's an 
IPv4 address, do you know what order the octets come in?  
   
The difficulty with all of these things, is that they're, fun, 
funny.  And then you catch your breath and say, he's not kidding.  
These are real, serious problems, and they don't have answers, 
except from a lot of context and a lot of knowledge.  Our problems
arise not when we're working in our own scripts, but in somebody 
else's.  So now we come back to the place where people started 
becoming aware of these problems - with internationalization of the
DNS.  If I can make a string in one script, or partially in one
script, look like a string in some other script, I suddenly have an 
opportunity, especially if I'm what the security people call a bad 
guy.  But those kinds of attacks cannot be deliberate.  They can be
deliberate or accidental, depending on what's going on.  We spent a
lot of time thinking in the early days of IDNs, believing if only 
we can prevent people from mixing scripts we'd be okay.  The 
example Dave gave shows how far from okay that is.  We're almost at
the point of believing that prohibiting mixed scripts is probably 
still worthwhile but it makes so little difference if somebody is 
trying to mount an a attack that it's really not a defense.  

If you have names in scripts that are not used in the user's area,
and the user is not familiar with them, many scripts become 
indistinguishable chicken scratch to a user who is not used to that 
script.  And all chicken scratches are indistinguishable from other 
chicken scratches, except for certain species of chickens.  

We talked from time to time about user interface design, and 
whether it should warn the user when displaying things from unknown
sources, or strange environments, or mixed scripts, but the UI may 
not be able to tell.  

We're in a situation with many applications these days that we are 
coloring, and putting into italics, and marking, and putting lines 
under or around so many things, that the user cannot keep track of 
what's a warning and what's emphasis and what's a funny name.  

And as was mentioned earlier, some browsers try to fix this problem
by displaying A labels, xn4 .  Our problem there is that those 
things are impossible to remember.  And one of the things we 
discovered fairly early is if we take a user who has been living 
for years with some nasty inadequate ASCII transliteration of her 
name, and we suddenly offer them, instead of that name written 
properly in its own characters, we offer them something completely 
non-nemonic, starting with X and followed by what Stuart calls line 
noise, and for some reason, the user doesn't think that's an an 
improvement.  

We've also recently discovered another problem we should have 
noticed earlier.  There are two strings, or one string depending on 
which operating system you're using, which are confusable with 
anything.  If one of these strings shows up in your environment, 
and you don't have the fonts or rendering machinery to render it, 
the system does something.  It can turn it to blanks, which is 
pretty useless.  But what most often happens is it's turned into 
some character which the system uses to represent characters it 
can't display.  

A string of question marks can either be a string of question marks
or it can be some set of characters for which you don't have fonts.
A string of little boxes can be either a string of little boxes, or 
some six characters for which you don't have fonts, or it can be an 
approximation to a string of question marks.  

And thus two strings in an environment in which you don't have the 
fonts installed can be confusable with anything.  

Now, the question is, what does a user do?  Well, it should be a 
warning to the user that something is strange.  But we know 
something about users from our security experience, which is if we 
pop-up a box which says, 'aha! this is strange - would you like to 
go ahead anyway?' the users almost always do the same thing, which 
is click okay, and go on.  

So, this is the string which can get you what you can't even read.  
And usually, depending on the operating system, trying to copy this 
into another environment by some kind of cut-and-paste situation 
will not work.  The number of colorful ways of it not working, but 
not working, is pretty consistent.  

So, we started talking about mapping before. In a perfect world we
would have a consistent system that allows us to perform the
comparison for us. Now, that sounds obvious.

In the ASCII DNS, when that was defined, we wrote a rule which said 
matching was going to be case insensitive, and the server goes off 
and does something case insensitive.  They get stored in case 
sensitive ways, more or less, but queries in one case, match stored 
values in another case.  It's all done on the server, nothing 
changes the other things.  

If you don't have intelligent mapping on the server, and you want 
to try to simulate it, which is what we've been trying to do over 
and over again in the international environment, where we're trying
to not change the server, or how we think about things very much, 
one of the possibilities is to map both strings into some pre-
defined conical form and compare the results.  

That sort of works.  It doesn't permit matching based upon close 
enough principles or something fuzzy, and that's right in some 
cases and terribly wrong in others.  But when we start converting 
characters, we lose information.  We convert a visual form of one
variety, into another form which is more easily understood, that's
fine for matching purposes.  

But if we need to recover the original form, and we've made the 
conversion, we may be in trouble, depending on what we've done.  
The mapping process in inherently loses information when we start 
changing one character into another one.  

Sometimes it's pretty harmless.  Case conversion may be harmless or
not harmless, depending on what it is you're doing.  Converting 
half width or full width characters to full width is normally 
harmless, depending on what you're doing.  Unicode has 
normalization operations which turn strings in one form into 
strings of another form, making the E with an accent character and 
the E with followed by an over-striking non-spacing accent into the
same kind of thing so they can be compared.  Usually safe.  

Unicode has other operations which take characters which somebody 
thought were perfectly valid independent characters and turns them
into something else, because somebody else thought they weren't 
independent and valid enough.  If that conversion is taking a 
mathematical script, lower case A and turning it into a plain A, 
it's probably safe.  If it's taking a character which is used in 
somebody's name and changing it into a character which is used in 
somebody else's name, it's probably not such a hot idea.  

And the difficulty is that we try to write simplified rules that 
get all these things right, and there are probably no such rules.  

So the mapping summary is, making up your own mapping systems is 
probably not a very good idea.  People who have gotten, who have 
been experts in it, spend years worrying about how to get it right, 
can't get it right either, because there is no right.  It depends 
on context.  And finding the correct mapping for a particular use 
very often depends on the language in use, and very often, when 
we're trying to do these comparisons - DNS is a perfect example, 
but not the only one - we don't know what the language is which is
being used.  If you need language-dependent mapping, and you don't 
know the language, you're in big trouble.  If you use a non- 
language dependent mapping in an environment where the user expects
a dependent mapping, you can expect the user to get upset.  In an 
international world, upset users are probably fate, but we need to 
get smarter how we handle them.  

(Dave Thaler)  Our next topic is the issue of encoding, which is 
the topic that our working draft focuses on.  

So, if we look at some of the RFCs that we have right now, and we 
can step back and construct a simplified architecture, this is the 
simplified version.  We are on a host, we have an application, it 
sits on top of and uses the DNS resolver library.  That's our over-
simplified model.  There are two problems.  And by the way, the 
IDNA work, for example, talks about inserting the punycode encoding 
algorithm in between those two.  

The two problems with this over-simplification: one, DNS is not the
only protocol.  Different protocols use different encoding today - 
and I'll get to this in a second...
  
And the second problem is that the public Internet name space, in 
DNS, is not the only name space.  In DNS.  As John mentioned 
earlier, the Chinese TLDs are not in the public root.  And 
different name spaces, as we'll see, use different encodings today.
  
So this is the more realistic, more complicated version of that 
previous picture.  On a host, you have an application.  That sits 
on top of some name resolution library, such as sockets or 
whatever.  Between those, they communicate with whatever the native 
encoding is of the operating system of choice.  UTF-8 and UTF-16 
are most common.  Underneath the name resolution library, you have 
some variety of protocols, and there's the union of different 
things that exist on various operating systems.  

And then this host is a attached, for example, to multiple local 
LANs, each of which may or may not be connected to the public 
internet.  And it may also be connected to a VPN, for example.  
Each of these is a potentially different naming context that you 
can resolve names in.  

So let's talk about problem No. 1 first, which is a multitude of 
name resolution protocols.  Now, it turns out that many of these 
are actually defined to use the same syntax.  What that means is if 
somebody hands you a FQDN, this thing with dots, you cannot tell 
what protocol is going to be used.  It might be resolved by looking
in your host file.  It might be resolved by querying DNS, or 
resolved in the local LAN by the server TCP.  Each of these are 
defined to use the same type of identifier space, same syntax.  

And so what happens, the name resolution library takes a request
from the application and tries to figure out where to send it, 
which protocol or protocols to try, and in what order?  And of 
course, if you have different implementations of different 
libraries that end up choosing different orders, you get 
interesting results.  

To make it more difficult, different protocols specify in different 
encodings, and so when you put those things together, that means 
the application can't tell which encoding, or in the case of 
multiple name resolution protocols being tried, which *set* of
encodings are going to be attempted because that's the decision 
made by the name resolution library.  

Let's talk just for a a moment about the history of what is a legal 
name.  All right.  The naming resolution library gets something, 
and is that a legal name?  What's that something?  Let's briefly 
walk through the history, so to understand sort of where the world 
is at today.  Back in 1985, RFC 952 defined the name of the host 
file.  It may be internet host names, gateway names, domain names, 
or whatever.  This is the one that said it contains ASCII letters, 
digits and hyphens, or LDH.
  
In 1989 is when DNS came along, published in RFC 1034, 1035, and it 
includes a section called 'preferred name syntax' which repeats 
the same description of LDH.  The confusion comes from the word 
'preferred' there.  Well, remember, this was before RFC 2119 
language.  Is that preferred a SHOULD or a MUST?.  Or is preferred 
mandatory?  There's confusion there.
  
That was 1989.  By 1997, 8 years later, we had RFC 2181, which was 
a clarification to the DNS specification, because of a number of 
areas of am a ambiguity and confusion that were resulting.  These 
are three direct quotes with emphasis added.  First one says 'any 
binary string', whatever, can be used as the label of any resource 
record.  'Any binary string' can serve as the value of any record 
that includes a domain name.  And, as Stuart mentioned, 
applications can have restrictions imposed on what particular 
values are acceptable in their environment.  

Okay.  So, to clarify, the DNS protocol itself places no 
restrictions whatsoever, but users of entries in DNS could place 
restrictions, and many have.  

Now, that was 1997, and in that same year there was work on the 
IETF policy which was published in, I think, January of '98, which
is RFC 2277.  This is the one that Stuart referred to, and here's 
the quotes from theirs.  The first one you saw earlier - 'protocols 
must be able to use the UTF-8 character set'.  And it then 
continues, 'protocols may specify, in addition, how to use other 
character sets or other character encoding schemes'.  And finally, 
'using a default other than UTF-8 is acceptable.' 

What's also worth pointing out - it's not just what it says, it is
also what it doesn't say.  What it can't say is anything about 
case, whether the E with accent, any types of combined characters, 
how things get sorted, etc.  There's no policy, the IETF policy did
not talk about such cases.  

And so, as a result, two unicode strings often cannot be compared 
to yield what you'd expect without some additional processesing.  
Now, since the protocols must be able to use UTF-8, but could 
potentially use other things, and since the simultaneously produced
DNS RFC said any binary string is fine, that means it complies with
the IETF policy.  

So, if the policy is used, UTF-8 and DNS comply with that policy, 
that means starting in that year, people started using UTF-8 in 
private namespaces.  By private namespaces, we mean things like 
enterprises, corporate networks.  By private namespace, again, we 
mean 'not resolvable outside of that particular network, not 
resolvable from the public internet.'  In their own world they go 
off and use UTF-8 and it became widely deployed in those private 
networks.  5 years after that, UTF-8 was widely deployed.  This 
included the work on punycode encoding for work in the public DNS 
name space.  

So, just to summarize here, UTF-8 is widely deployed in private 
namespaces.  Punycode encoded strings or A labels deployed on 
the public DNS name space.  

Now, within the internationalization community, there's been a 
bunch of discussions on link issues, and I think it's important for
the wider community to understand.  DNS itself introduces a 
restriction on the length of names: 63 octets per label, 255 octets 
per name (not including a zero byte at the end if you're passing it
around in an API.
  
The point is that non ASCII characters, as Stuart showed, use a 
variable number of octets and encodings that are relevant here.  
Now, 256 UTF-16 octets, 256 UTF-8 octets, and 256 A label octets 
are all different lengths.  So that existing strings can be 
represented within the length restrictions, in punycode-encoded A 
labels, but can't be encoded within the same length restrictions 
within UTF-8.  There also exists strings that can be encoded in 
UTF-8, but cannot fit within punycode and get an A label.  So, you
can imagine some interesting discussions there.  

Let's recap.  We've talked about multiple encodings of the same 
unicode characters.  There are things we called U labels.  WIth U 
think unicode, with A think ASCII.  U labels, you have something 
that is usually written out in that way.  A labels are things that
start with xn--
  
You have different encodings, say, the top form and bottom form, 
that are used by different encodings and different networks, even
within DNS.  Punycode A labels on the Internet and private 
intranets.  And even different applications that start to pay 
attention to different RFCs, ones that actually implement the IDNA 
document, and ones that don't.  Because you have all the 
differences across the protocols, networks and so on, you can 
imagine the confusion that results.  If you have one application 
that launches another application, and passes it some name or URL 
or whatever to use, the launching application may be able to access 
the directory of stuff, and you click on something, or you cause it
to launch another application the some way, and it passes the name 
and whether the an indication that just got launched, or the use of
the identifier in the same way.  In general, all bets are off.  It 
may or may not.  

You may get a failure, may get to some different site than what you
got to from the launching application.  

Similarly, if you have two applications that are both trying to do 
the same thing - two jabber clients, for example - and one happens 
to work and the other one doesn't happen to work, there would be a
switching incentive to say 'all I have to do is switch to the other 
one.'  

So let's walk through a couple of examples of applications that 
have actually tried to do a bunch of work to improve the user 
experience to deal with these cases, and you have to deal with the 
multiplicity of encoding, and I don't want to get to the wrong 
place or get failures.  So what we found is some applications have
tried to improve the algorithm to deal with this case that RFCs 
don't tell you how to deal with the multiple encoding issues.  

And so most of the time they actually get it right.  There are a 
couple of corner cases where they don't solve it 100 percent.  
Here's one.  You type in something into an address bar in a 
browser.  And in this example, the 'IDN-aware' application is one
that understands that there exists UTF-8 in some private namespaces
that it's connected to, and punycode and the public namespace it's 
connected to.  And so it knows which networks it's connected to, 
and may have some information about what the names are that are 
likely to appear on them, so it runs some algorithm to decide if 
this is an intranet or internet name.  At this point the string is 
being held internally in memory in, let's say, UTF-16 or UTF-8, 
whatever the native storage of the operating system is.  

In this example, let's say it decides it's going to be an intranet 
name.  So in this case it leaves it in sort-of the native encoding, 
does not run a punycode algorithm, and passes it to the name 
resolution API.  Then it goes to DNS and says we're using UTF-8, 
and sends it to the DNS server in UTF-8.  

If you have host B in this example, if that one has chosen to 
register its name in DNS, in the punycode-encoded form, the A label 
form, if that's the name that actually matches, that's going to 
fail.  If the host is host A, where it's using the same type of 
algorithm as the one on the top, it's going to succeed.  

So the normal expectation is that most of the hosts in that 
environment are all cooperating, or all have the same knowledge or 
configuration, and you actually get to host A.  If instead it's in 
the mode of host B, it will fail.  

Now, let's take a case where the application decided by looking at 
it that it's going to be an Internet name.  And by deciding by 
looking at it it could try one or the other.  In this case it runs 
the punycode on it.  The xn--4.  This goes to the public DNS.  

In this example, let's say that name does not in fact exist in the
DNS, and so the name resolution API wants to fall back and try a 
local LAN resolution and try LDNS or LMRR.  In this case, LDNS is 
defined to say the protocol spec says that 'if the name is 
registered there and resolvable, better ask for it in UTF-8 or 
won't get the answer.'  Here, if there is an indication to put it 
in the A a label form first before passing it to MDNS.  If MDNS 
puts it out there, it's not going to find a match.  Most of the 
time it does the right thing in both environments, but there are 
corner cases in both cases where things will fail.  

The next category is where you have some application that has 
become IDN aware, and another application that doesn't do anything,
it just takes whatever the user types in and passes it directly to 
name resolution with no inspection or conversion, because the name 
resolution APIs in this example are UTF-16 APIs.  So on the left, 
if this one is IDN aware, it will convert it using punycode to the 
A label form and it will go out and find the registration DNS in 
the punycode encoded form, whereas the other application passes it 
down in UTF-16.  DNS will convert it to UTF-8, and it will go out 
and not find it.  It doesn't find it, but there actually exists 
unicode code points with those binary strings and any string could
peer in the DNS.  

So what if the UTF-8 version magically found its way out there, 
either accidentally or intentionally.  They would get to a 
different site than what they expected.
  
Finally, the other category of differences is applications that 
want to say 'I don't know which one it's going to be, so I'm going 
to try them both.'  In some order...  So, consider 2 applications, 
one that decides to try the UTF-8 version first, and one that 
decides to try the A label first.  The one on the left converts it 
to punycode first, to the A label version first, and it goes out 
and finds the P or the A label version.  The other one might try 
UTF-8 first, and again might find a different version in which one
might be unreachable.  So you get non deterministic behavior.  Of 
course, the other one is intelligent too, so if that was 
unreachable you get the reverse.  So this is what applications 
actually do today.  

So, the basic principle, the basic learning from these, the fact of 
physics, right, is that conversion to an A label, or UTF-8, or 
whatever else is going to appear on the wire, can only be done by 
some entity that knows which entity or protocol namespace is going
to be used.  What is the encoding that is appropriate for that 
particular environment, or that type of resolution.  When an 
application tries to resolve a name, the name resolutions may try 
multiple of them.  So there's no single or right choice at the 
application layer.  This leads to two sort of remaining categories 
of hard issues.

In general, the client (using the term generically), whether it's a
host or application or whatever, because again, while we're using 
host names in many of our examples, the problems we're talking 
about are not limited to host names.  Many of the ones we've talked 
about today may be unique to host names, but they could occur in 
other identifier spaces - 'may or may not' I should say.  

The first one is the client.  The client has to guess, or learn, 
whatever the server encoding expects.  In many cases it may be 
defined by the protocol, and that's fine.  But if there are 
multiple protocols, it's part of the learning or guessing.  Names 
appear inside in other types of identifiers, and each identifier 
type today often has its own encoding conventions.  What is this 
identifier space?  Is it UTF-8?  Is it A label form?  Is it percent
or ampersand form or whatever...
  
And anything that converts from one name space to another name 
space, such as extracting an e-mail address from mail, or 
extracting a host name from URL, you have to convert from those
two sets of requirements.  Now, just saying, well, if they all used
a single encoding, they wouldn't have to do any of this transcoding 
in the movement between layers. 

By comparison, that's the easy part.  That's not the hardest part 
of the problem.  That's sufficient only if the only thing you're 
going to do is display it.  All other things besides the encoding 
issue - comparison, matching, sorting - they all require more work.
So just like RFC 952 defined what ASCII characters were legal in a 
host name, we need to define the unicode subsets for other 
identifiers.  

What are the things that are legal?  The optimal subset for one 
protocol or type of identifier may be different from what's optimal
for some other one.  Now, there also exists cases where based on, 
say, implementation differences, the way that two things display 
visually look different.  Usually, this is due to a bug.  Now, the
problem is nobody agrees which one is the bug or the correct 
behavior.  So that's a hard issue.

Stuart - back to you.  
   
(Stuart Cheshire) Thank you, Dave.  

So, Dave is right - having a single encoding does not solve all our 
problems, although, having lots of different encodings definitely 
does add to them.  This is not news.  We've known this for a while.  

There used to be computers using different character sets, and we 
recognized if some computers used ASCII, and some used another one,
the receiver had to work out which it was, and this was not going 
to give a good experience.  So the wire protocols used ASCII when
they could.  And if you had a computer that used the other one, you
needed a mapping table so you could convert to the common language 
on the wire and convert back upon reception.  We recognized that in
1969, but we seemed to have forgotten it now.  

To get out of the current chaos we need to go can beyond the 
current recommendation.  Merely supporting UTF-8 as one of the many
options doesn't solve the problem.  I think we need to move to a 
world where we only use UTF-8, and when you receive an identifier, 
or you receive a text string off the network, you don't have to 
guess what the encoding is, because there is only one encoding.  

So the summary is, for text that end users see, we want to have 
rich text, and that means unicode.  And for compatibility on the 
wire, that means using UTF-8 to encode those unicode code points.  
The corollary of this is for identifiers that are protocol 
identifiers, that are used for communication between computers to
tell each computer what to do, and aren't seen by end users, it is
much harder to make the argument why those should be unicode, 
because the bigger the alphabet, the more the scope for confusion 
and the more chance of things not interoperating.
  
With that, I'd like to open the mic for questions.  I think we 
should do half an hour for questions on this internationalization 
presentation, and then that will leave half an hour for general 
questions to the IAB.  

We will take new questions at the middle mic and the end ones, and
the in between ones for follow-ups.


Open Mic:

(Bob Briscoe) A question for John.  How long have we known about 
the security problems?  Because, it was sort of quiet hearing 
about them, but this is the Internet, and we ought to be fixing 
these things.  
   
(John Klensin) What do you mean by 'knowing about the security 
problems?'
  
(Bob Briscoe) Well, the problems of being able to spoof one 
character with another, and change fonts, etc.
  
(John Klensin) Since long before this process started.  We've known 
about confusability in characters since we started looking at 
multiple scripts.  We've known about some of these confusion 
problems in titles of things since we deployed MIME with multiple 
character sets, and that would have been in, I'm guessing from 
memory, but like 1990 or shortly thereafter.  I gave a presentation
at an ICANN meeting Melborne that exhibited some of these abilities
to write different things in different scripts.  At that time, it 
was a general warning about these things.  

We've certainly seen more subtlety, as we've understood these 
things better.  I used to joke that one of the properties of this 
whole internationalization situation, when one is actually trying 
to use the strings and identifiers, rather than printing them, is 
that every time we looked at a new script, we found a new and 
different set of problems.  It was like going through a field and 
turning over rocks, and each time you found something new.  So I'm
not certain how to answer your question.  

This is just epidemic in an environment where we're suddenly moving 
identifiers from a world in which the maximum number of characters 
we treat as different is around 36, to an environment where the 
maximum number of characters we treat as different is in the range 
of tens of thousands.  

(Bob Briscoe)  I guess my question is, your presentation told us 
about the problems.  If we've known about these problems for 19 
years or so, are there, you know, could we do a presentation on a
solution space?  Is there any solution space?
  
(John Klensin)  Let me give you an a different answer - we've had
these problems for somewhere between two and 6,000 years.  

(Bob Briscoe)  Time to fix it.  

(John Klensin)  Absolutely time to fix it.  

(Bob Briscoe)  You might do it before something goes to full 
standard.
  
(John Klensin)  The fundamental issues here really rely on two 
things.  One of which is that we can design very, very highly 
distinguishable fonts.  And we possibly need to design highly 
distinguishable fonts a across the entire unicode set, and they
would be so ugly nobody would want to use them.  

We, in theory, could teach everyone about all of these 6,000 
separate languages, and only slightly smaller number of scripts in
the world, but that isn't going to happen.  

So the answer to your question is that there's a tremendous amount
of reliance on user interface design here.  And what we need to 
understand is that there's both the problem and an opportunity.  
The opportunity, which is very important, is for people to use the 
internet, in their own script, in their own language in their own 
environments.  That's really important.  

Our problems arise when we start looking at, and operating in, 
environments which one of us doesn't understand.  I'm gradually 
learning to recognize a few Chinese characters, but my ability to
read Chinese or Japanese or Korean is zero.  I don't know about 
you.
  
But if your situation with regard to Chinese characters is the same
as mine, if I send you a message in Chinese characters, we are both 
having a problem.  If I send a message in a script I can read, or 
an identifier in a script I can read, but you can't, you've got a 
whole series of problems.  You can't read the characters, you 
probably can't figure out how to put the characters in a computer 
if you can read them, and you're going to be easily tricked.  And 
we're going to have to learn how to deal with that, just as we've 
had to learn about non interoperability of the human languages.  

If I have a face-to-face conversation with you, using a language 
which only one of us understands, then at a minimum we're going 
to have an interoperability problem.  At a maximum, if I can make
that language sound enough like something that you expect to hear, 
or you can do that to me, then we may have a nasty spoofing 
problem.  

And again, these issues are thousands of years old.  And we kind of 
learned to cope.  And we learn to cope by being careful, and we 
learn to cope by remembering that those little boxes are a big 
warning sign that we may not be able to read something.
  
Many of us have started filtering out any email which arrives in a
script which we can't read, because we know we're not going to be 
able to read it anyway.  And those are the kind of things we do.
It's very, very close to user level.  And I don't think there are 
any easy answers.  

But the alternative to this situation is, say, oh, oops, terrible, 
there might be a security problem so nobody gets to use their own
script and that answer is completely unacceptable.  

(Yoshiro Yoneya)  All right.  From my experience in the
internationalization of protocols, one of the hardest issues is to
keep backward compatibility.  So, inventing encoding is to get 
interoperability, or backward compatibility with existing protocol.
That's the reason why there are many encodings.  So, I hope to have 
migration, generic migration guidelines for the protocol 
internationalization, that will be very good future work.  
   
(Stuart Cheshire) I think one of the things we need to be careful 
of - it's easy to fall into this trap to say we need to be backward 
compatible.  And that actually means something else.  But if the 
thing at the receiving end doesn't know that it means something 
else, we've not got international text.  We have lots of percent 
signs.  

(Larry Masinter) This is a actually a followup about how long we've 
known about the problem.  I'll take some blame.  In 1993, I think, 
there was an internet draft where I proposed internationalization 
of URLs.  Based on discussions in 1992, when I thought it was a 
simple problem.  Just used UTF-8, and that there were regular URLs
and internationalized ones.  But I think part of the problem was 
the switch from thinking of these as, these weren't names, they 
weren't identifiers, they were locators.  The notion of comparing 
two of them to see if they were the same was not a requirement.  

And at the time, there were no caches.  And so, the notion of 
figuring out whether or not this URL was the same as that one, 
wasn't part of the protocol stack.  And, therefore, some of the 
problems we're seeing, that idea of phishing that would actually 
look at the name and believe something, merely because you saw it 
on your screen, didn't have anything to do with where you were 
trying to go.  That was a requirement that was added after the fact 
without a lot of thought.  

And, if you you think about it, we've add on some requirements that 
maybe shouldn't be there.  And so, I think if you look at all of 
your examples, there's still some problems, even if don't try and
compare.  But almost all of the problems that you've listed really
have to do with comparison, and of locators.  As lots of them do.
  
A lot of the problems have to do.  You had a lot of things - look 
at a this and that, and are they the same or different?  And if 
you didn't have the problem of trying to decide, a user trying to 
decide whether or not they were the same ahead of time, you 
wouldn't see a problem.
  
(Dave Thaler)  Larry, these issues exist when a system decides to
take a a label or a string which can be user input, and compare it
with something which is stored in in the database.  The classic 
matching look up problem in DNS or otherwise, and then the question
is whether the answer to the question, whether or not those 
matched, meets user expectations as far as the user is concerned 
is off the wall.  There's no way to avoid that particular problem, 
other than require user to have universal knowledge of exactly 
what's stored.  And I do do mean exactly.  

(Larry Masinter)  No.  I think, if you are, if you put a human 
communication in the loop, that you're going to print something on
the side of the bus that you want people to type into their 
computers, it is your responsibility at the time that you print 
that on the bus, to do it in a way in which the users will have a 
satisfactory experience.  It is not the responsibility of the 
intermediate system to make up for the fact that the printing was 
something that could be an O or could be a zero, or could be an L 
or one, you know, you get a password and I can't tell because the 
font used was bad.  

It's the responsibility of the printer to do that in a way that 
will cause appropriate behavior, and not to choose, to print things
that are unrecognizable or have ambiguous forms.  There are lots of 
systems that never go through that phase of the translate into a 
perceptual representation and translate back, and expect that to 
happen.  

So, I think that we can make progress by being more careful about 
what we choose to accept as requirements of the overall 
communication system.  
   
(Dave Thaler)  I just want to comment on one of the things you 
said, about whether most of the problems are due to such and such.
I want to summarize that we actually talked about at least two 
different, big categories of problems.  One category of problems is
when there are multiple unicode strings, in other words, multiple 
sets of unicode code point numbers that can be confused or not, or 
matched or whatever with each other.  There's one set of things 
that are inherent in that, and it's lot about user interface, 
display, and so on.  

The second is one set of unicode code points.  Those are two fairly 
different sets of problems that we talked about tonight.  

(Larry Masinter)  I think if follow the paths, these differing 
alternate forms that look the same don't fall from the sky.  They
don't appear magically in the middle of the system.  There's some 
data path that either transmits them, and along the way is screwing
them up, or there's some human perceptual path along the way that 
involves printing things out or reading it out loud and 
transcribing it in a way that's inappropriate.  

(Dave Thaler)  Pete, do you have follow up on this.  

(Pete Resnick)  I do.  I actually disagree with Larry, at one 
level.  We're talking about identifiers being used for user 
interaction, that are also being used for machine interaction, for 
protocols.  And that's inevitably going to get screwed up, because
the stuff that we use for user interaction has variants, it's got  
humans involved.  Once a user has to type and interpret something, 
and there are variations of how it might be typed or interpreted 
based on context, there's nothing to be done.  

What we've done is increased the probability of that happening from
those 37 odd characters to tens of thousands of characters, 
incredibly.  I used to be much more in the camp ten years ago if 
you said to me that today, I would say such a thing, I would 
have thought it ridiculous - we have to straighten this out with 
using proper encodings, this is done.  

You know, e-mail is no longer reliably delivered because of spam. I 
don't care anymore if e-mail is not delivered because a user cannot 
type in the e-mail address exactly the way I put it on the screen.  
There's no way to make that precise.  If we get unlucky, the person 
who chose that e-mail address gets what they pay for.  

(Stuart Cheshire)  I want to add one clarification to Larry's 
point.  When we talk about comparing strings, we're not talking 
about showing two strings to the user and saying, do you think 
these are the same.  

(Larry Masinter)  That was one category.  

(Stuart Cheshire)  We were talking about, when a DNS server has a 
million names in its own files, and a query comes in for the name 
the user types, the DNS has to go through its own file and work out 
which record that query addresses.  

And you mentioned the subject of phishing, that's not a requirement
that the IETF decided to put on identifiers.  That's something that 
criminals decided would be lucrative for them, and we have to think 
about the consequences.  

(Larry Masinter)  Let me see if I can clarify something.  I'm not 
saying it's not a problem here.  I'm just pointing out that trying
to solve it, at a different place than, I'm trying to point out 
where I think it is going to be most productive, as far as looking 
at solutions.  And it is inputing restrictions on what is output or 
displayed, in such a way that it is more unambiguous about how to 
enter it in a way that would be reliable.  
   
(Stuart Cheshire) Okay.  

(Larry Masinter) And to focus on that area.  

(John Klensin) You're asking people who design user input and 
output procedures to constrain their designs in a way which makes 
things unambiguous.  My experience with telling designers what they
can and cannot do has been pretty bad.  

(Larry Masinter) Somebody is going to have to do something, and 
trying to patch it somewhere else is not going to be effective.  

(John Klensin) Another way of looking at this is that these 
problems would be vastly diminished if we let no one on the 
internet who wasn't trained to be sophisticated about these kinds
of things.  And while there were times in my life while I probably
would have a approved of 'nobody uses a computer unless they pass 
the training course and get a license', I think that's probably 
harder to constrained than designers.  

(Spencer Dawkins)  Spencer Dawkins, and probably the least clued
person on this topic that stood up so far.  So I'm thinking, I'm 
thinking the kind of questions I would ask would be triage kind of 
questions.  So, is this situation getting worse or, have we already 
hit the bottom?  
   
(Stuart Cheshire)  It's still getting worse.  We think of it as an 
educational process in which we continue to learn.  

(Spencer Dawkins)  How much better does it have to get before it's
good?  Before it's okay?  I mean, how much do we have to fix?  

(Stuart Cheshire)  I think with a big name space identifiers, there 
is always going to be problems.  Our goal is to minimize the 
unnecessary problems.  

(Spencer Dawkins)  I see e-mails coming through and it's 
disappointing. 
 
(Stuart Cheshire) I think people who are working on this problem 
have job security.  Sort of like security in antispam and so on.  
As long as human languages continue to exist, as long as there are 
humans using the network, the problems will exist.  The one way to 
to make them go away would be to remove all the humans.
  
(Spencer Dawkins) So, tell me if I've got this right.  That, once 
upon a time, there was ASCII and there was the other system.  And
people on each side wanted to get to the resources on the other 
side.  So, there was a death match, and we picked ASCII and life 
went on.  

Are we in any danger of being able to have that kind of a covering 
today, I mean, do people worry that they can't get places, in other 
scripts and things like that?  Do people see this as a problem?  
And John has been, you know, demonstrating this, you know, on 
napkins and stuff like that for me for a while, just as a curiosity
kind of thing, so I congratulate you guys for managing to scare the
hell out of me yet again.  

But, like I say, I'm kind of curious about that.  So, I'll sit 
down.  

So you asked a question there, at think at the end, you said, I 
think part of the question you're implying is 'how often are people 
actually running into problems today, right?'
  
(Spencer Dawkins) Basically, like I said, the ASCII thing is, 
there's a computer I need to get to, and I can't get there.  
There's a computer in Saudi Arabia that I can't type the name of.
How big of a problem is that?  

(Stuart Cheshire) As an example, in some of the cases I showed, 
applications are trying to deal with the fact that there's multiple 
encodings, it's a corner cases that fail.  People run into that, 
but not very often.  So people have done a good job of compensating
for that.  But we keep its as rare as possible.  But the phishing 
attacks, whenever somebody tries to be dangerous, hopefully that 
isn't accomplished either.  

I'm goes to close the mic lines now, we have about 5 more minutes.
Do we have a followup there?  
  
(Bob Briscoe)  Maybe the question could be better posed as 'do we 
think there's sufficient protocols and languages that we're
standardizing for applications that need to be secure, to be able 
to be?'  And what I'm thinking is, if you're viewing a font and an 
encoding through an application that's some business, you know, 
important thing, legal, whatever, could the application writer say, 
well, normally in your locale, you'd be restricted to this, so if 
anything outside that range comes in, I can warn, et cetera, et 
cetera, and I can sign all your fonts and encodings.  Do you think 
there's enough support there for an application to do that?  

(Stuart Cheshire)  I think there is scope for heuristics to spot 
specific behavior, but there's trial and error, and they tend to be 
developed over a long period of time.  When find something that 
doesn't work, find the particular heuristic.  


(Dave Crocker)  So I got up before Pete, to ask you, Stuart, ask 
you about the end of your presentation, but it was, my question is, 
predicated on exactly the point that Pete was making.  Which is 
that much of the mess right now, well, there are inherent 
complexities in the topics, but most of the mess is a layer 
violation that we created in simpler times, and the simpler times 
probably helped things a lot back then, in terms of making the 
internet usable.  Making the arpanet usable.  So that e-mail 
addresses, andlater web URLs, and to a large extent domain names, 
had this user interface use, and over the wire use, we made a lot 
of things simple that way, but we built the problem we have now.
  
And, we continue to try to maintain the layer violation, and say
that's okay, we have to do that.
  
The end of your presentation, didn't phrase it this way, but 
essentially was going, no, maybe we really don't and we certainly
should try not to.  That is, we should go to a canonical over the 
wire representation.  

The piece, the little I touched this area, seems to suffer a lot 
from, and it will suffer even without this, but, it suffers worse,
is the difficulty of getting the distinction between the user 
interface human factors, use bit, the human side stuff, and 
distinguishing it from over the wire.  

And I totally understand the resistance to it.  But, our job is to
fix problems, and we really need to be careful we don't just 
maintain them.  

We've been having, in the years that the international stuff has
been worked on, we've been having to deal with some realities that 
forced us to make decisions that do maintain them.  So I think that
your suggestions at the end is, I mean, it's charmingly '70s.  It's
'go back to canonical forms and over the wire.'  And so the 
question has to do with achievability.  

How do we get there?  Do we get there before we get to IP v6?  Do 
we get there before we retire?  Well, some of us anyhow?  I mean, 
it's clearly the right goal.  But is there anything practical about
the goal, and if so, how?  
   
(Stuart Cheshire)  I think moving to UTF-8 does not solve all of 
the problems that we talked about, not by a long way.  But it 
solves one of the problems that at least we know which characters
we're talking about when we're trying to decide if they're equal.  

Who will solve it, is implementers writing software, they need to
write their software that way.  Working groups writing standards 
need to specify that.  I think I'm less pessimistic than are you 
are about the prospects of moving in a good direction here.  And in
the interest of time, we'll take the last question.  
 
(John Klensin)  Dave, I think the other part of the answer is 
precisely that we have to stop taking the short cuts.  Of assuming
that by dropping a mechanism for internationalized characters into 
something which was designed for ASCII only, that that's a solution
to the problem.  Occasionally it will be a solution to the problem.
But we may have to start thinking for the first time in our lives, 
seriously about presentation layers and identifiers which work in 
this kind of environment, rather than things that have been patched
for a little bit of internationalization in an ASCII environment.  

And I don't think those problems are insurmountable.  I don't think
the problems of getting serious about localization sensitivity are 
insolvable.  But we need to get serious and start working on them 
at some stage.  

(Dave Crocker)  The layer violation is the reason why the Internet
is successful, and popular.  Right?  

(John Klensin)  Was.  

(Dave Crocker)  Was.  Well, is.  It is.  But I think, I'll put in 
the process plug.  We had a bar BOF in Stockholm, and there was a 
lot of interest in internationalization and resource identifiers to
take this document, that there's an RFC proposed standard.  But 
having 9 different solutions and 9 different committees for how we 
approach the problem seems like a bad idea.  

And there are a lot of different groups working on their solution 
on how to go about it.  And I'm hoping we can converge into a 
single, if somewhat interesting working group.  So, I encourage you
to consider it.  Public dash IRI, and I think IRI to the working 
group.  

(Olaf Kolkman)  All right.  Thank you.  With that, I'll ask the 
rest of the IAB to come up on stage and we'll take general 
questions.  
 
So while the rest of the IAB comes to the stage for the open mic 
session, there was a suggestion yesterday to keep things short.  I 
would like to remember the audience of that, we all have other own 
responsibilities here.  

In previous sessions I wrote a mail to the audience before the 
plenary saying 'if you have a question, please write us a mail.'  
That would help us to actually think about an answer, and answer 
concisely.  And it would help to think about the question a little 
bit.  

That never happened, really.  But I think that might be a mechanism
for short mic lines and intelligent answers to your questions.  So, 
please keep that in your mind for the rest of the year, the open 
mic is not the only way you can approach the IAB, or the community.  

With that said, is there anything somebody wants to bring to the mic? O
Oh, I should introduce all.  Let's start at the far end with Jon.  

<Introduction of IAB members>

(Olaf Kolkman) Okay.  Thank you.  
   
(Tina Tsou)  So, the document that the IETF produced, which is one 
that is about NATs.
 
(Dave Thaler) It is mostly a repeat of a lot of the same points 
that have been made on the topics.  What the IAB's thoughts are, if
I can sum up what the RFC or the RFC-to-be says, is that the most
important point is to preserve end-to-end transparency.  

Now, there are multiple solutions that preserve end-to-end 
transparency.  So IPv6 nat is a solution, is something that's in 
the solution category.  Now, it's possible to do translation in 
ways that preserve end-to-end transparency, it's possible to use 
tunneling, it may be possible to do other things.  

The IAB statement is that it's important to preserve end-to-end 
transparency.  And there is no statement saying there must be NAT 
in IPv6 or not.  So the first main point is that on the topic of if
NAT could be done in a way that preserved end-to-end transparency
is neither arguing for or against that. 

The second thing that's the main point of the document, is that 
there exists a number of things that people see as advantages in 
IPv4 NATS that they use them for.  Re-numbering, all the things 
brought up in the IETF in the past.  Some of them were documented 
previously in RFC 4864, some of them were not entirely there and so
we elaborated on those.  Those are things that people see as 
requirements for solutions.
  
Today the simplest solution that people see is v6 NAT, but that may
or may not be the only or the best solution.  And so, the second 
point was there are some requirements there, that the community 
needs to work on solutions for.  

That's basically to sum up.  Anybody else want to add anything?  
That's what the IAB's thoughts are are.  Once you get into ways to 
meet those requirements, that's for the IETF to figure out.  But we 
wanted to comment on what we believe the requirements are, and what 
the constraints on are, and to what extent NAT does or does not 
meet those requirements.  

(Gregory Lebowitz)  I think the other thing that we tried to make 
very clear in the document was that every time you use a NAT to 
solve one of those problems, you give up something significant.  
And we tried to call out what those things were - the trade-off 
and the cost associated.  
   
(Dave Oran)  Well, I just want to mention that it is somewhat 
difficult to establish transparency in any translation system, but
it is in fact possible.  Where you run into trouble is trying to 
take the simple approach, where you with would attempt to confine 
the translation state independently, in individual boxes with no 
coordination of the translation state, and that results in the non- 
invertability of transformation and the loss of transparency.  So, 
something I would encourage the community to do is to look at NATs 
not as simply 'what can I get away with doing the minimal amount of
work in in order to maybe get something that I want', because the 
consequences in negative terms for transparency are pretty severe.  
And with somewhat more work, translation-like approaches may in 
fact be quite acceptable.  
   
(Olaf Kolkman)  Yes, my apologies for a minute ago to be a little 
bit fast in trying to close the mic lines, I see that people have 
collected, queued up.  

Peter, please.  

(Peter Lothberg) Okay.  I'm Peter.  So, there was an obvious reason
why we have IPv4 NATs, and then people made them do all sorts of 
fantastic things, and I think people have them because they want 
more addresses because they have more things inside their houses 
and said I don't want to go there.  But a major use of it is some 
kind of gate keeper, some kind of policy.  It's a policy box that 
sits there and implements what policy I decide I want to have 
coming into my house for people.  I look out the door, the door 
bell rings, how do they look, would I let them in or not.  

So, last time we forgot to do any work in the IETF and we ended up
with a mess.  I heard talks about smart grids, intelligent houses,
and assume for a second that if we use IPv6 addresses on all of 
them and they have their own unique address, we still want policy.
And those devices are so small, they probably need somebody to help 
them.  See maybe the IETF somebody should go look at, so okay, in 
the future, we still need a policy control device that sits at the 
boundary of something and something, in order to enforce policy, 
to make sure the pool man gets to the pool and the alarm company
gets to  alarm, and vice versa.  And let's get that done, before 
people make more kludges.
  
(Gregory Lebowitz) Don't we have those, aren't they called fire 
walls?  

(Dave Oran)  Can I jump in?  So, as somebody who has been skeptical
of most things firewalls do for as long as I can remember, I 
absolutely agree with Peter.  However, in some cases, trying to 
capture the correct policy semantics simply at the individual 
packet layer as one would with a fire wall runs into many, many, 
many problems that make things in fact worse rather than better.  

If the fire wall simply processes on a per packet basis, there's 
lots of -- if the box attempts to do packet inspection, either 
shallow or deep, I think we're all aware of the problems that that
type of approach goes into.  So there's going to be some kind of 
application intermediate, that's going to be needed for various 
applications to enforce policy.  Get used to it.  Don't try to do 
everything by per packet processing and firewalls, or even worse, 
to try and guess what the correct policy ought to be for an 
application by doing intermediate inspection of packets.  

(Peter Lothberg)  I was more thinking, more of the Swiss Army knife 
solution here.  I wasn't only thinking packet inspections.  I was 
thinking, this is the thing where I actually have stored my policy 
and all the devices I have in the house actually goes, and it's the
system where it gets stored, the database, the PKI, blah, blah, 
blah.  

(Dave Oran) Then we agree.  But it doesn't necessarily have to be 
the gateway box that sits physically at the boundary.  

(Peter Lothberg) Correct.  

(Dave Oran) But people don't want to buy many boxes.  

(Peter Lothberg)  Yes, they only want one box that needs to get attacked.  
Right.  

(Remi Despres)  That's a follow on of the first question, I think
you made the point that end-to-end transparency is something 
important.  And of course, I do agree.  Now, yesterday something 
strange, with reference to this happened.  Among the interesting
technologies which are proposed to restore end-to-end transparency 
and move in the right direction, there is one which is in IPv4, the 
extension of addresses with trenches.  Now, there was a BOF on A 
plus P.  Now, some of the major birds of a feather, those people 
who are interested in the subject, were not permitted to talk, to 
present I mean, that is, to present their contribution to that.  

And the conclusion was that we would no longer be permitted to talk
in any group, on this approach on end to end transparency.  I still 
expect that there will be a reversal of that decision, that it will
be possible in this area to work on A plus P.  
   
(Dave Thaler)  Part of that is a question for the IESG, and part of
it is for a question for IAB.  And I'll comment about the IAB
portion, which is about the end-to-end transparency, versus say the 
evolution of the model.  The IAB has another document about what 
the assumptions are and the impact of changing those are, and 
whether those impacts should be done, you know, obviously, the 
whole point of the evolution of the IP model is to say, well, the 
IP model does evolve, but evolution has to happen carefully.  

Architecturally, what I think those at the BOF are trying to weigh 
between is, for IPv4 you have some inherent problems in IPv4 - we 
know that.  So, at one hand, you said, well, not we the IAB, those 
in the BOF, we're looking at a one alternative that would say, 
maybe there's better end-to-end transparency, but more changes (for
some definition of changes). 

And at the other extreme, there might be less end-to-end 
transparency.  There's no single right answer, because 
architecturally the answer is remove the limitations in IPv4 and go
to IPv6, that would be the architectural solution which gives you 
end-to-end transparency and preserves the model.  So we see a 
tussle between the two sets of requirements that are trying to be 
met, but cannot be met at the same time architecturally.  

(Remi Despres)  For the tussle to be resolved, it should be 
possible to talk and explain.  

(Dave Thaler)  That's the question for for IESG, not IAB.  

(Remi Despres)  Okay.  That you thank you for the information.  

(Lorenzo Coletti)  Since it's open mic and as somebody who deployed
IPv6 network and services, that started because we needed more 
address space.  If I look at the papers, I read the paper about the
botnet that researchers got into control of the botnet and they had 
all of the access to the machines behind the botnet, and 80% had 
private IP addresses.  That's a high number because it tells us 
that the internet would be dead and buried if we didn't have NAT, 
but let's remember that it was created to fight address shortage.  

You know the old saying, when all you have is a hammer.  Yes, we 
started doing that because of address shortage, well it gives us 
security, well, not really.  Multi homing, kind of.  If we want 
those benefits, let's think outside the box.  Not do it the same 
way.  David was saying this.  People want to do things the same 
way as they're used to, but there's benefit to clean slate. 

There's a protocol that allows you you to do things in very 
different ways, apply security policies through the last 64 bits 
of the IP address.  You can do all a of this.  Try to think outside
the box.  Don't do it the same way and think of all the operational 
costs that's involved in having different scopes and different 
addresses.  

Finally, we use public IPv6 addresses internally.  And I can tell 
you, it's refreshingly simple to have one address.  It's just, you
just know what's going on.  And, all the security benefits that NAT
ostensibly has, personally, I don't buy them.  And I don't think 
they would be comparable even to the game that you have when you 
can actually understand things and have a clean simple design.  

(Jon Peterson)  That was not a question, but a contribution to the 
discussion?  

(Lorenzo Coletti) Open mic, right.  

(Jon Peterson) Certainly, if 80% of the hosts are behind NATS that 
tells something about the security that NATs grant.
  
(Erik Kline)  I'd like to add as well, could we possibly make a 
requirement that anybody who wants to implement IPv6 NAT actually 
run a reasonably large network for say, oh, a year.  Because I'm 
concerned about lots of things being done based off of experiments
and not valid requirements.  
   
(Jon Peterson) Well, we never did that with IPv4 and still they 
were developed.  

(Erik Kline) Right.  But everybody by then had several years of 
experience with IPv4 four, actual experience.  

(Jon Peterson)  I think that I'm going to disagree with that 
suggestion.  I think it's a dangerous thing, because I guarantee 
you, whoever puts that amount of effort in, will become committed 
to it.  

(Olaf Kolkman)  I'm going to carefully look around.  If there are
no further initiatives to move to the mic, and I don't see any, so,
now, again, very slowly, going, going, gone.  Thank you.