Network Working Group R. McGowan
Internet-Draft Unicode
Expires: September 25, 2003 March 27, 2003
About Unicode Consortium Procedures, Policies, Stability, and Public
Access
draft-rmcgowan-unicode-procs-02
Status of this Memo
This document is an Internet-Draft and is NOT offered in accordance
with Section 10 of RFC2026, and the author does not provide the IETF
with any rights other than to publish as an Internet-Draft.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at http://
www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 25, 2003.
Abstract
This memo describes various internal workings of the Unicode
Consortium for the benefit of participants in the IETF. It is
intended solely for informational purposes. Included are discussions
of how the decision-making bodies of the consortium work and what
their procedures are, as well as information on public access to the
character encoding & standardization processes.
McGowan Expires September 25, 2003 [Page 1]
Internet-Draft Unicode Consortium Procedures March 2003
1. About The Unicode Consortium
The Unicode Consortium is a corporation. Legally speaking it is a
"California Nonprofit Mutual Benefit Corporation", organized under
section 501 C(6) of the Internal Revenue Service Code. [see http://
www.irs.ustreas.gov/prod/bus_info/eo/bus-orgs.html]. As such, it is a
"business league" not focussed on profiting by sales or production of
goods and services, but neither is it formally a "charitable"
organization. It is an alliance of member companies whose purpose is
to "extend, maintain, and promote the Unicode Standard". To this end,
the consortium keeps a small office, a few editorial and technical
staff, World Wide Web presence, and mail-list presence.
The corporation is presided over by a Board of Directors who meet
annually. The Board is comprised of individuals who are elected
annually by the full members for three-year terms. The Board appoints
Officers of the corporation to run the daily operations.
Membership in the consortium is open to "all corporations, other
business entities, governmental agencies, not-for-profit
organizations and academic institutions" who support the consortium's
purpose. Formally, one class of voting membership is recognized, and
dues-paying members are typically for-profit corporations, research
and educational institutions, or national governments. Each such full
member sends representatives to meetings of the Unicode Technical
Committee (see below), as well as to a brief annual Membership
meeting.
McGowan Expires September 25, 2003 [Page 2]
Internet-Draft Unicode Consortium Procedures March 2003
2. The Unicode Technical Committee
The Unicode Technical Committee (UTC) is the technical decision
making body of the consortium. The UTC inherited the work and prior
decisions of the Unicode Working Group (UWG) that was active prior to
formation of the consortium in January 1991.
Formally, the UTC is a technical body instituted by resolution of the
board of directors. Each member appoints one principal and one or two
alternate representatives to the UTC. UTC representatives frequently
do, but need not, act as the ordinary member representatives for the
purposes of the annual meeting.
The UTC is presided over by a Chair and Vice-Chair, appointed by the
Board of Directors for an unspecified term of service.
The UTC meets 4 to 5 times a year to discuss proposals, additions,
and various other technical topics. There is no fee for participation
in the UTC meetings. Meeting agendas are not generally posted to any
public forum, but meeting dates, locations, and logistics are posted
well in advance at:
http://www.unicode.org/timesens/calendar.html
At the discretion of the UTC chair, meetings are open to
participation of member and liaison organizations, and to observation
by others. The minutes of meetings are posted publicly on the Unicode
Web site:
http://www.unicode.org/consortium/utc-minutes.html
Meetings of the UTC are held in North America, frequently in the San
Francisco Bay Area, where the majority of full members have offices.
Meetings typically last 3 to 4 full days. Rarely, a portion of a
meeting will be declared a "closed caucus" for member
representatives.
All UTC meetings are held jointly with INCITS Technical Committee L2,
the body responsible for Character Code standards in the United
States. They constitute "ad hoc" meetings of the L2 body. Further
information on L2 is available on the INCITS web page:
http://www.incits.org
McGowan Expires September 25, 2003 [Page 3]
Internet-Draft Unicode Consortium Procedures March 2003
3. Unicode Technical Committee Procedures
The formal procedures of the UTC are publicly available in a document
entitled "UTC Procedures" available from the Consortium, and on the
website:
http://www.unicode.org/consortium/utc-procedures.html
Despite the invocation of Robert's Rules of Order, UTC meetings are
conducted with relative informality in view of the highly technical
nature of most discussions. Meetings focus on items from a technical
agenda organized and published by the UTC Chair prior to the meeting.
Technical items are usually proposals in one of the following
categories:
1. Addition of new scripts
2. Addition of new characters or small batches of characters
3. Preparation and Editing of Technical Reports and Standards
4. Changes in the semantics of specific characters
5. Changes to the encoding architecture and forms of use
(Note: The last category of changes are, of course, rare and not
undertaken without significant lead time and involvement of
organizations, such as W3C and IETF, having standards that are likely
to be affected by such changes and with which the UTC maintains
liaisons. Changes that affect existing normalizations in particular
are normally disallowed according to UTC procedures, so that W3C and
IETF standards which rely on them are not thereby broken. See section
5 below.)
Typical outputs of the UTC are:
1. The Unicode Standard, major and minor versions
2. Unicode Technical Reports
3. Stand-alone Unicode Technical Standards
4. Formal resolutions
5. Liaison statements and instructions to the Unicode liaisons to
other organizations.
For each technical item on the meeting agenda, there is a general
McGowan Expires September 25, 2003 [Page 4]
Internet-Draft Unicode Consortium Procedures March 2003
process as follows:
1. Introduction by the topic sponsor
2. Proposals and discussion
3. Consensus statements or formal motions
McGowan Expires September 25, 2003 [Page 5]
Internet-Draft Unicode Consortium Procedures March 2003
4. Unicode Technical Committee Motions
Technical topics of any complexity never proceed from initial
proposal to final ratification or adoption into the standard in the
course of one UTC meeting. The UTC members and presiding officers are
aware that technical changes to the standard have broad consequences
to other standards, implementors, and end-users of the standard.
Input from other organizations and experts is often vital to the
understanding of various proposals and for successful adoption into
the standard.
Technical topics are decided in UTC through the use of formal
motions, either taken in meetings, or by means of 30-day letter
ballots. Formal UTC motions are of two types:
1. Simple motions
2. Precedents
Simple motions may pass with a simple majority constituting more than
50% of the qualified voting members; or by a special majority
constituting 2/3 or more of the qualified voting members.
Precedents are defined, according to the UTC Procedures as either
(A) an existing Unicode Policy, or
(B) an explicit precedent.
Prececents must be passed or overturned by a special majority.
Examples of implicit precedents include:
1. Publication of a character in the standard
2. Published normative character properties
3. Algorithms required for formal conformance
An Explicit Precedent is a policy, procedure, encoding, algorithm, or
other item that is established by a separate motion saying (in
effect) that a particular prior motion establishes a precedent.
The UTC also issues formal committee consensus statements, which are
tracked and listed like motions in the committee minutes. Consensus
statements are essentially equivalent to formal motions, except that
no formal vote is taken. When the committee members are in "broad
agreement" to a proposal but a vote is not required, a consensus
McGowan Expires September 25, 2003 [Page 6]
Internet-Draft Unicode Consortium Procedures March 2003
statement is usually issued. Such a consensus statement has the
weight of a motion as far as establishing precedents.
McGowan Expires September 25, 2003 [Page 7]
Internet-Draft Unicode Consortium Procedures March 2003
5. Unicode Consortium Policies
Because the Unicode Standard is continually evolving to approach the
ideal of encoding "all the world's scripts", new characters will
constantly be added. In this sense, the standard is unstable: in the
standard's useful lifetime, there may never be a final point at which
no more characters are added. Realizing this, the consortium has
adopted certain policies to promote and maintain stability of the
characters that are already encoded, as well as laying out a Roadmap
to future encodings.
The overall policies of the consortium with regard to encoding
stability are published on the web at this URL:
http://www.unicode.org/policies
Deliberations and encoding proposals in the UTC are bound by these
policies.
The general effect of the policies may be stated in this way: once a
character is encoded, it will not be moved or removed and its name
will not be changed. Any of those actions has the potential for
causing obsolescence of data, and they are not permitted. The
canonical combining class and decompositions of characters will not
be changed in any way that affects normalization. In this sense
normalization, such as that used for International Domain Naming and
"early normalization" for use on the World Wide Web, is fixed and
stable for every character at the time that character is encoded.
(Any changes that are undertaken because of outright errors in
properties or decompositions are dealt with by means of an adjunct
data file so that normalization stability can still be maintained by
those who need it.)
Property values of characters, such as directionality for the Unicode
Bidi algorithm, may be changed in some circumstances. As less-well
documented characters and scripts are encoded, the exact character
properties and behavior may not be well known at the time the
characters are first encoded. As more experience is gathered in
implementing the newly encoded characters, adjustments in the
properties may become necessary. This re-working is kept to a
minimum. New and old versions of the relevant property tables are
made available on the Consortium's web site.
Normative and some informative data about characters is kept in the
Unicode Character Database. The structure of many of these property
values will not be changed. Instead, when new properties are defined,
the consortium adds new files for these properties, so as not to
affect the stability of existing implementations that use the values
McGowan Expires September 25, 2003 [Page 8]
Internet-Draft Unicode Consortium Procedures March 2003
and properties defined in the existing formats and files. The latest
version of the Unicode Character Database can be reached from this
URL:
http://www.unicode.org/ucd
McGowan Expires September 25, 2003 [Page 9]
Internet-Draft Unicode Consortium Procedures March 2003
6. UTC and ISO (WG2 and WG20)
The character repertoire, names, and general architecture of the
Unicode Standard are identical to the parallel international standard
ISO/IEC 10646. Unicode provides additional properties and
implementation information that ISO/IEC 10646 does not.
Implementations conformant to Unicode are conformant to ISO/IEC
10646.
ISO/IEC 10646 is maintained by the committee ISO/IEC JTC1/SC2/WG2,
which maintains a web presence at:
http://www.dkuug.dk/jtc1/sc2/wg2/
The WG2 committee is composed of national body representatives to
ISO. Details of ISO organization may be found at:
http://www.iso.ch
Details and history of the relationship between ISO/IEC JTC1/SC2/WG2
and Unicode, Inc. may be found in Appendix C of The Unicode Standard.
(A PDF rendition of the most recent printed edition of the Unicode
Standard can be found on the Unicode web site.)
WG2 shares with UTC the policies regarding stability: WG2 neither
removes characters nor changes their names once published. Changes in
both standards are closely tracked by the respective committees, and
a very close working relationship is fostered to maintain
synchronization between the standards.
The Unicode Collation Algorithm (UCA) is one of a small set of other
independent standards defined and maintained by UTC. It is not,
properly speaking, part of the Unicode Standard itself, but is
separately defined in Unicode Technical Standard #10 (UTS #10).
There is no conformance relationship between the two standards,
except that conformance to a specific base version of the Unicode
Standard (e.g., 3.0) is specified in a particular version of a UTS.
The collation algorithm specified in UTS #10 is intended to remain in
conformance to ISO/IEC 14651 maintained by ISO/IEC JTC1/SC2/WG20, and
the two organizations maintain a close relationship.
McGowan Expires September 25, 2003 [Page 10]
Internet-Draft Unicode Consortium Procedures March 2003
7. Process of Technical Changes to the Unicode Standard
Changes to The Unicode Standard are of two types: architectural
changes, and character additions.
Some architectural changes may not affect ISO/IEC 10646, for example,
the addition of some informative properties to Unicode. Those
architectural changes that do affect both standards, such as
additional UTF formats or allocation of planes, are very carefully
coordinated by the committees. As always, on the UTC side,
architectural changes that establish precedents are carefully
monitored and the above-described rules and procedures are followed.
Additional characters for inclusion in the The Unicode Standard must
be approved both by the UTC and by WG2. Proposals for additional
characters enter the standards process in one of several ways:
through...
1. a national body member of WG2
2. a member company or associate of UTC
3. directly from an individual "expert" contributor
The two committees have jointly produced a "Proposal Summary Form"
that is required to accompany all additional character proposals. It
may be found online at:
http://www.dkuug.dk/JTC1/SC2/WG2/docs/form1.html
Instructions for submitting proposals to UTC may be found online at:
http://www.unicode.org/pending/proposals.html
Often, submission of proposals to both committees (UTC and WG2) is
simultaneous. Members of UTC also frequently forward to WG2 proposals
that have been initially reviewed by UTC.
In general, a proposal that is submitted to UTC before being
submitted to WG2 passes through several stages:
1. Initial presentation to UTC
2. Review and re-drafting
3. Forwarding to WG2 for consideration
4. Re-drafting for technical changes
McGowan Expires September 25, 2003 [Page 11]
Internet-Draft Unicode Consortium Procedures March 2003
5. Balloting for approval in UTC
6. Re-forwading and recommendation to WG2
7. At least two rounds of international balloting in ISO
About two years are required to complete this process. Proposals of
any type are almost never directly approved by UTC on first viewing,
but are usually sent back to the submitters. Repertoire addition
proposals that are submitted to WG2 before Unicode are generally
forwarded immediately to UTC through committee liaisons. The crucial
parts of the process (steps 5 through 7 above) are never
short-circuited. Two-thirds majority in UTC is required for approval
at step 5.
Proposals for additional scripts are required to be coordinated with
relevant user communities. Often there are ad-hoc subcommittees of
UTC or expert mail list participants who are responsible for actually
drafting proposals, garnering community support, or representing user
communities.
The rounds of international balloting (steps 7) have participation
both by UTC and WG2, though UTC does not directly vote in the ISO
process.
Occasionally a proposal approved by one body is considered too
immature for approval by the other body, and may be blocked de-facto
by either of the two. Only after both bodies have approved the
additional characters do they proceed to the rounds of international
balloting. (The first round is a draft international standard during
which some changes may occur, the second round is final approval
during which only editorial changes are made.)
This process assures that proposals for additional characters are
mature and stable by the time they appear in a final international
ballot.
McGowan Expires September 25, 2003 [Page 12]
Internet-Draft Unicode Consortium Procedures March 2003
8. Public Access to the Character Encoding Process
While Unicode, Inc, is a membership organization, and the final say
in technical matters rests with UTC, the process is quite open to
public input and scrutiny of processes and proposals. There are many
influential individual experts and industry groups who are not
formally members, but whose input to the process is taken seriously
by UTC.
Internally, UTC maintains a mail list called the "Unicore" list,
which carries traffic related to meetings, technical content of the
standard, and so forth. Members of the list are UTC representatives;
employees and staff of member organizations (such as the Research
Libraries Group); individual liaisons to and from other standards
bodies (such as WG2 and IETF); and invited experts from institutions
such as the Library of Congress and some universities. Subscription
to the list for external individuals is subject to "sponsorship" by
the corporate officers.
Unicode, Inc. also maintains a public discussion list called the
"Unicode" list. Subscription is open to anyone, and proceedings of
the "Unicode" mail list are made public via FTP on an occasional
basis. Details are located at:
http://www.unicode.org/unicode/consortium/distlist.html
All technical proposals for changes to the standard are posted to
both of these mail lists on a regular basis. Discussion on the public
list is also monitored by many members of UTC, and frequently the
results of these public discussions are brought up later at UTC
meetings. All technical issues and other standardization "events" of
any significance, such as beta releases and availablility of draft
documents, are announced and then discussed in this public forum,
well before standardization is finalized.
Anyone may make a character encoding or architectural proposal to
UTC. Membership in the organization is not required to submit a
proposal. To be taken seriously, the proposal must be framed in a
substantial way, and be accompanied by sufficient documentation to
warrant discussion. Examples of proposals are easily available by
following links from the "Proposed Characters" heading available at
the Unicode web site. The main proposal page is at:
http://www.unicode.org/unicode/alloc/Pipeline.html
(Guidelines for proposals are given at the web location mentioned in
the previous section.)
McGowan Expires September 25, 2003 [Page 13]
Internet-Draft Unicode Consortium Procedures March 2003
In general, proposals are publicly aired on the "Unicode" mail list,
sometimes for a long period, prior to formal submission. Generally
this is of benefit to the proposer as it tends to reduce the number
of times the proposal is sent back for clarification or with requests
for additional information. Once a proposal reaches the stage of
being ready for discussion by UTC, the proposer will have received
contact through the public mail list with one or more UTC members
willing to explain or defend it in a UTC meeting.
McGowan Expires September 25, 2003 [Page 14]