SMTP Extension for Internationalized Email
draft-ietf-eai-rfc5336bis-16

[Ballot comment]
Please consider this editorial comment from the Gen-ART Review by
  Pete McCann on 17-Oct-2011:

  Section 3.6:
    The message being sent via the MAIL command with the UTF8SMTPbis
    parameter has still a chance of that the message transmitted is not
    an internationalized message.
  SHOULD BE:
    There is still a chance that a message being sent via the MAIL
    command with the UTF8SMTPbis parameter is not an internationalized
    message.

Upon approval of this document, IANA will perform the following actions:

ACTION 1:

IANA will register the following in the SMTP Service Extensions at
http://www.iana.org/assignments/mail-parameters:

UTF8SMTPbis | Internationalized email address | [RFCXXXX]


ACTION 2:

IANA will update the references for the following Enumerated Status
Codes at http://www.iana.org/assignments/smtp-enhanced-status-codes:

Code:      X.6.7
Sample Text:    non-ASCII addresses not permitted
      for that sender/recipient
Associated basic status code: 550, 553
Description:    This indicates the reception of a MAIL or RCPT
          command that non-ASCII addresses are not permitted
Defined:      RFC XXXX (Standard track)
Submitter:    Jiankang YAO
Change controller: ima@ietf.org

Code:      X.6.8
Sample Text:    UTF-8 string reply is required,
          but not permitted by the SMTP client
Associated basic status code: 252, 550, 553
Description:    This indicates that a reply containing a UTF-8
          string is required to show the mailbox name,
but that form of response is not
permitted by the SMTP client.
Defined:      RFC XXXX (Standard track)
Submitter:    Jiankang YAO
Change controller: ima@ietf.org

Code:      X.6.9
Sample Text:    UTF-8 header message can not be transferred
          to one or more recipient so the message
must be rejected
Associated basic status code: 550
Description:    This indicates that transaction failed
      after the final "." of the DATA command.
Defined:      RFC XXXX (Standard track)
Submitter:    Jiankang YAO
Change controller: ima@ietf.org

Code:      X.6.10
Description:    This is a duplicate of X.6.8 and
      is thus deprecated.


ACTION 3:

IANA will add or update the following registrations in the WITH protocol
types registry at http://www.iana.org/assignments/mail-parameters:

| UTF8SMTP    | ESMTP with UTF8SMTP          | [RFCXXXX]
| UTF8SMTPA    | ESMTP with UTF8SMTP and SMTP  | [RFC4954]
|              | AUTH                          | [RFCXXXX]
| UTF8SMTPS    | ESMTP with UTF8SMTP and      | [RFC3207]
|              | STARTTLS                      | [RFCXXXX]
| UTF8SMTPSA  | ESMTP with UTF8SMTP and both  | [RFC3207]
|              | STARTTLS and SMTP AUTH        | [RFC4954]
|              |                              | [RFCXXXX]
| UTF8LMTP    | LMTP with UTF8SMTP            | [RFCXXXX]
| UTF8LMTPA    | LMTP with UTF8SMTP and SMTP  | [RFC4954
|              | AUTH                          | [RFCXXXX]
| UTF8LMTPS    | LMTP with UTF8SMTP and        | [RFC3207]
|              | STARTTLS                      | [RFCXXXX]
| UTF8LMTPSA  | LMTP with UTF8SMTP and both  | [RFC3207]
|              | STARTTLS and LMTP AUTH        | [RFC4954] 
|              |                              | [RFCXXXX]

[Ballot comment]
It would be helpful to cite RFC 3629 in Section 1.1.

It is nice that Section 1.1 says "this specification replaces an earlier, experimental, approach to the same problem [RFC5336]." It would be even nicer to describe the results of the experiment and the resulting protocol differences (probably in an Appendix).

Section 3.2 states: "Any domain name to be looked up in the DNS MUST allow for [RFC5890] behavior." I don't understand what this means, and I find the use of the passive voice confusing here. Does this mean than an SMTP server advertising support for the UTF8SMTPbis extension MUST accept DNS domain names that conform to RFC 5890? If so, let's say that directly.

Section 3.2 states: "it may choose its own way to deal with this scenario according to the provisions of [RFC4409] or its future versions." Do we mean "MAY"?

Section 3.5 states: "A UTF8SMTPbis-aware SMTP client MUST only send an internationalized message to an SMTP server that supports UTF8SMTPbis." I think it would be clearer to say "A UTF8SMTPbis-aware SMTP client MUST NOT send an internationalized message to an SMTP server that does not support UTF8SMTPbis."

Section 5 states: "Another security aspect to be considered is related to the ability by security team members to quickly understand, read and identify email addresses from the logs, when they are tracking an incident." Because reading and understanding an email address would require the person to be capable of reading the script and understanding the language, I think "identify" is all we can hope to promise here (and even that is unlikely in some situations given the existence of confusable characters).

[Ballot comment]
When the changes section gets removed, the hint of what happened to the appendix in 5336 that updated 4952 goes with it. Would it be easy to add a short note somewhere letting the reader know to go look for that in 4952bis?

[Ballot discuss]
As Pete notes in the tracker, the document currently has downrefs for 2033 and 4952bis. To keep these, 3967 currently requires explicitly calling these out as part of Last Call, which I don't think has happened.

[Ballot comment]
It would be nice to condense Section 7 into "Changes from RFC 5336" and
retain it in the document.

---

I could not parse the Note in Section 1 well enough to understand where
the keyword to replace "UTF8SMTPbis" will actually be defined. It is
possible that [RFC5336bis-SMTP] is the intended document, however, it
is not mentioned anywhere in this I-D.

A way to handle this might be to put in some real (i.e. not a note to
be removed) text that makes a normative reference. Such as:
  The keyword UTF8SMTPbis is defined in [RFC5336bis-SMTP].
You would also need to add a normative reference to [RFC5336bis-SMTP]
When the note is acted on, this text will become correct.

I am hoping that the Responsible AD understands this issue well enough
to explain it to the RFC Editor.

[Ballot comment]
#1) Do the authors also wish to make RFC 5336 Historic?

#2) Please add a section that lists the difference between RFC 5336 and this document.

[Ballot discuss]
#1) s1.2 is titled "Updates to Other Specifications".  This pretty strongly implies that an "Updates:" should appear on the document's first page.  Just trying to understand why this isn't the case.

[Ballot comment]
Editorial comment that should be taken care of with the rest of Last Call comments: In 3.2 and 3.6, references to 2045 and 2047 are sometimes just poorly worded and sometimes incorrect. For example, in 3.2:

  If the UTF8SMTPbis SMTP extension is not offered by the SMTP server,
  the UTF8SMTPbis-aware SMTP client MUST NOT transmit an
  internationalized email address and MUST NOT transmit a mail message
  containing internationalized mail headers as described in
  [RFC5335bis] at any level within its MIME structure [RFC2045] and
  [RFC2047].

The last bit should simply be "within its MIME [RFC2045] structure". There should be no reference to 2047 here. See also section 3.6.

  (1.a) Who is the Document Shepherd for this document? Has the
        Document Shepherd personally reviewed this version of the
        document and, in particular, does he or she believe this
        version is ready for forwarding to the IESG for publication?

Joseph Yee (jyee@afilias.info)
Yes, I reviewed the draft.
I believe this version is ready for IESG for publication.


  (1.b) Has the document had adequate review both from key WG members
        and from key non-WG members? Does the Document Shepherd have
        any concerns about the depth or breadth of the reviews that
        have been performed?

Yes, many WG members reviewed and commented about this draft, and
external reviews were conducted in Nov 2010.
I do not have any concerns about the depth or breadth of the reviews.
 
  (1.c) Does the Document Shepherd have concerns that the document
        needs more review from a particular or broader perspective,
        e.g., security, operational complexity, someone familiar with
        AAA, internationalization or XML?

No.

  (1.d) Does the Document Shepherd have any specific concerns or
        issues with this document that the Responsible Area Director
        and/or the IESG should be aware of? For example, perhaps he
        or she is uncomfortable with certain parts of the document, or
        has concerns whether there really is a need for it. In any
        event, if the WG has discussed those issues and has indicated
        that it still wishes to advance the document, detail those
        concerns here. Has an IPR disclosure related to this document
        been filed? If so, please include a reference to the
        disclosure and summarize the WG discussion and conclusion on
        this issue.

No.  Other than the keyword 'UTF8SMTPbis' is a placeholder now.
That keyword affects this document and several others from this
WG.  It will need to be assigned after document approval by a
process to be worked out between the responsible AD, WG
co-chairs, and IANA.  Assignment of the keyword has been deferred
until the documents are approved in order to prevent preliminary
implementations that may not reflect all properties of the final,
consensus specification from using it, thereby creating
interoperability problems.

  (1.e) How solid is the WG consensus behind this document? Does it
        represent the strong concurrence of a few individuals, with
        others being silent, or does the WG as a whole understand and
        agree with it? 

There were extensive reviews from both WG members and external
members in last 12 months.  Many concerns and issued have been
raised; the responses and solution reached WG rough consensus. 

  (1.f) Has anyone threatened an appeal or otherwise indicated extreme
        discontent? If so, please summarise the areas of conflict in
        separate email messages to the Responsible Area Director. (It
        should be in a separate email because this questionnaire is
        entered into the ID Tracker.)

No.  All potential issues appear to have been raised and WG
consensus reached. There are some who are not happy with all of
the details, but there has been no evidence of extreme discontent.

  (1.g) Has the Document Shepherd personally verified that the
        document satisfies all ID nits? (See the Internet-Drafts Checklist
        and http://tools.ietf.org/tools/idnits/). Boilerplate checks are
        not enough; this check needs to be thorough. Has the document
        met all formal review criteria it needs to, such as the MIB
        Doctor, media type and URI type reviews?

Yes, I verified the document with ID-Nits tool.

There are 4 downward normative references.  Three of them
    (RFC4952bis, RFC5335bis, RFC5337bis) will be resolved once these
documents are approved and published.  (RFC4952bis is already in
the RFC Editor queue, having been approved by the IESG slightly
over a year ago.)  The last one, RFC2033, is necessary because
of the four new LMTP-related keywords defined in the IANA
section; readers need to understand the LMTP protocol to use the
keywords properly with the extension specified in this document;
it is irrelevant if LMTP is not being used.  And knowledge of LMTP
is not required to use the protocol specified in this document, so
it is questionable whether the reference should actually be
considered normative at all.

There is one warning about [RFCXXX] in IANA table section, due to
formatting, hence it's not an issue. 

The tool also warns of about two instances of
non-RFC5735-compliant IPv4 addresses.  There are no IPv4
addresses in this document, the two instance are section number
'3.7.4.1' and '3.7.4.2'. 

The tool comments about disclaimer for pre-RFC5378 work.  This
document carries -bis after 10 Nov 2008, but the document cites and
extends ABNF rules from RFC5321.


The tool also comments that the abstract doesn't seem to
mention obsoleting RFC5336, but the document does.


  (1.h) Has the document split its references into normative and
        informative? Are there normative references to documents that
        are not ready for advancement or are otherwise in an unclear
        state? If such normative references exist, what is the
        strategy for their completion? Are there normative references
        that are downward references, as described in [RFC3967]? If
        so, list these downward references to support the Area
        Director in the Last Call procedure for them [RFC3967].

Yes.  Details documented in response (1.g)

The reference of RFC4409 should be replaced by
draft-ietf-yam-rfc4409bis if that document is published before
this one.  A XML comment was noted for RFC Editor.



  (1.i) Has the Document Shepherd verified that the document IANA
        consideration section exists and is consistent with the body
        of the document? If the document specifies protocol
        extensions, are reservations requested in appropriate IANA
        registries? Are the IANA registries clearly identified? If
        the document creates a new registry, does it define the
        proposed initial contents of the registry and an allocation
        procedure for future registrations? Does it suggest a
        reasonable name for the new registry? See [RFC5226]. If the
        document describes an Expert Review process has Shepherd
        conferred with the Responsible Area Director so that the IESG
        can appoint the needed Expert during the IESG Evaluation?

Yes, I verified that the IANA consideration section exists and is
consistent with the body of the document.


  (1.j) Has the Document Shepherd verified that sections of the
        document that are written in a formal language, such as XML
        code, BNF rules, MIB definitions, etc., validate correctly in
        an automated checker?

Yes, I verified the BNF rules in this document.

  (1.k) The IESG approval announcement includes a Document
        Announcement Write-Up. Please provide such a Document
        Announcement Write-Up? Recent examples can be found in the
        "Action" announcements for approved documents. The approval
        announcement contains the following sections:
    Technical Summary
        Relevant content can frequently be found in the abstract
        and/or introduction of the document. If not, this may be
        an indication that there are deficiencies in the abstract
        or introduction.

This document specifies an SMTP extension for transport and delivery
  of email messages with internationalized email addresses or header
  information.

    Working Group Summary
        Was there anything in WG process that is worth noting? For
        example, was there controversy about particular points or
        were there decisions where the consensus was particularly
        rough?

There were many discussions but none of the consensuses were
tough to reach nor had tough resistance from the consensus.

    Document Quality
        Are there existing implementations of the protocol? Have a
        significant number of vendors indicated their plan to
        implement the specification? Are there any reviewers that
        merit special mention as having done a thorough review,
        e.g., one that resulted in important changes or a
        conclusion that the document had no substantive issues? If
        there was a MIB Doctor, Media Type or other expert review,
        what was its course (briefly)? In the case of a Media Type
        review, on what date was the request posted?


This document received lots of attentions and reviews since
Nov/Dec 2010 and is in very good shape.  As mentioned in
Acknowledgement section, many thanks to Dave Crocker's suggestions
that led to refinements in the ABNF. 

State changed to In Last Call from Last Call Requested.

The following Last Call Announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Subject: Last Call:  (SMTP Extension for Internationalized Email) to Proposed Standard


The IESG has received a request from the Email Address
Internationalization WG (eai) to consider the following document:
- 'SMTP Extension for Internationalized Email'
  as a Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2011-10-20. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  This document specifies an SMTP extension for transport and delivery
  of email messages with internationalized email addresses or header
  information.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-eai-rfc5336bis/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-eai-rfc5336bis/


No IPR declarations have been submitted directly on this I-D.

Most of my AD review comments for -04 were addressed.

Updated AD review as per -05. (I still need to check the ABNF changes, I will do that separately.)

4.  IANA Considerations

  +---------------+-----------------------------+---------------------+
  | WITH protocol | Description                | Reference          |
  | types        |                            |                    |
  +---------------+-----------------------------+---------------------+
  | UTF8SMTPbis  | UTF8SMTPbis with Service    | [RFCXXXX]          |
  |              | Extensions                  |                    |
  | UTF8SMTPbisA  | UTF8SMTPbis with SMTP AUTH  | [RFC4954] [RFCXXXX] |
  | UTF8SMTPbisS  | UTF8SMTPbis with STARTTLS  | [RFC3207] [RFCXXXX] |
  | UTF8SMTPbisSA | UTF8SMTPbis with both      | [RFC3207] [RFC4954] |
  |              | STARTTLS and SMTP AUTH      | [RFCXXXX]          |
  +---------------+-----------------------------+---------------------+

Do we really need to change the WITH protocol types?

Also need to add the relevant LMTP WITH protocol types.


X.6.8 and Y.6.8 Enhanced Status codes registrations need to be compressed into 1,
X.6.10 needs to be marked as obsolete in the Enhanced Status Code registry

AD review of -04 (In addition to ABNF errors I've spotted).

2.  Overview of Operation

  This specification describes an optional extension to the email
  transport mechanism that permits non-ASCII [ASCII] characters in both
  the envelope and header fields of messages, which are encoded with
  UTF-8 [RFC3629] characters.  The extension is identified with the
  token "UTF8SMTPbis".  In order to provide information that may be
  needed in downgrading, an optional alternate ASCII address may be
  needed if an SMTP client attempts to transfer an internationalized
  message and encounters a server that does not support this extension.

I think the last sentence no longer applies and need to be deleted.

In 3.1:

  9.  The UTF8SMTPbis extension is valid on the submission port
      [RFC4409].

I think this sentence should also say that this extension can be used with LMTP.
This will make the reference to RFC 2033 Normative.


In 3.2:

  A UTF8SMTPbis aware MUA/MSA sending to a legacy SMTP server [RFC5321]
  and [RFC5322] MAY convert the ASCII@non-ASCII

Native speakers: should the "the" be replaced with "a" here?

  address into the format
  of ASCII@A-label [RFC5890] if the email address is in the format of
  ASCII@non-ASCII.



3.4.  UTF8 addresses and Response Codes

  An "internationalized message" as defined in the appendix of this
  specification

I think the part that reads "as defined in the appendix of this specification"
needs to be removed, because the definition is already in another document
and this document no longer has any appendix.

  MUST NOT be sent to an SMTP server that does not
  support UTF8SMTPbis.  Such a message should be rejected by a server
  if it lacks the support of UTF8SMTPbis.

3.6.2.  Mail eXchangers

  Organizations often authorize multiple servers to accept mail
  addressed to them.  For example, the organization may itself operate
  more than one server, and may also or instead have an agreement with
  other organizations to accept mail as a backup.  Authorized servers
  are generally listed in MX records as described in RFC 5321.  When
  more than one server accepts mail for the domain-part of a mailbox,
  it is strongly advised that either all or none of them support the
  UTF8SMTPbis extension.  Otherwise, surprising downgrades can happen
  during temporary failures, which users might perceive as a serious
  reliability issue.

The last sentence: I think this needs to be reworded or deleted,
as "surprising downgrades" are no longer an issue.


3.6.4.2.  VRFY and EXPN Commands and the UTF8REPLY Parameter

  VERIFY (VRFY) and EXPAND (EXPN) command syntaxes are changed to:

      "VRFY" SP ( uLocal-part / uMailbox ) [ SP "UTF8REPLY" ] CRLF
              ; uLocal-part and uMailbox are defined in
              ; Section 3.3 of this document.

Section 3.3 no longer contains definitions of uLocal-part and uMailbox.
See my separate email message about ABNF errors in rfc5335bis/rfc5336bis.

Also note that RFC 5321 is using:

      vrfy = "VRFY" SP String CRLF

So I suggest using the same format (with "vrfy =") for consistency.


      "EXPN" SP ( uLocal-part / uMailbox ) [ SP "UTF8REPLY" ] CRLF
              ; uLocal-part and uMailbox are defined in
              ; Section 3.3 of this document.

As above.
Similarly, I suggest using:

      expn = "EXPN" SP ( uLocal-part / uMailbox ) [ SP "UTF8REPLY" ] CRLF


  If the SMTP reply requires UTF-8 strings, but UTF-8 is not allowed in
  the reply, and the server supports enhanced mail system status codes
  [RFC3463], the enhanced response code is either "X.6.8" or "X.6.10"

Can somebody remind me why we have 2 different Enhanced Status Code which
mean the same thing?

  [RFC5248], meaning "A reply containing a UTF-8 string is required to
  show the mailbox name, but that form of response is not permitted by
  the client".


4.  IANA Considerations

  +---------------+-----------------------------+---------------------+
  | WITH protocol | Description                | Reference          |
  | types        |                            |                    |
  +---------------+-----------------------------+---------------------+
  | UTF8SMTPbis  | UTF8SMTPbis with Service    | [RFCXXXX]          |
  |              | Extensions                  |                    |
  | UTF8SMTPbisA  | UTF8SMTPbis with SMTP AUTH  | [RFC4954] [RFCXXXX] |
  | UTF8SMTPbisS  | UTF8SMTPbis with STARTTLS  | [RFC3207] [RFCXXXX] |
  | UTF8SMTPbisSA | UTF8SMTPbis with both      | [RFC3207] [RFC4954] |
  |              | STARTTLS and SMTP AUTH      | [RFCXXXX]          |
  +---------------+-----------------------------+---------------------+

Do we really need to change the WITH protocol types?



  [RFC2033]  Myers, J., "Local Mail Transfer Protocol", RFC 2033,
              October 1996.

Make this reference Normative per my earlier comment.