Post-Delivery Message Downgrading for Internationalized Email Messages
RFC 6857

[Ballot comment]
While the writeup mentions:

      Consequently the base IMAP and POP3
      documents are no longer dependent on particular downgrading
      choices and that two methods presented are, to a considerable
      extent, just examples.

I believe that the two methods should be Informational, as opposed to Standards Track.

However, I now see the following sentence, which was essential to me:
    While this document specifies a well designed mechanism, it is only
  an interim solution while clients are being upgraded
  [I-D.ietf-eai-rfc5721bis] [I-D.ietf-eai-5738bis].

So I'll clear my DISCUSS.

Regards, Benoit

[Ballot comment]
Consider reinforcing in the security considerations section that the actions described by this document do not include removing any signatures from the original message - discouraging a server implementation from trying to be 'helpful' by removing a signature they know will fail.

[Ballot discuss]

  Why is this document on the standards track?  I would expect the
  standards track if the other documents in this cluster required this
  document to be followed if the optional downgrade capability is
  implemented.  However, that is not the situation.  Benoit Claise
  seems to have the same concern.

[Ballot comment]

- Should the security considerations here have a MUST or SHOULD
statement calling for the server to strip existing Downgraded-*
header fields? If not, why not?

- Would it be worthwhile having a PDF version of this document
that contained examples that show actual non-ASCII characters in
appendix A? If so, then pointing to that in the ASCII appendix A
would be a good thing too.

[Ballot discuss]
The writeup mentions:

      Consequently the base IMAP and POP3
      documents are no longer dependent on particular downgrading
      choices and that two methods presented are, to a considerable
      extent, just examples.

If these are just examples, why are they standards tracks? They should be informational.
From RFC 2026:

4.2.2  Informational

  An "Informational" specification is published for the general
  information of the Internet community, and does not represent an
  Internet community consensus or recommendation.


Along the same lines, quoting Barry:

    The only good answer to this issue (having a client that doesn't support EAI using a server that does and has an "EAI message" to present to the client) is to get the client upgraded.  *Anything* else (see Pete's parenthesized list above) is bad in some regard, and the decision of how to handle the situation until an upgrade can happen
    simply has to be at the discretion of the server

Please mention in the draft (maybe in section 1.3, after the first paragraph), something such as: While this document is a specification, the only good solution is to upgrade the client [RFC5721bis] [RFC5738bis]

My justification is the following: yes, this is a spec, but we don't want to promote it, we just want the clients to be upgraded.

Note: this is done in https://datatracker.ietf.org/doc/draft-ietf-eai-simpledowngrade/

  This document specifies a way to present such messages to the client.
  It values simplicity of implementation over fidelity of
  representation, since implementing a high-fidelity downgrade
  algorithm is likely more work than implementing proper support for
  [RFC5721] and/or [RFC5738].

Btw, should be RFC 5738bis and RFC5721bis.

IANA has reviewed draft-ietf-eai-popimap-downgrade-07 and has the following comments:

IANA has questions about the IANA actions requested in this document.

IANA understands that, upon approval of this document, there is a single
IANA action that needs to be completed.

This document requests adding five message header field names to
the Permanent Message Header Field Names registry located at:

http://www.iana.org/assignments/message-headers/perm-headers.html

The five message header field names to be added are:

Header field name: Downgraded-Message-Id
Applicable protocol: mail
Status: standard
Specification document(s): [ RFC-to-be ]

Header field name: Downgraded-In-Reply-To
Applicable protocol: mail
Status: standard
Specification document(s): [ RFC-to-be ]

Header field name: Downgraded-References
Applicable protocol: mail
Status: standard
Specification document(s): [ RFC-to-be ]

Header field name: Downgraded-Original-Recipient
Applicable protocol: mail
Status: standard
Specification document(s): [ RFC-to-be ]

Header field name: Downgraded-Final-Recipient
Applicable protocol: mail
Status: standard
Specification document(s): [ RFC-to-be ]

Currently the Permanent Message Header Field Names registry is maintained
through expert review as defined in RFC 5226.

IANA Question -> has the document been reviewed by the Permanent Message Header
Field Names registry expert?

Note: The actions requested in this document will not be completed until
the document has been approved for publication as an RFC.

The following Last Call announcement was sent out:

From: The IESG
To: IETF-Announce
CC:
Reply-To: ietf@ietf.org
Subject: Last Call:  (Post-delivery Message Downgrading for Internationalized Email Messages) to Proposed Standard


The IESG has received a request from the Email Address
Internationalization WG (eai) to consider the following document:
- 'Post-delivery Message Downgrading for Internationalized Email
Messages'
  as Proposed Standard

Please note: This document is one a set of four interdependent
documents:

draft-ietf-eai-5738bis
draft-ietf-eai-popimap-downgrade
draft-ietf-eai-rfc5721bis
draft-ietf-eai-simpledowngrade

These documents should be reviewed, evaluated, and understood
together.

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action. Please send substantive comments to the
ietf@ietf.org mailing lists by 2012-09-20. Exceptionally, comments may be
sent to iesg@ietf.org instead. In either case, please retain the
beginning of the Subject line to allow automated sorting.

Abstract


  The Email Address Internationalization (SMTPUTF8) extension to SMTP
  allows UTF-8 characters in mail header fields.  Upgraded POP and IMAP
  servers support internationalized Email messages.  If a POP/IMAP
  client does not support Email Address Internationalization, POP/IMAP
  servers cannot deliver Internationalized Email Headers to the client
  and cannot remove the message.  To avoid the situation, this document
  describes a conversion mechanism for internationalized Email messages
  to be in traditional message format.  In the process, message
  elements requiring internationalized treatment are recoded or removed
  and receivers are able to know that they received messages containing
  such elements even if they cannot process the internationalized
  elements.




The file can be obtained via
http://datatracker.ietf.org/doc/draft-ietf-eai-popimap-downgrade/

IESG discussion can be tracked via
http://datatracker.ietf.org/doc/draft-ietf-eai-popimap-downgrade/ballot/


No IPR declarations have been submitted directly on this I-D.

    Document Shepherd Writeup - draft-ietf-eai-popimap-downgrade-07



      Date: 2012-08-22

      Shepherd: John C Klensin, john-ietf@jck.com



1.

  What type of RFC is being requested (BCP, Proposed Standard, Internet
  Standard, Informational, Experimental, or Historic)?  Why is this the
  proper type of RFC?  Is this type of RFC indicated in the title page
  header?

  Proposed Standard.  This set of documents are all protocol
  specifications, following up earlier Experimental treatment of POP3
  and IMAP access to messages with internationalized envelopes and/or
  header fields.

  Questions were raised within the WG as to whether these two documents
  (draft-ietf-eai-popimap-downgrade and draft-ietf-eai-simpledowngrade)
  should be published as Proposed Standards or Informational.  The WG
  concluded that Proposed Standard was more appropriate: they do
  specify protocol elements and formats, are well-understood (the
  former draws heavily on the experimental EAI specifications and the
  associated testing and the latter has been implemented) and that the
  appropriate place for any needed statements about application to
  particular cases belonged in a separate Applicability Statement, not
  in the document classification.  However, if the IESG decides that
  Informational would be more appropriate, there is probably
  insufficient energy or consensus in the WG to strongly oppose that
  position (on the other hand, a few participants might insist that the
  IESG explain such a decision in a way that can be generalized to
  future work in the IETF).


2.

  The IESG approval announcement includes a Document Announcement
  Write-Up.  Please provide such a Document Announcement Write-Up.
  Recent examples can be found in the "Action" announcements for
  approved documents.  The approval announcement contains the following
  sections:

  Technical Summary
      Relevant content can frequently be found in the abstract and/or
      introduction of the document.  If not, this may be an indication
      that there are deficiencies in the abstract or introduction.

      These documents make up a set of four that are interdependent and
      should be reviewed, evaluated, and understood together.  Their
      abstracts have been examined and verified to sufficiency to
      describe the individual documents.

      The abstract for this particular document reads:

        The Email Address Internationalization (SMTPUTF8) extension to
        SMTP allows UTF-8 characters in mail header fields.  Upgraded
        POP and IMAP servers support internationalized Email messages.
        If a POP/IMAP client does not support Email Address
        Internationalization, POP/IMAP servers cannot deliver
        Internationalized Email Headers to the client and cannot remove
        the message.  To avoid the situation, this document describes a
        conversion mechanism for internationalized Email messages to be
        in traditional message format.  In the process, message
        elements requiring internationalized treatment are recoded or
        removed and receivers are able to know that they received
        messages containing such elements even if they cannot process
        the internationalized elements.

  Working Group Summary
      Was there anything in WG process that is worth noting?  For
      example, was there controversy about particular points or were
      there decisions where the consensus was particularly rough?

      Short answer: No.  Longer answer: The WG had extensive and
      constructive discussions about the role of "downgrading" (e.g.,
      converting a message stored on the server that contains non-ASCII
      header or envelope information) in the transition to an all-i18n
      environment.  Some of those issues and tradeoffs are discussed in
      draft-ietf-eai-popimap-downgrade and
      draft-ietf-eai-simpledowngrade.  In some cases, the best strategy
      may be to "hide" those messages that cannot be delivered without
      change to legacy clients either with or without some attempt at an
      error message.  A complete treatment of those options is
      impossible because the optimal strategies will depend considerably
      on local circumstances.  Consequently the base IMAP and POP3
      documents are no longer dependent on particular downgrading
      choices and that two methods presented are, to a considerable
      extent, just examples.  They are recommended as alternative
      Standards Track documents because they are protocol specifications
      and their sometimes-subtle details have have been carefully worked
      out, even though the WG has no general recommendation to make
      between them (or other strategies).

      While opinions differ in the WG about which downgrading mechanisms
      are likely to see the most use, if any, consensus is strong that
      these four documents represent the correct output.

  Document Quality
      Are there existing implementations of the protocol?  Have a
      significant number of vendors indicated their plan to implement
      the specification?  Are there any reviewers that merit special
      mention as having done a thorough review, e.g., one that resulted
      in important changes or a conclusion that the document had no
      substantive issues?  If there was a MIB Doctor, Media Type or
      other expert review, what was its course (briefly)?  In the case
      of a Media Type review, on what date was the request posted?

      Some development and interoperability testing has occurred and is
      progressing.  There are strong commitments in various countries to
      implement and deploy the EAI (more properly, SMTPUTF8) messages
      and functions specified in RFCs 6530 through 6533.  Those messages
      will be inaccessible to many users without POP3 and IMAP support,
      so these specifications are quite likely to be implemented and
      deployed in a timely fashion.

      Reviewers who made particular contributions prior to IETF Last
      Call are acknowledged in the documents.  See Section 3 for
      additional information.

  Personnel
      Who is the Document Shepherd?  Who is the Responsible Area
      Director?



      Document Shepherd:  John C Klensin

      Responsible Area Director:  Pete Resnick

        Note that Pete Resnick is listed as a co-author on one of these
        documents as a result of contributions well before he became AD
        (and primarily to its the Experimental predecessor.  He has not
        been actively involved in an author or editor role since
        joining the IESG.


3.

  Briefly describe the review of this document that was performed by
  the Document Shepherd.  If this version of the document is not ready
  for publication, please explain why the document is being forwarded
  to the IESG.

  The document shepherd, with assistance from the other co-chair, did
  extensive, line by line and paragraph by paragraph reviews during the
  WG LC window with the intention of identifying and eliminating as
  many issues that might otherwise be spotted during IETF review as
  possible.  Those reviews were posted to the WG mailing list; the
  documents being submitted include changes made on that basis.


4.

  Does the document Shepherd have any concerns about the depth or
  breadth of the reviews that have been performed?

  This particular document shepherd has almost always been concerned
  about breadth and quality of reviews in the IETF.  HOwever, the co-
  chairs have identified areas of expertise and perspective needed for
  reviews of the specifications in these documents and their
  relationship to the widely-deployed and well-tested IMAP and POP3
  specifications and are confident that the reviews are adequate.

  Although considerable improvements have been made in readability and
  editorial and technical quality, the base IMAP
  (draft-ietf-eai-5738bis) and POP (draft-ietf-eai-rfc5721bis)
  documents represent an orderly and uncontroversial evolution from
  their Experimental predecessors.

  It is probably worth pointing out, as draft-ietf-eai-5738bis does,
  that the transition to general adoption of SMTPUTF8 mail will not be
  an easy one in many environments.  In the case of the transport and
  mail header specifications of RFCs 6530ff, the model that permits a
  sender to test whether the potential receiver can handle the message
  is clear, as is an orderly response if it is not (even if that
  response may not be completely satisfactory from the user's
  standpoint).  That same relationship does not apply to these
  specifications because, for many environments, a POP3 or IMAP server
  must be prepared to deal with clients who do not have the needed
  capabilities and there is no completely satisfactory way with either
  protocol to either tell a client that it cannot access a message that
  is known to be waiting nor to deliver an intact version of the
  message (where "intact" includes, e.g., being able to pass signature
  verification on body parts and/or headers).


5.

  Do portions of the document need review from a particular or from
  broader perspective, e.g., security, operational complexity, AAA,
  DNS, DHCP, XML, or internationalization?  If so, describe the review
  that took place.

  It is our strong belief that all such issues and perspectives have
  been addressed by the WG and reviews already obtained.


6.

  Describe any specific concerns or issues that the Document Shepherd
  has with this document that the Responsible Area Director and/or the
  IESG should be aware of?  For example, perhaps he or she is
  uncomfortable with certain parts of the document, or has concerns
  whether there really is a need for it.  In any event, if the WG has
  discussed those issues and has indicated that it still wishes to
  advance the document, detail those concerns here.

  All such substantive issues have been identified and resolved within
  the WG and incorporated into the documents.  I do expect that IETF
  Last Call will turn up demands to solve problems that the WG has
  concluded are impossible (some discussed above) but it is unlikely
  that any will be a significant surprise.

  It is worth noting that the WG believes that there are a relatively
  large number of potential approaches to the situation in which a
  message that required SMTPUTF8 capability appears in a mail store but
  a POP or IMAP client is expected to deliver that message to a legacy
  client with no such capability.  The WG has strong consensus that
  none of those approaches are fully satisfactory (and says so in
  draft-ietf-eai-5738bis) but that these two approaches are worth
  describing in detail.  Most, or all, of the other approaches
  concentrate on administrative arrangements rather than on message
  modification.  The WG does not believe it is appropriate to try to
  make a case analysis of appropriateness of different approaches and
  different situations, at least until considerable additional
  experience has been accumulated.

  This document contains a normative dependency on
  draft-leiba-5322upd-from-group which authorizes the use of "Group"
  syntax in the "From:" mail header field.  The WG is aware of that
  dependency and considers deferring publication of this document until
  after that is approved to be acceptable, but hopes it can be
  processed as soon as reasonably possible.


7.

  Has each author confirmed that any and all appropriate IPR
  disclosures required for full conformance with the provisions of BCP
  78 and BCP 79 have already been filed.  If not, explain why.

  Authors of all four documents have been queried to verify that they
  have examined BCP 78 and 79 and are in compliance with them.  The
  three authors who have not yet replied are expected to be in
  Vancouver and acknowledgments will be extracted from them there.


8.

  Has an IPR disclosure been filed that references this document?  If
  so, summarize any WG discussion and conclusion regarding the IPR
  disclosures.

  No IPR disclosures have been filed on any of the four documents.


9.

  How solid is the WG consensus behind this document?  Does it
  represent the strong concurrence of a few individuals, with others
  being silent, or does the WG as a whole understand and agree with it?

  The WG's meeting participants and mailing list have included a rather
  large proportion of people who are anxious to see well-defined
  standards in this area agreed upon and deployed, but who behave as if
  they have little interest or expertise in the details of the
  technology (some of them are probably correct about the latter).  Of
  those who have participated technically and more actively, the
  consensus that these documents are ready to go seems rather solid.
  In particular, multiple inquiries and WG Last Calls have not turned
  up any significant controversy or unresolved issues.


10.

  Has anyone threatened an appeal or otherwise indicated extreme
  discontent?  If so, please summarise the areas of conflict in
  separate email messages to the Responsible Area Director.  (It should
  be in a separate email because this questionnaire is publicly
  available.)

  No


11.

  Identify any ID nits the Document Shepherd has found in this
  document.  (See http://www.ietf.org/tools/idnits/ and the Internet-
  Drafts Checklist).  Boilerplate checks are not enough; this check
  needs to be thorough.

  o  This document contains a normative downward reference to
      I-D.leiba-5322upd-from-group.  The issue is discussed in Section 6
      above.

  o  The reference to the now-obsolete RFC 5504 (in the context of IANA
      Considerations) is intentional and necesary.


12.

  Describe how the document meets any required formal review criteria,
  such as the MIB Doctor, media type, and URI type reviews.

  No such review criteria apply to any of these four documents.


13.

  Have all references within this document been identified as either
  normative or informative?

  Yes


14.

  Are there normative references to documents that are not ready for
  advancement or are otherwise in an unclear state?  If such normative
  references exist, what is the plan for their completion?

  There is a normative reference to draft-leiba-5322upd-from-group as
  discussed in Section 6 above.


15.

  Are there downward normative references references (see RFC 3967)?

  These documents are intended for Proposed Standard.  There are no
  normative references to Experimental or Informational documents.


16.

  Will publication of this document change the status of any existing
  RFCs?  Are those RFCs listed on the title page header, listed in the
  abstract, and discussed in the introduction?  If the RFCs are not
  listed in the Abstract and Introduction, explain why, and point to
  the part of the document where the relationship of this document to
  the other RFCs is discussed.  If this information is not in the
  document, explain why the WG considers it unnecessary.

  This document does not change the status of any existing RFC.  Its
  closest predecessor, RFC 5504, was obsoleted by RFC 5630.


17.

  Describe the Document Shepherd's review of the IANA considerations
  section, especially with regard to its consistency with the body of
  the document.  Confirm that all protocol extensions that the document
  makes are associated with the appropriate reservations in IANA
  registries.

  Confirm that any referenced IANA registries have been clearly
  identified.  Confirm that newly created IANA registries include a
  detailed specification of the initial contents for the registry, that
  allocations procedures for future registrations are defined, and a
  reasonable name for the new registry has been suggested (see RFC
  5226).

  The IANA Considerations section modifies the Permanent Message Header
  Field registry.  The instructions seem to be clear and to be
  consistent with those in RFC 3864.  I believe that IANA has provided
  a preliminary review of the way the instructions are presented.  No
  new registries are created.


18.

  List any new IANA registries that require Expert Review for future
  allocations.  Provide any public guidance that the IESG would find
  useful in selecting the IANA Experts for these new registries.

  No new IANA registries are created by any of this set of documents.


19.

  Describe reviews and automated checks performed by the Document
  Shepherd to validate sections of the document written in a formal
  language, such as XML code, BNF rules, MIB definitions, etc.

  Sections 3 and 4 of this document contain snippets of ABNF but not a
  complete grammar that could be run through an automated checker (a
  complete grammar would require incorporating extensive material from
  RFC 5322, which the WG was strongly advised against).  The
  productions and partial productions that are present have been hand-
  checked by the author and several WG participants.