Email Address Internationalization K. Hurtta
(EAI) March 17, 2007
Internet-Draft
Intended status: Informational
Expires: September 18, 2007
Message Store requirements for Internationalized Email
draft-hurtta-eai-messagestore-00
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 18, 2007.
Copyright Notice
Copyright (C) The IETF Trust (2007).
Abstract
The Email Address Internationalization (EAI) is implemented by
allowing UTF-8 characters in SMTP envelope and mail headers.
UTF8SMTP extension of ESMTP takes care that mails with UTF-8
characters in SMTP envelope and mail headers are not delivered to EAI
non-compliant SMTP servers. This document describes mechanism how to
keep messages with UTF-8 characters on mail headers separated from
EAI non-compliant Mail User Agents. This document also describes
Hurtta Expires September 18, 2007 [Page 1]
Internet-Draft EAI Message Store March 2007
general requirements for UTF8SMTP Message Store.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1. Message Store with IMAP and POP access . . . . . . . . . . 5
2.2. Message Store with direct file system access . . . . . . . 6
3. Message Store requirements . . . . . . . . . . . . . . . . . . 6
4. Mail Delivery Agent requirements . . . . . . . . . . . . . . . 7
5. Final delivery MTA requirements . . . . . . . . . . . . . . . 8
6. Traditional Unix mailboxes . . . . . . . . . . . . . . . . . . 8
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9
8. Security Considerations . . . . . . . . . . . . . . . . . . . 10
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 10
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 10
10.1. Normative References . . . . . . . . . . . . . . . . . . . 10
10.2. Informative References . . . . . . . . . . . . . . . . . . 11
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 11
Intellectual Property and Copyright Statements . . . . . . . . . . 13
Hurtta Expires September 18, 2007 [Page 2]
Internet-Draft EAI Message Store March 2007
1. Introduction
Internationalized email [ietf-eai-framework] includes UTF-8
characters [RFC3629] on email headers [ietf-eai-utf8headers].
Internationalized email is incompatible with EAI unaware Mail
Transport and User Agents. Therefore it is required that
Internationalized email are separated from EAI unaware Mail Agents.
UTF8SMTP extension [ietf-eai-smtpext] of ESMTP takes care that mails
with UTF-8 characters in SMTP envelope and mail headers are not
delivered to EAI non-compliant SMTP servers. EAI compliant Message
Store need take care that mails with UTF-8 characters in mail headers
are not seen by EAI non-compliant Mail User Agents.
IMAP protocol extension [ietf-eai-imap-utf8] and POP protocol
extension [ietf-eai-pop] takes care that mails with UTF-8 characters
in mail headers are not seen by EAI non-compliant Mail User Agents
when served via IMAP and POP protocol. Therefore this this document
focus more traditional Unix mailboxes, where Mail User Agents access
mail via file system access.
Focus of this document is on requirements of Message Stores and not
actual implementation choice although some suggestions are given.
IMAP and POP protocol assumes existence of Message Store.
Requirements given on this document apply also Message Stores used by
POP and IMAP servers.
Mail submission is not discussed on this document.
2. Model
Terms "Mail Transfer Agents" (MTA), "Mail User Agent" (MUA), "Message
Delivery Agent" (MDA) and "Message Store" (MS) are used according of
[crocker-email-arch].
Term "final delivery MTA" is used according of [ietf-eai-framework].
Term "UTF8SMTP message" are used for messages which follow
[ietf-eai-utf8headers].
Term "ASCII header message" are used for messages which follow
[RFC2822].
Hurtta Expires September 18, 2007 [Page 3]
Internet-Draft EAI Message Store March 2007
This document assumes following model for mail architecture.
+----------+ +---------+
-- SMTP ---> | final | -----> | |
(a) | delivery | (b) | MDA |
| MTA | | |
+----------+ +---------+
| |
write \ /
(c) |
(---------)
( )
( MS )
( )
(---------)
|
(d) / \
access | |
| |
+---------+
| |
| MUA |
| |
+---------+
(a) Mail arrives via SMTP [RFC2821].
(b) Final MTA passes mail to MDA with proprietary method or via LMTP
[RFC2033].
(c) MDA writes message to MS.
(d) MUA access messages on MS with some means.
This model is divided to two special model:
1. Case where MUA access MS with POP and IMAP protocol, and
2. Case where MUA access MS via file system access.
Hurtta Expires September 18, 2007 [Page 4]
Internet-Draft EAI Message Store March 2007
2.1. Message Store with IMAP and POP access
This model assumes that MUAs do not access MS via file system access.
+----------+
-- SMTP ---> | final |
(a) | delivery |
| MTA |
+----------+
|
| (b)
|
v (----)
+-----------------------+ (c) ( )
| MDA | ----- ( )
|.......................| ( )
| | ( MS )
| mail server | (d) ( )
| | ----- ( )
| | ( )
+-----------------------+ (----)
| | |
(e) IMAP (f)POP (g)HTTP
| | |
| v |
+---------+ +-------+ +--------+
| | | | | WWW |
| MUA | | MUA | | browser|
| | | | +--------+
+---------+ +-------+
|
(-------)
( MUA's )
(folders)
(-------)
(a)-(c) As on general model.
(d) Mail server access messages on MS with some means.
(e) MUA access mail server (IMAP server) with IMAP [RFC3501].
(f) MUA access mail server (POP server) with POP [RFC1939].
(g) WWW browser access mail server with HTTP [RFC2616].
Hurtta Expires September 18, 2007 [Page 5]
Internet-Draft EAI Message Store March 2007
2.2. Message Store with direct file system access
This model assumes that MUA access Message Store with file system
access.
+----------+ +---------+
-- SMTP ---> | final | -----> | |
(a) | delivery | (b) | MDA |
| MTA | | |
+----------+ +---------+
||
(c) write ||
vv
(----------------------------)
( MS [incoming mailboxes] )
(----------------------------)
| |
| | (d) read,write
| |
+-------+
| |
| MUA |
| |
+-------+
|
(-------)
( MUA's )
(folders)
(-------)
(a)-(c) As on general model.
(d) MUA access MS via file system access.
Traditionally Unix incoming mailboxes are files /var/mail/ or /var/
spool/mail/ directory or similar places. User's incoming mail is
located on file which name correspond to username of user. These
files are on "mbox" format [RFC4155].
NOTE: Also other arrangements and formats exists.
3. Message Store requirements
Requirements for UTF8SMTP aware Message Store are following:
o UTF8SMTP messages must be accessible to UTF8SMTP aware MUAs on
UTF8SMTP form without information lost.
Hurtta Expires September 18, 2007 [Page 6]
Internet-Draft EAI Message Store March 2007
o UTF8SMTP ignorant MUAs must not see UTF8SMTP messages (messages
can be hidden or downgraded [ietf-eai-downgrade] for UTF8SMTP
ignorant MUAs).
o All messages should be accessible to UTF8SMTP aware MUAs on that
form which MS received them. This requirements is needed that
possible signatures and hashes calculated for message can be
verified.
There is two mail reason why UTF8SMTP ignorant MUAs must not see
UTF8SMTP messages:
o Earlier standards allows only ASCII header fields. Therefore
UTF-8 may cause malfunction for MUA.
o Address syntax on UTF8SMTP header fields includes fallback
address. Therefore UTF8SMTP ignorant MUAs are not able to parse
these header fields.
Notes:
o It is not required UTF8SMTP ignorant MUA can see UTF8SMTP message
on downgraded form. When MUAs access MS via file system this is
seen to difficult (or wasteful) requirement.
o Although this document suggest that messages with ASCII [ASCII]
header fields and UTF8SMTP messages are handled separately it is
allowed that UTF8SMTP aware MS handles all messages as UTF8SMTP
messages. ASCII header message can be treated as subset of
UTF8SMTP messages. This implies that UTF8SMTP ignorant MUA does
not see any message (if downgrading is not provided).
o On some environment it is common that Subject: and some other
header fields are not encoded sometimes, but use some random 8-bit
(presumably system local) character set. If used character set is
not UTF-8, these are not UTF8SMTP messages. Therefore these
messages can not be treated as UTF8SMTP messages as ASCII header
messages can be treated.
o First requirement (messages accessible on UTF8SMTP form)
practically forbids storing messages only on downgraded form.
4. Mail Delivery Agent requirements
Requirements for UTF8SMTP aware Mail Delivery Agent are following:
o MDA must not deliver UTF8SMTP messages to UTF8SMTP unaware Message
Store.
Notes:
o In practice this means that if Mail Delivery Agent is UTF8SMTP, MS
must be constructed that way that requirements previous section
(Section 3) are fulfilled.
Hurtta Expires September 18, 2007 [Page 7]
Internet-Draft EAI Message Store March 2007
o UTF8SMTP messages does not include label, which which tells which
messages are UTF8SMTP messages and which messages are ASCII header
messages. To discover message is UTF8SMTP message may require
that all message header fields (including header fields from MIME
body parts) are parsed. Because MDAs are not expected to parse
messages, it is suggested that final delivery MTA pass that
information to MDA.
o On some legacy environments it is common that Subject: and some
other header fields are not encoded. Therefore presence of 8-bit
bytes itself does not indicate that message is UTF8SMTP message.
Test is also needed that sequence of 8-bit bytes forms UTF-8
characters.
5. Final delivery MTA requirements
UTF8SMTP aware MTAs must fill requirements of [ietf-eai-smtpext].
For UTF8SMTP aware final delivery MTA there is following additional
requirements:
o Final delivery MTA must not deliver UTF8SMTP messages to UTF8SMTP
unaware mail delivery agent.
Notes:
o If final delivery MTA is UTF8SMTP aware, it is recommended that
MDA is arranged that way that it is UTF8SMTP aware.
o If communication between final delivery MTA and MDA use LMTP,
UTF8SMTP response to LHLO command tells that MDA is UTF8SMTP
aware.
o In general final delivery MTA can be expected to parse messages,
so it knows when them are UTF8SMTP messages. Passing that
information to MDA may require that LMTP is extended.
6. Traditional Unix mailboxes
On this section suggestion to how handle traditional Unix mailboxes
is given.
Let's assume that incoming mail for user is stored to /var/mail/
{username} file on UTF8SMTP unaware MS by using "mbox" format
[RFC4155], where {username} is user's account name. To make MS
UTF8SMTP aware, MS is modified following way:
o UTF8SMTP messages are stored to /var/mail/{username}:UTF8 file by
MDA (for UTF8SMTP messages ':UTF8' is appended to name of file.)
If environment is fully UTF8SMTP aware, all messages are stored to
/var/mail/{username}:UTF8 file by MDA.
Hurtta Expires September 18, 2007 [Page 8]
Internet-Draft EAI Message Store March 2007
o ASCII header messages can be still stored to /var/mail/{username}
file.
o MDA is allowed (but not required) to store copy of UTF8SMTP
messages on downgraded form to /var/mail/{username} file. This
means that if downgrading is provided, storage space requirements
are doubled. If downgrading fails, MDA must not reject message,
but instead just store original UTF8SMTP message to /var/mail/
{username}:UTF8 file.
o On some cases it is common that Subject: and some other header
fields are not encoded and use some random 8-bit character set
(presumably local character set of system). If UTF-8 is not used
on header fields, these messages are not UTF8SMTP messages.
Therefore they are not stored to /var/mail/{username}:UTF8 file.
Handling of these messages is not changed.
It is very unlikely that Unix account names includes ':' characters,
therefore it is expected that ':UTF8' suffix does not conflict with
user's account names.
UTF8SMTP aware MUA needs to do following:
o When incoming mailbox is opened, both files /var/mail/{username}
and /var/mail/{username}:UTF8 are parsed.
o If message with same Message-ID header field is presented both on
/var/mail/{username} and /var/mail/{username}:UTF8 file, only
message from /var/mail/{username}:UTF8 file is presented.
o UTF8SMTP aware MUA must not write UTF8SMTP messages to /var/mail/
{username} file.
It is recommended that configuration for MDA is provided, which
allows specify
o which users accepts only non-UTF8SMTP messages, and
o which users accepts also UTF8SMTP messages.
In lack of configuration, following heuristic is suggested:
o Presence of /var/mail/{username}:UTF8 file indicates that user
accepts UTF8SMTP messages.
o If file /var/mail/{username} exists and /var/mail/{username}:UTF8
file do not exists, that can be treated that user accepts only
non-UTF8SMTP messages.
o Presence of /var/mail/{username}:UTF8 file without /var/mail/
{username} file can be treated that all messages should be treated
as UTF8SMTP.
o If either /var/mail/{username} or /var/mail/{username}:UTF8 file
exists, that can be treated as temporary error condition.
7. IANA Considerations
There is no IANA considerations in this document.
Hurtta Expires September 18, 2007 [Page 9]
Internet-Draft EAI Message Store March 2007
8. Security Considerations
If user uses UTF8SMTP unaware MUA, UTF8SMTP messages may look for
his/her that they go to bit bucket although they appear to be
delivered as far sender is considered.
See "Security considerations" section in [ietf-eai-framework] for
more discussion.
9. Acknowledgements
Various requirements and ideas are suggested on IMA mailing list
discussions.
10. References
10.1. Normative References
[RFC2822] Resnick, P., Ed., "Internet Message Format", RFC 2822,
April 2001.
[RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO
10646", RFC 3629, STD 63, November 2003.
[ietf-eai-framework]
Klensin, J. and Y. Ko, "Overview and Framework for
Internationalized Email", draft-ietf-eai-framework-05
(work in progress), February 2007.
[]
Yeh, J. and Abel, "Internationalized Email Headers",
draft-ietf-eai-utf8headers-04 (work in progress),
March 2007.
[ietf-eai-downgrade]
YONEYA, Y., Ed. and K. Fujiwara, Ed., "Downgrading
mechanism for Email Address Internationalization",
draft-ietf-eai-downgrade-03 (work in progress),
March 2007.
[ietf-eai-smtpext]
Yao, J., Ed. and W. Mao, Ed., "SMTP extension for
internationalized email address",
draft-ietf-eai-smtpext-04 (work in progress), March 2007.
[crocker-email-arch]
Hurtta Expires September 18, 2007 [Page 10]
Internet-Draft EAI Message Store March 2007
Crocker, D., "Internet Mail Architecture",
draft-crocker-email-arch-06 (work in progress),
March 2007.
10.2. Informative References
[ASCII] American National Standards Institute (formerly United
States of America Standards Institute), "USA Code for
Information Interchange", ANSI X3.4-1968, 1968.
ANSI X3.4-1968 has been replaced by newer versions with
slight modifications, but the 1968 version remains
definitive for the Internet.
[RFC1939] Myers, J. and M. Rose, "Post Office Protocol - Version 3",
RFC 1939, STD 53, May 1996.
[RFC2033] Myers, C., "Local Mail Transfer Protocol", RFC 2033,
October 1996.
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
Masinter, L., and T. Berners-Lee, "Hypertext Transfer
Protocol -- HTTP/1.1", RFC 2616, June 1999.
[RFC2821] Klensin, J., Ed., "Simple Mail Transfer Protocol",
RFC 2821, April 2001.
[RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION
4rev1", RFC 3501, March 2003.
[RFC4155] Hall, E., "The application/mbox Media Type", RFC 4155,
September 2005.
[ietf-eai-imap-utf8]
Resnick, P. and C. Newman, "IMAP Support for UTF-8",
draft-ietf-eai-imap-utf8-01 (work in progress),
March 2007.
[ietf-eai-pop]
Newman, C., "POP3 Support for UTF-8",
draft-ietf-eai-pop-01 (work in progress), January 2007.
Hurtta Expires September 18, 2007 [Page 11]
Internet-Draft EAI Message Store March 2007
Author's Address
Kari Hurtta
Kala-Matti 4 B 24
02230 Espoo
FI
Email: hurtta-ietf@elmme-mailer.org
URI: http://iki.fi/keh/
Hurtta Expires September 18, 2007 [Page 12]
Internet-Draft EAI Message Store March 2007
Full Copyright Statement
Copyright (C) The IETF Trust (2007).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Acknowledgment
Funding for the RFC Editor function is provided by the IETF
Administrative Support Activity (IASA).
Hurtta Expires September 18, 2007 [Page 13]