[Search] [txt|pdf|bibtex] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01                                                         
Internet Draft: IMAP Submit Without Download          R. Gellens, Editor
Document: draft-ietf-lemonade-submit-01.txt                     Qualcomm
Expires: April 2004                                         October 2003


                      IMAP Submit Without Download


Status of this Memo

    This document is an Internet-Draft and is in full conformance with
    all provisions of Section 10 of RFC2026.

    Internet-Drafts are working documents of the Internet Engineering
    Task Force (IETF), its areas, and its working groups.  Note that
    other groups may also distribute working documents as
    Internet-Drafts.

    Internet-Drafts are draft documents valid for a maximum of six
    months and may be updated, replaced, or obsoleted by other documents
    at any time.  It is inappropriate to use Internet- Drafts as
    reference material or to cite them other than as "work in progress."

    The list of current Internet-Drafts can be accessed at
    <http://www.ietf.org/ietf/1id-abstracts.txt>

    The list of Internet-Draft Shadow Directories can be accessed at
    <http://www.ietf.org/shadow.html>.


Copyright Notice

    Copyright (C) The Internet Society (2003).  All Rights Reserved.


Abstract

    IMAP clients, especially those operating over low-bandwidth or
    high-latency links (such as cellular telephones) need to be able to
    submit mail messages containing all or part of previously-received
    IMAP messages.  Clients need to be able to do this without sloshing
    the bits both ways, that is, without being forced to download IMAP
    messages solely to be able to upload the content in a submitted
    message.








Gellens                   [Page 1]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


    This document currently identifies two main approaches to doing this
    (called "IMAP push" and "IMAP pull"), one which adds submission to
    IMAP, and another which adds the expansion of IMAP references to
    message submission.  This version of the document attempts to lay
    out the protocol mechanisms along with associated trade-offs and
    security considerations of each.  Once a decision has been made to
    go with a particular technique, this document will describe the
    specifics of that choice as a standards-track proposal.











































Gellens                   [Page 2]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


Table of Contents

     1.  Conventions Used in this Document  . . . . . . . . . . . . .  3
     2.  Fundamental Requirement . . . . . . . . . . . . . . . . . .   3
     3.  Additional Requirements, Goals, Etc: . . . . . . . . . . . .  4
     4.  "IMAP Push" Versus "IMAP Pull"  . . . . . . . . . . . . . .   4
       4.1.  "IMAP Push"  . . . . . . . . . . . . . . . . . . . . . .  4
         4.1.1.  Overview  . . . . . . . . . . . . . . . . . . . . .   4
         4.1.2.  "IMAP Push" Using an External Agent  . . . . . . . .  4
         4.1.3.  "IMAP Push" Using APPEND  . . . . . . . . . . . . .   5
         4.1.4.  "IMAP Push" Using Proxy Model  . . . . . . . . . . .  6
         4.1.5.  "IMAP Push" Using COMPOSE . . . . . . . . . . . . .   7
         4.1.6.  Unresolved Issues  . . . . . . . . . . . . . . . . .  7
         4.1.7.  Security Considerations . . . . . . . . . . . . . .   7
         4.1.8.  Advantages . . . . . . . . . . . . . . . . . . . . .  8
         4.1.9.  Disadvantages . . . . . . . . . . . . . . . . . . .   8
       4.2.  "IMAP Pull"  . . . . . . . . . . . . . . . . . . . . . .  8
         4.2.1.  Overview  . . . . . . . . . . . . . . . . . . . . .   8
         4.2.2.  Mechanisms . . . . . . . . . . . . . . . . . . . . .  9
         4.2.3.  IMAP Access Authorization . . . . . . . . . . . . .  10
         4.2.4.  Security Considerations  . . . . . . . . . . . . . . 11
         4.2.5.  Advantages  . . . . . . . . . . . . . . . . . . . .  11
         4.2.6.  Disadvantages  . . . . . . . . . . . . . . . . . . . 12
     5.  Security Considerations . . . . . . . . . . . . . . . . . .  12
     6.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . 12
     7.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . .  12
     8.  Normative References . . . . . . . . . . . . . . . . . . . . 12
     9.  Informative References  . . . . . . . . . . . . . . . . . .  13
    10.  Editor's Address . . . . . . . . . . . . . . . . . . . . . . 13
       Intellectual Property Statement . . . . . . . . . . . . . . .  13
       Full Copyright Statement . . . . . . . . . . . . . . . . . . . 14


1.  Conventions Used in this Document

    The key words "REQUIRED", "MUST", "MUST NOT", "SHOULD", "SHOULD
    NOT", and "MAY" in this document are to be interpreted as described
    in "Key words for use in RFCs to Indicate Requirement Levels"
    [KEYWORDS].

    In protocol examples, "C:" designates lines sent by the client,
    while "S:" show lines sent by the server.  In such examples, line
    breaks are for editorial clarity only.








Gellens                   [Page 3]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


2.  Fundamental Requirement

    The fundamental requirement is to allow [IMAP] clients (especially
    those on low-bandwidth or high-latency links) to submit messages
    containing all or part of received messages without having to slosh
    all the bits both ways (that is, without needing to download the
    content of received messages or parts thereof just to be able to
    upload it within a submitted message).


3.  Additional Requirements, Goals, Etc:

    Being able to store the sent message on the IMAP server is a goal.


4. "IMAP Push" Versus "IMAP Pull"

    Two main approaches have been proposed.  We can call these "IMAP
    push" and "IMAP pull", since in one case the content is "pushed"
    from the IMAP server to the [message submission] server, while in
    the other case the content is "pulled" from the IMAP server by the
    submit server.


4.1. "IMAP Push"

4.1.1.  Overview

    In the "IMAP push" model, the client uses IMAP to submit new
    messages.  When the new message contains (or consists of) all or
    part of another message, the IMAP server copies the data into the
    new message.  It is the IMAP server that performs the initial
    message submission.


4.1.2. "IMAP Push" Using an External Agent

    In this approach, the client creates a message, APPENDs it to a
    mailbox, and when desired adds a "queued" flag and a [message
    submission] envelope using [annotate].  An external agent notices
    the "queued" annotation and submits the message.










Gellens                   [Page 4]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


    Because the actual submission is done by an external agent, no
    changes to IMAP are needed apart from [annotations] (and
    specification of the "queued" flag name).


4.1.3. "IMAP Push" Using APPEND

    In the APPEND variant of "IMAP push", the client creates a skeleton
    for a message it desires to submit.  This can include references to
    previously-received IMAP messages or portions thereof.  All other
    aspects of the message (such as headers, new content and in-line
    quoted text) appear in the skeleton as they are to appear in the
    message that is actually sent.  The client APPENDs this message in a
    mailbox on the IMAP server, marking it as a draft or skeleton in
    some manner (perhaps a flag or using [annotate]).

    When desired, the client instructs the IMAP server to submit the
    message.  At or subsequent to this point, the IMAP server connects
    to the submit server, sends the appropriate commands, and then
    (because it is indicated as a skeleton) sends the message skeleton
    one body part at a time.  If it encounters a part which has any
    content-type other than message/external-body, it sends it
    (including its headers) as-is.  If the IMAP server finds a body part
    with a content-type of message/external-body with an access-type of
    URL which points to a message body on the IMAP server, it sends the
    encapsulated headers of the message/external-body (not including
    those headers specific to the content-type) followed by the contents
    of the IMAP message body indicated by the external-body (which may
    be part of an IMAP message).

    The IMAP server does not need to examine the contents of the body
    parts it is retrieving from elsewhere in the IMAP store; it fetches
    them normally, redirecting the output to the submit server rather
    than to the requesting client.  The IMAP server does not recurse
    through the referenced body parts looking for more external-body
    parts.

    The client MUST set an appropriate content-transfer-encoding for
    each embedded part.

    This method requires the IMAP server to maintain a mail queue, have
    the ability to add and delete messages in this queue, monitor for
    failures, etc.  In general mail queues are rather complex, although
    in this case less so than for a full SMTP server.  However, the
    operations for a single mail queue are essentially the same as for a
    bunch of queues.





Gellens                   [Page 5]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


    The IMAP server would also need the ability to generate proper DSNs.

    Note that one likely implementation strategy may be to bolt sendmail
    onto an IMAP server.


4.1.4. "IMAP Push" Using Proxy Model

    In a proxy model, the IMAP server opens a connection to the submit
    server in real-time and assembles the data as it sends the message.
    It does not return OK to the client until it gets the 250 response
    from the submit server.  The IMAP submission command may thus take
    the 20 minutes SMTP permits for the MAIL FROM, RCPT TO, and DATA
    commands plus the data itself.  Of course, the new IMAP command
    could impose a shorter timeout.

    Client can close the connection before it gets an OK, in which case
    the command continues.  The client would then need to examine the
    message in the mailbox to learn if the submission failed, or look
    for DSNs in the mailbox.

    The IMAP server can mark the message as to success or failure, and
    if failure include specific error text from the submit server,
    perhaps in an [annotation].

    Another IMAP command could be added to get the EHLO response from
    the submit server.  It would be possible to require support for a
    base set of extensions.  The IMAP client can issue this new command
    if it needs to use an extension not required to be supported; it may
    also issue this command on occasion (once per session, per day, per
    week) to cache the supported extensions.

    A new IMAP command would be defined to submit mail.  The desired
    SMTP envelope can be sent as a literal or added using [annotate].
    The message contents can be sent as a literal or as a reference.
    The name of an IMAP folder into which the message will be deposited
    could be permitted.  A flag could be included to fail the entire
    message if any recipients fail.  IMAP sends as unsolicited response
    each response code.

    Proxy authentication can be solved by having the IMAP server
    authenticate to the submit server as an administrative user (perhaps
    with TLS client certificates), or if the client permits, by the IMAP
    server sending the client's credentials to the submit server.  The
    submit server can be required to support SASL authorization IDs as
    well as MAIL FROM ...  AUTH=...  Support for an authorization ID
    allows the IMAP server to send messages from itself (administrative
    messages) and to send messages on behalf of a user, and for these
    cases to be distinguished.


Gellens                   [Page 6]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


    Can use SASL external if the submit server trusts the IMAP server.


4.1.5. "IMAP Push" Using COMPOSE

    In the COMPOSE variant of "IMAP push", the client uses a new COMPOSE
    command.  This IMAP command would be similar to APPEND (with the
    mailbox in which to append, an optional flag list and optional
    date/time string), except that instead of a message literal, it
    would take as parameters any number of literals and "message part
    identifiers".  All of the literals and the message parts would be
    appended together in the destination mailbox as one message.  The
    client would be responsible for providing the appropriate MIME
    boundaries in the literals.


4.1.6.  Unresolved Issues

    Deletion of referenced data before expansion is a concern,
    especially with the APPEND variant, although one that could be
    ignored, as it would be a result of client action (a timing hole
    exists with any solution, although it is larger or smaller in some
    models).

    A number of variances are possible.  These include how the envelope
    information (including return-path, recipients and extensions) is
    indicated; the precise mechanism for references to IMAP messages or
    portions, including how such mechanism interacts with [mailbox
    referrals], and if the mechanism requires that all referenced data
    exist on the same IMAP server doing the submission; if the final
    submission completes synchronously or synchronously or if this is an
    option; how errors during the submission process are to be handled.

    One approach to the envelope issue would be for the client to store
    the complete desired submission envelope in a message annotation per
    the [annotate] extension.  Another is to simply include this in the
    IMAP command which submits the message.  In either case, the desired
    envelope can be sent by the client as a literal, or can be
    integrated into IMAP.  The former has the advantage of tying IMAP
    submission directly to [message submission], thus guaranteeing that
    all current and future [message submission] extensions are valid
    within the IMAP submit mechanism.  This avoids potentially creating
    two parallel but separate Internet mail submission mechanisms, one
    for IMAP clients and one for all other clients (including POP and
    submit-only).  The latter has the advantage of providing a clear way
    to inform the client of the submission extensions supported by the
    server.




Gellens                   [Page 7]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


4.1.7.  Security Considerations

    This approach requires that the submit server trust the IMAP server
    to submit messages on behalf of the end user.  In addition, since
    new functionality is being added to IMAP, including expansion of
    referenced data, implementation errors have the potential to create
    vulnerabilities that could compromise the IMAP server, giving access
    to all of the user's IMAP data, all IMAP data for all users, or root
    access to the system.


4.1.8.  Advantages

    * Avoids requiring Submit server to understand MIME.
    * The IMAP server already stores the messages which are referenced,
    and hence can get at them conveniently.
    * The IMAP server can easily store the sent message in the user's
    Outbox or Sent Mail folder, where they are immediately available
    (there is the potential for delay in the "IMAP pull" model).
    * Messages can remain in a "sendable but not queued" state, where
    they are complete but not eligible to be submitted, for any desired
    period of time, where they can be edited or deleted.


4.1.9.  Disadvantages

    * Only external-body references to messages on the IMAP server can
    be expanded.
    * Adds message submission to IMAP, with the potential for two
    separate Internet mail submission mechanisms
    * Adds complexity to IMAP for a limited case
    * Administrative complexity in setting up the trust relationship
    between the servers
    * Knowing what happened to a message that was not fully successful
    or which the client doesn't know (connection closed) requires a sent
    mail folder with annotated message
    * The APPEND variant has a number of disadvantages discussed in its
    section.


4.2. "IMAP Pull"

4.2.1.  Overview








Gellens                   [Page 8]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


    In the "IMAP pull" model, the client connects to a [message
    submission] server and submits a message.  This message may contain
    references which the client requests the server to expand.  During
    or subsequent to the session, the server dereferences these URLs and
    creates the complete message.  This allows arbitrary content to be
    incorporated into messages without first downloading by the client.


4.2.2.  Mechanisms

    One possible solution for storing copies of sent mail in the IMAP
    server would be to standardize an IMAP mailbox annotation using
    [annotate] which contains an email address that posts directly to
    the mailbox. (This would be a desirable feature for shared mailboxes
    as well as message submission.) Another approach would be for the
    client to use the proposed COMPOSE command (as described in section
    4.1.5 above) to create the final message in the desired IMAP
    mailbox, then instruct the submit server to send that message.

    Several techniques are possible for the client to indicate to the
    submit server that the message contains references to be expanded.
    One approach, as described in [BURL], creates a new [message
    submission] verb that includes a URL reference which the server
    expands.  This new BURL command directs the server to fetch the data
    object to which the URL refers, perform any necessary content
    transfer encoding conversions on that object and include it in the
    message.

    BURL can be combined with the CHUNKING extension to SMTP added by
    [binary SMTP].  The CHUNKING extension allows submission of a series
    of byte-counted message fragments using the BDAT command.  One or
    more BURL commands could be interleaved with the BDAT commands.
    This allows the message to be composed without any sort of hokey
    post-processing.

    For example, a message could be submitted as follows:

        BDAT ....                     headers and first part
        BURL 8bit <imap-url> retry    forwarded 8bit content from IMAP
        BDAT ....                     MIME delimiters or whatever
        BURL base64 <imap-url>        get IMAP content; base64 encode
        BDAT ....                     final data

    BURL can include a flag indicating how errors are to be handled; the
    example above uses 'retry', meaning that the submit server should
    retry instead of failing the command (and any subsequent BURL or
    BDAT commands).




Gellens                   [Page 9]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


    Potentially, this could be pipeline-able into one round-trip.

    The BURL command can also include a flag, similar to that used with
    BDAT, to indicate that this is the last such command.  This would
    allow a client to compose a message using BURL to fetch the end of
    the message, or to submit a message which consists of one other
    message, using a single BURL command and no BDAT commands.  For
    example:

        BURL 8bit <imap-url> last failnow

    The BURL extension should advertise what types of URLs it supports
    (the initial focus should of course be for IMAP; additional URL
    types could be considered later if they are desirable) and which
    hosts trust it.  For example:

        250-BURL imap://trusted-host.example.com/


4.2.3.  IMAP Access Authorization

    The significant issue with "IMAP pull" is how to give the [message
    submission] server access to the referenced IMAP data.  In many
    cases the servers will have a trust relationship, but it is
    desirable for the solution to not require this.

    One approach is to include authorization within the reference.  The
    IMAP URL used to indicate which messages or message parts are to be
    included in the submitted message can include authorization
    credentials.  This is separate from any authentication mechanism or
    credentials.

    The "pawn token" mechanism proposed in [URLAUTH] provides
    authorization for the posseser of the token to access one specific
    piece of data.  The submit server could use the SASL ANONYMOUS
    mechanism (or equivalent), or authenticate as a role (that is, as a
    submit server) and then use one or more tokens to authorize access
    to one or more messages or message parts.

    The limited-use "pawn token" URL can be generated by the client as
    described in [URLAUTH].  The URL can expire at a specific time and
    date, and/or can be limited to use by a specified user or role (such
    as the submit server).








Gellens                   [Page 10]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


    As described in [URLAUTH], the submit server would resolve the
    URL(s) by the new URLFETCH command.  This command returns a literal
    of the relevant message/body part(s) if the authorization token is
    valid.

4.2.4.  Security Considerations

    This method requires either that the IMAP server trusts the submit
    server or that the "pawn ticket" mechanism be used which authorizes
    access to specific limited content (this URL could expire after some
    amount of time and/or be usable only by the submit server).

    If the IMAP server trusts the submit server, the IMAP server can
    include in its CAPABILITIES response a list of such trusted
    submission servers.  Or, the submit server can indicate which IMAP
    servers trust it.

    Note that the primary target for forward-without-download is email
    on cell phones, where the carrier will operate both the IMAP and
    submit servers and there is thus a pre-existing trust relationship
    between these servers.

    Also note that plain text passwords (with or without TLS) are in
    widespread use (indeed are dominant) for authentication.
    Authenticating to the submit server may thus disclose authentication
    credentials sufficient to grant the submit server access to the
    user's IMAP account.

    The URI-based token mechanism poses a risk for channels which are
    not secure (secure channels are not required by [message submission]
    or [SMTP]).  Given a compromise there, an attacker could anticipate
    the download of a body part by dereferencing the URI before the
    submit server (by taking a second or third BURL-reference, for
    example, to win the race condition).  This compromises the body
    part.  However, since this vulnerability requires a clear-text
    channel, this is no worse a problem that submitting a new message
    over an unprotected channel in the first place.  That is, if the
    channel between the client and the submit server is not protected,
    then all messages submitted over that channel are vulnerable to
    eavesdropping anyway.


4.2.5.  Advantages








Gellens                   [Page 11]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


    * Keeps message submission within [message submission].
    * Avoids adding significant new functionality to [IMAP]
    * Can potentially be expanded to include other types or URIs and
    additional associated services

    In conjunction with the COMPOSE mechanism described in 4.1.5 above:

    * The IMAP server can easily store the sent message in the user's
    Outbox or Sent Mail folder, where they are immediately available
    (there is the potential for delay in the "IMAP pull" model).
    * Messages can remain in a "sendable but not queued" state, where
    they are complete but not eligible to be submitted, for any desired
    period of time, where they can be edited or deleted.


4.2.6.  Disadvantages

    * Adds significant new functionality to [message submission] servers
    * Requires submit server to have access to (at least specific pieces
    of) IMAP data


5.  Security Considerations

    Security considerations are currently discussed within each of the
    two main alternatives above.


6.  IANA Considerations

    None identified yet.  Stay tuned.


7.  Acknowledgements

    The editor is grateful for and would like to acknowledge the
    significant contributions made to this document by several members
    of the lemonade group, including Mark Crispin, Cyrus Daboo, Larry
    Greenfield, Ted Hardie, Chris Newman, and Pete Resnick.


8.  Normative References

    [binary SMTP] "SMTP Service Extensions for Transmission of Large and
    Binary MIME Messages", G. Vaudreuil, December 2000, RFC 3030.






Gellens                   [Page 12]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


    [BURL] Newman, C., "Message Composition",
    draft-newman-lemonade-compose-xx (work in progress).

    [IMAP] "INTERNET MESSAGE ACCESS PROTOCOL -- VERSION 4rev1", M.
    Crispin, March 2003, RFC 3501.

    [message submission] "Message Submission", R. Gellens, J. Klensin,
    December 1998, RFC 2476.

    [URL access-type] "Definition of the URL MIME External-Body
    Access-Type", N. Freed, K. Moore, A. Cargille, October 1996, RFC
    2017.

    [URLAUTH] Crispin, Newman, "Internet Message Access Protocol (IMAP)
    - URLAUTH Extension", draft-crispin-imap-urlauth-xx (work in
    progress).


9.  Informative References

    [annotate] Gellens, Daboo, "IMAP ANNOTATE Extension" (work in
    progress).

    [mailbox referrals] "IMAP4 Mailbox Referrals", M. Gahrns, September
    1997, RFC 2193.

    [SMTP] "Simple Mail Transfer Protocol", J. Klensin, Ed., April 2001,
    RFC 2821.


10.  Editor's Address

    Randall Gellens
    QUALCOMM Incorporated
    5775 Morehouse Drive
    San Diego, CA  92121
    USA
    randy@qualcomm.com


Intellectual Property Statement

    The IETF takes no position regarding the validity or scope of any
    intellectual property or other rights that might be claimed to
    pertain to the implementation or use of the technology described in
    this document or the extent to which any license under such rights
    might or might not be available; neither does it represent that it
    has made any effort to identify any such rights.  Information on the
    IETF's procedures with respect to rights in standards-track and


Gellens                   [Page 13]                   Expires April 2004

Internet Draft         IMAP Submit Without Download         October 2003


    standards-related documentation can be found in BCP-11.  Copies of
    claims of rights made available for publication and any assurances
    of licenses to be made available, or the result of an attempt made
    to obtain a general license or permission for the use of such
    proprietary rights by implementors or users of this specification
    can be obtained from the IETF Secretariat.

    The IETF invites any interested party to bring to its attention any
    copyrights, patents or patent applications, or other proprietary
    rights which may cover technology that may be required to practice
    this standard.  Please address the information to the IETF Executive
    Director.


Full Copyright Statement

    Copyright (C) The Internet Society 2003.  All Rights Reserved.

    This document and translations of it may be copied and furnished to
    others, and derivative works that comment on or otherwise explain it
    or assist in its implementation may be prepared, copied, published
    and distributed, in whole or in part, without restriction of any
    kind, provided that the above copyright notice and this paragraph
    are included on all such copies and derivative works.  However, this
    document itself may not be modified in any way, such as by removing
    the copyright notice or references to the Internet Society or other
    Internet organizations, except as needed for the purpose of
    developing Internet standards in which case the procedures for
    copyrights defined in the Internet Standards process must be
    followed, or as required to translate it into languages other than
    English.

    The limited permissions granted above are perpetual and will not be
    revoked by the Internet Society or its successors or assigns.

    This document and the information contained herein is provided on an
    "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
    TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
    BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
    HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
    MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.










Gellens                   [Page 14]                   Expires April 2004