IMAP Extensions Working Group                                   B. Leiba
Internet Draft                           IBM T.J. Watson Research Center
Document: draft-ietf-imapext-list-extensions-05.txt           April 2004
                                                    Expires October 2004



                     IMAP4 LIST Command Extensions


Status of this Document


   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.  Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF), its areas,
   and its working groups.  Note that other groups may also distribute
   working documents as Internet-Drafts.


   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."


     The list of current Internet-Drafts can be accessed at
     http://www.ietf.org/ietf/1id-abstracts.txt
     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html.


   A revised version of this draft document will be submitted to the RFC
   editor as an Proposed Standard for the Internet Community.
   Discussion and suggestions for improvement are requested, and should
   be sent to ietf-imapext@imc.org.  This document will expire before 31
   October 2004.  Distribution of this memo is unlimited.


Abstract


   IMAP4 has two commands for listing mailboxes: LIST and LSUB.  As we
   have added extensions that have required specialized lists (see
   [MboxRefer] for an example) we have had to expand the number of list
   commands, since each extension must add its function to both LIST and
   LSUB, and these commands are not, as they are defined, extensible.
   If we've needed the extensions to work together, we've had to add a
   set of commands to mix the different options, the set increasing in
   size with each new extension.  This document describes an extension
   to the base LIST command that will allow these additions to be done
   with mutually compatible options to the LIST command, avoiding the
   exponential increase in specialized list commands.



1. Conventions used in this document


   In examples, "C:" indicates lines sent by a client that is connected
   to a server.  "S:" indicates lines sent by the server to the client.


   The words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" are
   used in this document as specified in RFC 2119 [Keywords].



2. Introduction


   The extensions to the LIST command will be accomplished by amending
   the syntax to allow options to be specified.  The list of options
   will replace the several commands that are currently used to mix and
   match the information requested.  The new syntax is backward-
   compatible, with no ambiguity: if the first word after the command
   name begins with a parenthesis, the new syntax is being used; if it
   does not, it's in the original syntax.


   By adding options to the LIST command, we are announcing the intent
   to phase out and eventually to deprecate the RLIST and RLSUB commands
   described in [MboxRefer].  We are also defining the mechanism to
   request extended mailbox information, such as is described in the
   "Child Mailbox Extension" [ChildMbox].  The base
   LSUB command is not deprecated by this extension; rather, this
   extension adds a way to obtain subscription information with more
   options, with those server implementations that support it.  Clients
   that simply need a list of subscribed mailboxes, as provided by the
   LSUB command, SHOULD continue to use that command.



3. LIST Command Options


   The LIST command syntax is extended by adding a parenthesized list of
   command options between the command name and the reference name (see
   the formal syntax in section 6 for specific details).  Command
   options will be defined in this document and in approved extension
   documents; each option will be enabled by a capability string (one
   capability may enable multiple options), and a client MUST NOT send
   an option for which the server has not advertised support.  A server
   MUST respond to options it does not recognize with a NO response.
   This extension is identified by the capability string "LISTEXT", and
   support for it is a prerequisite for any future extensions that
   require specialized forms of the LIST command.  Such extensions MUST
   refer to this document and MUST add their function through command
   options as described herein.  This document also defines the "LIST-
   SUBSCRIBED" capability string; see the "SUBSCRIBED" option below.
   This extension also defines extensions to the LIST response, allowing
   a series of extended fields at the end, a parenthesized list of
   attribute/value pairs.  Each attribute is a string, each value may
   be a string or a nested parenthesized list of the same
   attribute/value pairs.  An example of this extended set might be


       (("tablecloth" (("fringe" "lacy")("color" "white")))("X-Sample"
       "text"))


   or...


       (("tablecloth" ("fringe" "lacy"))("X-Sample" "text"))



   See the formal grammar, below, for the full syntatic details.


   The options defined in this specification are


       SUBSCRIBED - causes the LIST command to list subscribed
          mailboxes, rather than the actual mailboxes.  This will often
          be a subset of the actual mailboxes.  It's also possible for
          this list to contain the names of mailboxes that don't exist.
          In any case, the list MUST include exactly those mailbox names
          that match the selection criteria and are subscribed to.  This
          option is intended to supplement the LSUB command, and support
          for it is optional -- a server that supports the SUBSCRIBED
          option indicates so through the LIST-SUBSCRIBED capability.
          Of particular note are the mailbox flags as returned by this
          option, compared with what is returned by LSUB.  With the
          latter, the flags returned may not reflect the actual flag
          status on the mailbox, and the \NoSelect flag has a special
          meaning (it indicates that this mailbox is not, itself,
          subscribed, but that it has child mailboxes that are).  With
          the SUBSCRIBED option described here, the flags are accurate
          and complete, and have no special meanings.
          "LSUB" and "LIST (SUBSCRIBED)" are, thus, not the same thing,
          and some servers must do significant extra work to respond to
          "LIST (SUBSCRIBED)".  Because of this, clients SHOULD continue
          to use "LSUB" unless they specifically want the additional
          information offered by "LIST (SUBSCRIBED)".  At the same time,
          servers SHOULD support the LIST-SUBSCRIBED capability even if
          it entails extra work, because a client that wants the
          information will still obtain it by using LSUB followed by a
          series of LIST commands, so servers might as well make it
          easier.


          This option defines a new mailbox flag, "\NonExistent", that
          indicates that a mailbox is subscribed to, but does not
          actually exist.  The "\NonExistent" flag MUST be supported and
          MUST be accurately computed.


       REMOTE - causes the LIST command to show remote mailboxes as
          well as local ones, as described in [MboxRefer].  This option
          is intended to replace the RLIST command and, in conjunction
          with the SUBSCRIBED option, the RLSUB command.  This option is
          only available on servers that also support RFC 2193.


       CHILDREN - Requests mailbox child information as originally
          proposed in [ChildMbox].  See section 4, below, for details.
          Support for this is optional, but this option MUST be accepted
          by all servers (though it MAY be ignored).


   The LISTEXT capability also defines a new mailbox flag,
   "\PlaceHolder", that indicates that the designated mailbox does not
   meet the selection criteria of the given LIST command, but that it
   has one or more child mailboxes that do <<EDITORIAL NOTE: "might"?>>.
   The LSUB command indicates this condition by using the "\NoSelect"
   flag, but the LIST (SUBSCRIBED) command MUST NOT do that, since
   "\NoSelect" retains its original meaning here.  Further, the
   "\PlaceHolder" flag is more general, in that it can be used with any
   extended set of selection criteria.



4. The CHILDREN Option


   The CHILDREN option implements the Child Mailbox Extension,
   originally proposed by Mike Gahrns and Raymond Cheng, of Microsoft
   Corporation.  Most of the information in this section is taken
   directly from their original specification [ChildMbox].  The CHILDREN
   option is simply an indication that the client wants this
   information; a server MAY provide it even if the option is not
   specified, or MAY ignore the option entirely.
   Many IMAP4 [IMAP4] clients present to the user a hierarchical view of
   the mailboxes that a user has access to.  Rather than initially
   presenting to the user the entire mailbox hierarchy, it is often
   preferable to show to the user a collapsed outline list of the
   mailbox hierarchy (particularly if there is a large number of
   mailboxes).  The user can then expand the collapsed outline hierarchy
   as needed.  It is common to include within the collapsed hierarchy a
   visual clue (such as a ''+'') to indicate that there are child
   mailboxes under a particular mailbox.   When the visual clue is
   clicked the hierarchy list is expanded to show the child mailboxes.
   The Child Mailbox Extension provides a mechanism for a client to
   efficiently determine if a particular mailbox has children, without
   issuing a LIST "" * or a LIST "" % for each mailbox name.
   The Child Mailbox Extension defines two new attributes that MAY be
   returned within a LIST response: \HasChildren and \HasNoChildren.
   While these attributes MAY be returned in response to any LIST
   command, the CHILDREN option is provided to indicate that the client
   particularly wants this information.  If the CHILDREN option is
   present, the server SHOULD return these attributes even if their
   computation is expensive.


\HasChildren - The presence of this attribute indicates that the mailbox
       has child mailboxes.
       A server SHOULD NOT set this attribute if there are child
       mailboxes, and the user does not have permissions to access any
       of them. In this case, \HasNoChildren SHOULD be used.
       In many cases, however, a server may not be able to efficiently
       compute whether a user has access to all child mailboxes.  As
       such a client MUST be prepared to accept the \HasChildren
       attribute as a hint.  That is, a mailbox MAY be flagged with the
       \HasChildren attribute, but no child mailboxes will appear in
       the LIST response.


\HasNoChildren - The presence of this attribute indicates that the
       mailbox has NO child mailboxes that are accessible to the
       currently authenticated user.


   In some instances a server that supports the Child Mailbox Extension
   might not be able to determine whether a mailbox has children.  For
   example it may have difficulty determining whether there are child
   mailboxes when LISTing mailboxes while operating in a particular
   namespace.
   In these cases, a server MAY exclude both the \HasChildren and
   \HasNoChildren attributes in the LIST response.  As such, a client
   can not make any assumptions about whether a mailbox has children
   based upon the absence of a single attribute.  In particular, some
   servers may not be able to combine the SUBSCRIBED and CHILDREN
   options.  Such servers MUST honour the SUBSCRIBED option, and they
   will simply ignore the CHILDREN option if both are requested.
   It is an error for the server to return both a \HasChildren and a
   \HasNoChildren attribute in a LIST response.


   Note: the \HasNoChildren attribute should not be confused with the
   IMAP4 [IMAP4] defined attribute \NoInferiors which indicates that no
   child mailboxes exist now and none can be created in the future.



5. Examples


   The first example shows the complete local hierarchy that will be
   used for the other examples.


       C: A01 LIST "" "*"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST () "/" "Fruit"
       S: * LIST () "/" "Fruit/Apple"
       S: * LIST () "/" "Fruit/Banana"
       S: * LIST () "/" "Tofu"
       S: * LIST () "/" "Vegetable"
       S: * LIST () "/" "Vegetable/Broccoli"
       S: A01 OK done


   In the next example, we'll see the subscribed mailboxes.  This is
   similar, but not equivalent, to <LSUB "" "*">.  Note that the mailbox
   called "Fruit/Peach" is subscribed to, but does not actually exist
   (perhaps it was deleted while still subscribed).  And the "Fruit"
   mailbox is not subscribed to, but it has two subscribed children.


       C: A02 LIST (SUBSCRIBED) "" "*"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\PlaceHolder) "/" "Fruit"
       S: * LIST () "/" "Fruit/Banana"
       S: * LIST (\NonExistent) "/" "Fruit/Peach"
       S: A02 OK done


   The next example shows the use of the CHILDREN option.  The client,
   without having to list the second level of hierarchy, now knows which
   of the top-level mailboxes have sub-mailboxes (children) and which do
   not.  Note that it's not necessary for the server to return the
   \HasNoChildren flag for the inbox, because the \NoInferiors flag
   already implies that, and has a stronger meaning.


       C: A03 LIST (CHILDREN) "" "%"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\HasChildren) "/" "Fruit"
       S: * LIST (\HasNoChildren) "/" "Tofu"
       S: * LIST (\HasChildren) "/" "Vegetable"
       S: A03 OK done


   In this example we see more mailboxes, which reside on another server
   to which we may obtain referrals.  This is similar to the command
   <RLIST "" "%">.  We also see the mixing of two options.  Note that in
   the case of the remote mailboxes, the server might or might not be
   able to include CHILDREN information; it includes it if it can, and
   omits it if it can't.


       C: A04 LIST (REMOTE CHILDREN) "" "%"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\HasChildren) "/" "Fruit"
       S: * LIST (\HasNoChildren) "/" "Tofu"
       S: * LIST (\HasChildren) "/" "Vegetable"
       S: * LIST () "/" "Bread"
       S: * LIST (\HasChildren) "/" "Meat"
       S: A04 OK done



6. Formal Syntax


   The following syntax specification uses the augmented Backus-Naur
   Form (BNF) as described in [ABNF].  Terms not defined here are taken
   from [IMAP4].


   child-mbox-flag    = "\HasChildren" / "\HasNoChildren"
                        ; flags for Child Mailbox Extension, at most one
                        ; possible per LIST response


   list               = "LIST" [SP list-options] SP mailbox SP list-mailbox


   list-options       = "(" [option *(SP option)] ")"


   mailbox-list       = "(" [mbx-list-flags] ")" SP
                        (DQUOTE QUOTED-CHAR DQUOTE / nil) SP mailbox
                        [SP mbox-list-extended]


   mbox-list-extended = "(" [mbox-list-extended-item
                        *(SP mbox-list-extended-item)] ")"


   mbox-list-extended-item = "(" string SP (nstring /
                             mbox-list-extended-item) ")"
                             / mailbox-list-extended


   mbox-list-oflag    = child-mbox-flag / "\NonExistent" / "\PlaceHolder"


   option             = "SUBSCRIBED" / "CHILDREN" / "REMOTE" /
                        option-extension


   option-extension   = atom



7. Security Considerations


   This document describes syntactic changes to the specification of the
   IMAP4 commands LIST, LSUB, RLIST, and RLSUB, and the modified LIST
   command has the same security considerations as those commands.  They
   are described in [IMAP4] and [MboxRefer].


   The Child Mailbox Extension provides a client a more efficient means
   of determining whether a particular mailbox has children.  If a
   mailbox has children, but the currently authenticated user does not
   have access to any of them, the server SHOULD respond with a
   \HasNoChildren attribute.  In many cases, however, a server may not
   be able to efficiently compute whether a user has access to all child
   mailboxes.  If such a server responds with a \HasChildren attribute,
   when in fact the currently authenticated user does not have access to
   any child mailboxes, potentially more information is conveyed about
   the mailbox than intended.   In most situations this will not be a
   security concern, because if information regarding whether a mailbox
   has children is considered sensitive, a user would not be granted
   access to that mailbox in the first place.



8. References


8.1. Normative References


   [IMAP4] Crispin, M., "Internet Message Access Protocol - Version
   4rev1", RFC 3501, University of Washington, March 2003.


   [MboxRefer]; Gahrns, M.; "IMAP4 Mailbox Referrals"; RFC 2193;
   Microsoft Corporation; September 1997.


   [Keywords]; Bradner, S.; "Key words for use in RFCs to Indicate
   Requirement Levels"; RFC 2119; Harvard University; March 1997.


   [ABNF]; Crocker, D., and Overell, P. "Augmented BNF for Syntax
   Specifications: ABNF", RFC 2234, November 1997.



8.2. Informative References


   [ChildMbox]; Gahrns, M. & Cheng, R.; "IMAP4 Child Mailbox Extension";
   RFC 3348; Microsoft Corporation; July 2002.



9. Acknowledgements


   Mike Gahrns and Raymond Cheng of Microsoft Corporation originally
   devised the Child Mailbox Extension and proposed it in 1997; the
   idea, as well as most of the text in section 4, is theirs.


   This document is the result of discussions on the IMAP4 mailing list
   and is meant to reflect consensus of this group.  In particular,
   Mark Crispin, Cyrus Daboo, Timo Sirainen, Ken Murchison, Alexey
   Melnikov, Rob Siemborski, Steve Hole, Arnt Gulbrandsen, Larry
   Greenfield and Pete Maclean were active participants in this
   discussion or made suggestions to this document.



10. Author's Address


   Barry Leiba
   IBM T.J. Watson Research Center
   30 Saw Mill River Road
   Hawthorne, NY  10532
   Phone: 1-914-784-7941
   Email: leiba@watson.ibm.com



11.  Full Copyright Statement

    Copyright (C) The Internet Society 2004.  All Rights Reserved.

    This document and translations of it may be copied and furnished to
    others, and derivative works that comment on or otherwise explain it
    or assist in its implementation may be prepared, copied, published
    and distributed, in whole or in part, without restriction of any
    kind, provided that the above copyright notice and this paragraph
    are included on all such copies and derivative works.  However, this
    document itself may not be modified in any way, such as by removing
    the copyright notice or references to the Internet Society or other
    Internet organizations, except as needed for the purpose of
    developing Internet standards in which case the procedures for
    copyrights defined in the Internet Standards process must be
    followed, or as required to translate it into languages other than
    English.

    The limited permissions granted above are perpetual and will not be
    revoked by the Internet Society or its successors or assigns.

    This document and the information contained herein is provided on an
    "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
    TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
    BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
    HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
    MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.



    Acknowledgement


    Funding for the RFC Editor function is currently provided by
    the Internet Society.



12.   Intellectual Property


    The IETF takes no position regarding the validity or scope of any
    Intellectual Property Rights or other rights that might be claimed to
    pertain to the implementation or use of the technology described in
    this document or the extent to which any license under such rights
    might or might not be available; nor does it represent that it has
    made any independent effort to identify any such rights.  Information
    on the procedures with respect to rights in RFC documents can be
    found in BCP 78 and BCP 79.


    Copies of IPR disclosures made to the IETF Secretariat and any
    assurances of licenses to be made available, or the result of an
    attempt made to obtain a general license or permission for the use of
    such proprietary rights by implementers or users of this
    specification can be obtained from the IETF on-line IPR repository at
    http://www.ietf.org/ipr.


    The IETF invites any interested party to bring to its attention any
    copyrights, patents or patent applications, or other proprietary
    rights that may cover technology that may be required to implement
    this standard.  Please address the information to the IETF at
    ietf-ipr@ietf.org.