IMAP Extensions Working Group                                   B. Leiba
Internet Draft                           IBM T.J. Watson Research Center
Document: draft-ietf-imapext-list-extensions-06.txt             May 2004
                                                   Expires November 2004


                     IMAP4 LIST Command Extensions

Status of this Document

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.  Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF), its areas,
   and its working groups.  Note that other groups may also distribute
   working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

     The list of current Internet-Drafts can be accessed at
     http://www.ietf.org/ietf/1id-abstracts.txt
     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html.

   A revised version of this draft document will be submitted to the RFC
   editor as an Proposed Standard for the Internet Community.
   Discussion and suggestions for improvement are requested, and should
   be sent to ietf-imapext@imc.org.  This document will expire before 31
   November 2004.  Distribution of this memo is unlimited.

   This documents obsoletes RFC 3348 and updates RFC 2193.


Abstract

   IMAP4 has two commands for listing mailboxes: LIST and LSUB.  As we
   have added extensions that have required specialized lists (see
   [MboxRefer] for an example) we have had to expand the number of list
   commands, since each extension must add its function to both LIST and
   LSUB, and these commands are not, as they are defined, extensible.
   If we've needed the extensions to work together, we've had to add a
   set of commands to mix the different options, the set increasing in
   size with each new extension.  This document describes an extension
   to the base LIST command that will allow these additions to be done
   with mutually compatible options to the LIST command, avoiding the
   exponential increase in specialized list commands.


1. Conventions used in this document

   In examples, "C:" indicates lines sent by a client that is connected
   to a server.  "S:" indicates lines sent by the server to the client.

   The words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" are
   used in this document as specified in RFC 2119 [Keywords].


2. Introduction and overview

   The extensions to the LIST command will be accomplished by amending
   the syntax to allow options to be specified.  The list of options
   will replace the several commands that are currently used to mix and
   match the information requested.  The new syntax is backward-
   compatible, with no ambiguity: if the first word after the command
   name begins with a parenthesis, the new syntax is being used; if it
   does not, it's in the original syntax.

   By adding options to the LIST command, we are announcing the intent
   to phase out and eventually to deprecate the RLIST and RLSUB commands
   described in [MboxRefer].  We are also defining the mechanism to
   request extended mailbox information, such as is described in the
   "Child Mailbox Extension" [ChildMbox].  The base
   LSUB command is not deprecated by this extension; rather, this
   extension adds a way to obtain subscription information with more
   options, with those server implementations that support it.  Clients
   that simply need a list of subscribed mailboxes, as provided by the
   LSUB command, SHOULD continue to use that command.

   This document defines an IMAP4 extension that is identified by the
   capability string "LISTEXT". The LISTEXT extension makes the
   following changes to the IMAP4 protocol, which are described in
   more details in sections 3 and 4 of this document:

   a.  defines new syntax for LIST command options.
   b.  adds LIST command options: SUBSCRIBED, REMOTE and CHILDREN
   c.  adds new mailbox flags "\NonExistent", "\PlaceHolder",
       "\HasChildren" and "\HasNoChildren".


3. LIST Command Options

   The LIST command syntax is extended by adding a parenthesized list of
   command options between the command name and the reference name (see
   the formal syntax in section 6 for specific details).  Command
   options will be defined in this document and in approved extension
   documents; each option will be enabled by a capability string (one
   capability may enable multiple options), and a client MUST NOT send
   an option for which the server has not advertised support.  A server
   MUST respond to options it does not recognize with a NO response.

   This extension is identified by the capability string "LISTEXT", and
   support for it is a prerequisite for any future extensions that
   require specialized forms of the LIST command.  Such extensions MUST
   refer to this document and MUST add their function through command
   options as described herein.
   This extension also defines extensions to the LIST response, allowing
   a series of extended fields at the end, a parenthesized list of tagged
   data (also referred to as "extended data item"). The first element of
   an extended field is a tag, which identifies type of the data. Tags
   MUST be registered with IANA, as described in section 8.5 of this
   document. An example of such extended set might be

       ((tablecloth (("fringe" "lacy")("color" "white")))(X-Sample
       "text"))

   or...

       ((tablecloth ("fringe" "lacy"))(X-Sample "text" "and even more text"))

   See the formal grammar, below, for the full syntatic details.
   The server MAY return data in the extended fields that was not solicited
   by the client. The client MUST ignore all extended fields it doesn't
   recognize.

   The options defined in this specification are

       SUBSCRIBED - causes the LIST command to list subscribed
          mailboxes, rather than the actual mailboxes.  This will often
          be a subset of the actual mailboxes.  It's also possible for
          this list to contain the names of mailboxes that don't exist.
          In any case, the list MUST include exactly those mailbox names
          that match the selection criteria and are subscribed to.  This
          option is intended to supplement the LSUB command.
          Of particular note are the mailbox flags as returned by this
          option, compared with what is returned by LSUB.  With the
          latter, the flags returned may not reflect the actual flag
          status on the mailbox, and the \NoSelect flag has a special
          meaning (it indicates that this mailbox is not, itself,
          subscribed, but that it has child mailboxes that are).  With
          the SUBSCRIBED option described here, the flags are accurate
          and complete, and have no special meanings.
          "LSUB" and "LIST (SUBSCRIBED)" are, thus, not the same thing,
          and some servers must do significant extra work to respond to
          "LIST (SUBSCRIBED)".  Because of this, clients SHOULD continue
          to use "LSUB" unless they specifically want the additional
          information offered by "LIST (SUBSCRIBED)".

          This option defines a new mailbox flag, "\NonExistent", that
          indicates that a mailbox is subscribed to, but does not
          actually exist.  The "\NonExistent" flag MUST be supported and
          MUST be accurately computed.

       REMOTE - causes the LIST command to show remote mailboxes as
          well as local ones, as described in [MboxRefer].  This option
          is intended to replace the RLIST command and, in conjunction
          with the SUBSCRIBED option, the RLSUB command.  This option is
          only available on servers that also support RFC 2193.

       CHILDREN - Requests mailbox child information as originally
          proposed in [ChildMbox].  See section 4, below, for details.
          Support for this is optional, but this option MUST be accepted
          by all servers (though it MAY be ignored).

   The LISTEXT capability also defines a new mailbox flag,
   "\PlaceHolder", that indicates that the designated mailbox does not
   meet the selection criteria of the given LIST command, but that it
   has one or more child mailboxes that do <<EDITORIAL NOTE: "might"?>>.
   The LSUB command indicates this condition by using the "\NoSelect"
   flag, but the LIST (SUBSCRIBED) command MUST NOT do that, since
   "\NoSelect" retains its original meaning here.  Further, the
   "\PlaceHolder" flag is more general, in that it can be used with any
   extended set of selection criteria.


4. The CHILDREN Option

   The CHILDREN option implements the Child Mailbox Extension,
   originally proposed by Mike Gahrns and Raymond Cheng, of Microsoft
   Corporation.  Most of the information in this section is taken
   directly from their original specification [ChildMbox].  The CHILDREN
   option is simply an indication that the client wants this
   information; a server MAY provide it even if the option is not
   specified, or MAY ignore the option entirely.
   Many IMAP4 [IMAP4] clients present to the user a hierarchical view of
   the mailboxes that a user has access to.  Rather than initially
   presenting to the user the entire mailbox hierarchy, it is often
   preferable to show to the user a collapsed outline list of the
   mailbox hierarchy (particularly if there is a large number of
   mailboxes).  The user can then expand the collapsed outline hierarchy
   as needed.  It is common to include within the collapsed hierarchy a
   visual clue (such as a ''+'') to indicate that there are child
   mailboxes under a particular mailbox.   When the visual clue is
   clicked the hierarchy list is expanded to show the child mailboxes.
   The Child Mailbox Extension provides a mechanism for a client to
   efficiently determine if a particular mailbox has children, without
   issuing a LIST "" * or a LIST "" % for each mailbox name.
   The Child Mailbox Extension defines two new attributes that MAY be
   returned within a LIST response: \HasChildren and \HasNoChildren.
   While these attributes MAY be returned in response to any LIST
   command, the CHILDREN option is provided to indicate that the client
   particularly wants this information.  If the CHILDREN option is
   present, the server SHOULD return these attributes even if their
   computation is expensive.

\HasChildren - The presence of this attribute indicates that the mailbox
       has child mailboxes.
       A server SHOULD NOT set this attribute if there are child
       mailboxes, and the user does not have permissions to access any
       of them. In this case, \HasNoChildren SHOULD be used.
       In many cases, however, a server may not be able to efficiently
       compute whether a user has access to all child mailboxes.  As
       such a client MUST be prepared to accept the \HasChildren
       attribute as a hint.  That is, a mailbox MAY be flagged with the
       \HasChildren attribute, but no child mailboxes will appear in
       the LIST response.

\HasNoChildren - The presence of this attribute indicates that the
       mailbox has NO child mailboxes that are accessible to the
       currently authenticated user.

   In some instances a server that supports the Child Mailbox Extension
   might not be able to determine whether a mailbox has children.  For
   example it may have difficulty determining whether there are child
   mailboxes when LISTing mailboxes while operating in a particular
   namespace.
   In these cases, a server MAY exclude both the \HasChildren and
   \HasNoChildren attributes in the LIST response.  As such, a client
   can not make any assumptions about whether a mailbox has children
   based upon the absence of a single attribute.  In particular, some
   servers may not be able to combine the SUBSCRIBED and CHILDREN
   options.  Such servers MUST honour the SUBSCRIBED option, and they
   will simply ignore the CHILDREN option if both are requested.
   It is an error for the server to return both a \HasChildren and a
   \HasNoChildren attribute in a LIST response.

   Note: the \HasNoChildren attribute should not be confused with the
   IMAP4 [IMAP4] defined attribute \NoInferiors which indicates that no
   child mailboxes exist now and none can be created in the future.


5. Examples

   The first example shows the complete local hierarchy that will be
   used for the other examples.

       C: A01 LIST "" "*"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST () "/" "Fruit"
       S: * LIST () "/" "Fruit/Apple"
       S: * LIST () "/" "Fruit/Banana"
       S: * LIST () "/" "Tofu"
       S: * LIST () "/" "Vegetable"
       S: * LIST () "/" "Vegetable/Broccoli"
       S: A01 OK done

   In the next example, we'll see the subscribed mailboxes.  This is
   similar, but not equivalent, to <LSUB "" "*">.  Note that the mailbox
   called "Fruit/Peach" is subscribed to, but does not actually exist
   (perhaps it was deleted while still subscribed).  And the "Fruit"
   mailbox is not subscribed to, but it has two subscribed children.

       C: A02 LIST (SUBSCRIBED) "" "*"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\PlaceHolder) "/" "Fruit"
       S: * LIST () "/" "Fruit/Banana"
       S: * LIST (\NonExistent) "/" "Fruit/Peach"
       S: A02 OK done

   The next example shows the use of the CHILDREN option.  The client,
   without having to list the second level of hierarchy, now knows which
   of the top-level mailboxes have sub-mailboxes (children) and which do
   not.  Note that it's not necessary for the server to return the
   \HasNoChildren flag for the inbox, because the \NoInferiors flag
   already implies that, and has a stronger meaning.

       C: A03 LIST (CHILDREN) "" "%"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\HasChildren) "/" "Fruit"
       S: * LIST (\HasNoChildren) "/" "Tofu"
       S: * LIST (\HasChildren) "/" "Vegetable"
       S: A03 OK done

   In this example we see more mailboxes, which reside on another server
   to which we may obtain referrals.  This is similar to the command
   <RLIST "" "%">.  We also see the mixing of two options.  Note that in
   the case of the remote mailboxes, the server might or might not be
   able to include CHILDREN information; it includes it if it can, and
   omits it if it can't.

       C: A04 LIST (REMOTE CHILDREN) "" "%"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\HasChildren) "/" "Fruit"
       S: * LIST (\HasNoChildren) "/" "Tofu"
       S: * LIST (\HasChildren) "/" "Vegetable"
       S: * LIST () "/" "Bread"
       S: * LIST (\HasChildren) "/" "Meat"
       S: A04 OK done


6. Formal Syntax

   The following syntax specification uses the augmented Backus-Naur
   Form (BNF) as described in [ABNF].  Terms not defined here are taken
   from [IMAP4].

   child-mbox-flag    = "\HasChildren" / "\HasNoChildren"
                        ; flags for Child Mailbox Extension, at most one
                        ; possible per LIST response

   list               = "LIST" [SP list-options] SP mailbox SP list-mailbox

   list-options       = "(" [option *(SP option)] ")"

   mailbox-list       = "(" [mbx-list-flags] ")" SP
                        (DQUOTE QUOTED-CHAR DQUOTE / nil) SP mailbox
                        [SP mbox-list-extended]

   mbox-list-extended = "(" [mbox-list-extended-item
                        *(SP mbox-list-extended-item)] ")"

   mbox-list-extended-item = "(" mbox-list-extended-item-data ")"

   mbox-list-extended-item-data =  mbox-list-extended-item-tag SP nstring-list

   mbox-list-extended-item-tag  = vendor-tag / standard-tag
                        ; A tag registration template is described in section
                        ; 8.5 of this document.

   vendor-tag         = "V-" atom
                        ; a vendor specific tag for extended list data

   standard-tag         = atom
                        ; a tag for extended list data defined in a Standard
                        ; Track or Experimental RFC.

   nstring-list       = nstring /
                        "(" [nstring-list *(SP nstring-list)] ")"
                        ;; a recursive list definition

   mbox-list-oflag    = child-mbox-flag / "\NonExistent" / "\PlaceHolder"

   option             = "SUBSCRIBED" / "CHILDREN" / "REMOTE" /
                        option-extension
                        ; An option registration template is described in section
                        ; 8.3 of this document.

   option-extension   = option-vendor / option-public

   option-vendor      = "V-" atom
                        ; a vendor specific option

   option-public      = atom
                        ; an option defined in a Standard Track or
                        ; Experimental RFC


7. Security Considerations

   This document describes syntactic changes to the specification of the
   IMAP4 commands LIST, LSUB, RLIST, and RLSUB, and the modified LIST
   command has the same security considerations as those commands.  They
   are described in [IMAP4] and [MboxRefer].

   The Child Mailbox Extension provides a client a more efficient means
   of determining whether a particular mailbox has children.  If a
   mailbox has children, but the currently authenticated user does not
   have access to any of them, the server SHOULD respond with a
   \HasNoChildren attribute.  In many cases, however, a server may not
   be able to efficiently compute whether a user has access to all child
   mailboxes.  If such a server responds with a \HasChildren attribute,
   when in fact the currently authenticated user does not have access to
   any child mailboxes, potentially more information is conveyed about
   the mailbox than intended.   In most situations this will not be a
   security concern, because if information regarding whether a mailbox
   has children is considered sensitive, a user would not be granted
   access to that mailbox in the first place.


8. IANA Considerations

8.1.  Guidelines for IANA


   It is requested that IANA creates two new registries for LISTEXT
   options and LISTEXT extended response data. The templates and
   the initial registrations are detailed below.


8.2.  Registration procedure and Change control


   Registration of a LISTEXT option is done by filling in the template
   in section 8.3 and sending it via electronic mail to <iana@iana.org>.
   Registration of a LISTEXT extended data item is done by filling in the
   template in section 8.5 and sending it via electronic mail to <iana@iana.org>.
   IANA has the right to reject obviously bogus registrations, but will
   perform no review of claims made in the registration form.

   A LISTEXT option/extended data item name that starts with "V-" is reserved
   for vendor specific options/extended data items. All options, whether
   they are vendor specific or global, should be registered with IANA.
   If a LISTEXT extended data item is returned as a result of requesting
   a particular LISTEXT option, the name of the option SHOULD be used
   as the name of the LISTEXT extended data item.
   LISTEXT option/extended data item names are case insensitive.

   While the registration procedures do not require it, authors of LISTEXT
   options/extended data items are encouraged to seek community review and
   comment whenever that is feasible.  Authors may seek community review by
   posting a specification of their proposed mechanism as an Internet-
   Draft.  LISTEXT options/extended data items intended for widespread use
   should be standardized through the normal IETF process, when appropriate.

   Comments on registered LISTEXT options/extended response data should
   first be sent to the "owner" of the mechanism and/or to the IMAPEXT WG
   mailing list.
   Submitters of comments may, after a reasonable attempt to contact the
   owner, request IANA to attach their comment to the registration itself.
   If IANA approves of this, the comment will be
   made accessible in conjunction with the registration LISTEXT options/
   extended response data itself.

   Once a LISTEXT registration has been published by IANA, the
   author may request a change to its definition.  The change request
   follows the same procedure as the registration request.

   The owner of a LISTEXT registration may pass responsibility for the
   registered option/extended data item to another person or agency by
   informing IANA; this can be done without discussion or review.

   The IESG may reassign responsibility for a LISTEXT option/extended data item.
   The most common case of this will be to enable changes to be made to
   mechanisms where the author of the registration has died, moved out
   of contact or is otherwise unable to make changes that are important
   to the community.

   LISTEXT registrations may not be deleted; mechanisms which are
   no longer believed appropriate for use can be declared OBSOLETE by a
   change to their "intended use" field; such LISTEXT options/extended data
   items will be clearly marked in the lists published by IANA.

   The IESG is considered to be the owner of all LISTEXT options/extended data items
   which are on the IETF standards track.


8.3.  Registration template for LISTEXT options

     To: iana@iana.org
     Subject: Registration of LISTEXT option X

     LISTEXT option name:

     LISTEXT option description:

     Published specification (optional, recommended):

     Security considerations:

     Intended usage:
     (One of COMMON, LIMITED USE or OBSOLETE)

     Person & email address to contact for further information:

     Owner/Change controller:

     (Any other information that the author deems interesting may be
     added below this line.)



8.4.  Initial LISTEXT option registrations

   It is requested that the LISTEXT option registry is being populated
   with the following entries:

   1)

     To: iana@iana.org
     Subject: Registration of LISTEXT option SUBSCRIBED

     LISTEXT option name: SUBSCRIBED

     LISTEXT option description: Causes the LIST command to list
          subscribed mailboxes, rather than the actual mailboxes.

     Published specification : this RFC, section 3.

     Security considerations: this RFC, section 7.

     Intended usage: COMMON

     Person & email address to contact for further information:
        Alexey Melnikov <Alexey.Melnikov@isode.com>

     Owner/Change controller: IESG <iesg@ietf.org>

   2)

     To: iana@iana.org
     Subject: Registration of LISTEXT option REMOTE

     LISTEXT option name: REMOTE

     LISTEXT option description: causes the LIST command to return
          remote mailboxes as well as local ones, as described in
          RFC 2193.

     Published specification : this RFC, section 3.

     Security considerations: this RFC, section 7.

     Intended usage: COMMON

     Person & email address to contact for further information:
        Alexey Melnikov <Alexey.Melnikov@isode.com>

     Owner/Change controller: IESG <iesg@ietf.org>

   3)

     To: iana@iana.org
     Subject: Registration of LISTEXT option CHILDREN

     LISTEXT option name: CHILDREN

     LISTEXT option description: Requests mailbox child information.

     Published specification : this RFC, sections 3 and 4.

     Published specification : this RFC

     Security considerations: this RFC, section 7.

     Intended usage: COMMON

     Person & email address to contact for further information:
        Alexey Melnikov <Alexey.Melnikov@isode.com>

     Owner/Change controller: IESG <iesg@ietf.org>


8.5.  Registration template for LISTEXT extended data item

     To: iana@iana.org
     Subject: Registration of LISTEXT extended data item X

     LISTEXT extended data item tag:

     LISTEXT extended data item description:

     Which LISTEXT option(s) causes this extended data item
     to be returned (if any):

     Published specification (optional, recommended):

     Security considerations:

     Intended usage:
     (One of COMMON, LIMITED USE or OBSOLETE)

     Person & email address to contact for further information:

     Owner/Change controller:

     (Any other information that the author deems interesting may be
     added below this line.)


9. References

9.1. Normative References

   [Keywords]; Bradner, S.; "Key words for use in RFCs to Indicate
   Requirement Levels"; RFC 2119; Harvard University; March 1997.

   [ABNF]; Crocker, D., and Overell, P. "Augmented BNF for Syntax
   Specifications: ABNF", RFC 2234, November 1997.

   [IMAP4] Crispin, M., "Internet Message Access Protocol - Version
   4rev1", RFC 3501, University of Washington, March 2003.

   [MboxRefer]; Gahrns, M.; "IMAP4 Mailbox Referrals"; RFC 2193;
   Microsoft Corporation; September 1997.

   [ChildMbox]; Gahrns, M. & Cheng, R.; "IMAP4 Child Mailbox Extension";
   RFC 3348; Microsoft Corporation; July 2002.


10. Acknowledgements

   Mike Gahrns and Raymond Cheng of Microsoft Corporation originally
   devised the Child Mailbox Extension and proposed it in 1997; the
   idea, as well as most of the text in section 4, is theirs.

   This document is the result of discussions on the IMAP4 mailing list
   and is meant to reflect consensus of this group.  In particular,
   Mark Crispin, Cyrus Daboo, Timo Sirainen, Ken Murchison, Alexey
   Melnikov, Rob Siemborski, Steve Hole, Arnt Gulbrandsen, Larry
   Greenfield, Phlip Guenther and Pete Maclean were active participants
   in this discussion or made suggestions to this document.


11. Author's Address

   Barry Leiba
   IBM T.J. Watson Research Center
   30 Saw Mill River Road
   Hawthorne, NY  10532
   Phone: 1-914-784-7941
   Email: leiba@watson.ibm.com


12.  Full Copyright Statement

    Copyright (C) The Internet Society 2004.  All Rights Reserved.

    This document and translations of it may be copied and furnished to
    others, and derivative works that comment on or otherwise explain it
    or assist in its implementation may be prepared, copied, published
    and distributed, in whole or in part, without restriction of any
    kind, provided that the above copyright notice and this paragraph
    are included on all such copies and derivative works.  However, this
    document itself may not be modified in any way, such as by removing
    the copyright notice or references to the Internet Society or other
    Internet organizations, except as needed for the purpose of
    developing Internet standards in which case the procedures for
    copyrights defined in the Internet Standards process must be
    followed, or as required to translate it into languages other than
    English.

    The limited permissions granted above are perpetual and will not be
    revoked by the Internet Society or its successors or assigns.

    This document and the information contained herein is provided on an
    "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
    TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
    BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
    HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
    MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


    Acknowledgement

    Funding for the RFC Editor function is currently provided by
    the Internet Society.


13.   Intellectual Property

    The IETF takes no position regarding the validity or scope of any
    Intellectual Property Rights or other rights that might be claimed to
    pertain to the implementation or use of the technology described in
    this document or the extent to which any license under such rights
    might or might not be available; nor does it represent that it has
    made any independent effort to identify any such rights.  Information
    on the procedures with respect to rights in RFC documents can be
    found in BCP 78 and BCP 79.

    Copies of IPR disclosures made to the IETF Secretariat and any
    assurances of licenses to be made available, or the result of an
    attempt made to obtain a general license or permission for the use of
    such proprietary rights by implementers or users of this
    specification can be obtained from the IETF on-line IPR repository at
    http://www.ietf.org/ipr.

    The IETF invites any interested party to bring to its attention any
    copyrights, patents or patent applications, or other proprietary
    rights that may cover technology that may be required to implement
    this standard.  Please address the information to the IETF at
    ietf-ipr@ietf.org.