IMAP MESSAGELIMIT Extension
draft-ietf-extra-imap-messagelimit-08

Network Working Group                                        A. Melnikov
Internet-Draft                                                     Isode
Intended status: Experimental                             A. P. Achuthan
Expires: 4 August 2024                                    V. Nagulakonda
                                                                  Yahoo!
                                                                L. Alves
                                                         1 February 2024

                      IMAP MESSAGELIMIT Extension
                 draft-ietf-extra-imap-messagelimit-08

Abstract

   The MESSAGELIMIT extension of the Internet Message Access Protocol
   (RFC 3501/RFC 9051) allows servers to announce a limit on the number
   of messages that can be processed in a single
   FETCH/SEARCH/STORE/COPY/MOVE/APPEND/EXPUNGE command.  This helps
   servers to control resource usage when performing various IMAP
   operations.  This helps clients to know the message limit enforced by
   corresponding IMAP server and avoid issuing commands that would
   exceed such limit.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 4 August 2024.

Copyright Notice

   Copyright (c) 2024 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.

Melnikov, et al.          Expires 4 August 2024                 [Page 1]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

   This document may contain material from IETF Documents or IETF
   Contributions published or made publicly available before November
   10, 2008.  The person(s) controlling the copyright in some of this
   material may not have granted the IETF Trust the right to allow
   modifications of such material outside the IETF Standards Process.
   Without obtaining an adequate license from the person(s) controlling
   the copyright in such materials, this document may not be modified
   outside the IETF Standards Process, and derivative works of it may
   not be created outside the IETF Standards Process, except to format
   it for publication as an RFC or to translate it into languages other
   than English.

Table of Contents

   1.  Introduction and Overview . . . . . . . . . . . . . . . . . .   3
   2.  Document Conventions  . . . . . . . . . . . . . . . . . . . .   3
   3.  The MESSAGELIMIT extension  . . . . . . . . . . . . . . . . .   3
     3.1.  Returning limits on the number of messages processed in a
           single SEARCH/FETCH/STORE/COPY/MOVE/APPEND/EXPUNGE
           command . . . . . . . . . . . . . . . . . . . . . . . . .   4
     3.2.  UIDAFTER and UIDBEFORE SEARCH criteria  . . . . . . . . .   7
     3.3.  Interaction with SORT and THREAD extensions . . . . . . .   8
     3.4.  Interaction with SEARCHRES extension and IMAP4rev2  . . .   8
     3.5.  Effects of MESSAGELIMIT/SAVELIMIT extensions on non
           compliant clients . . . . . . . . . . . . . . . . . . . .   8
   4.  Formal syntax . . . . . . . . . . . . . . . . . . . . . . . .   9
   5.  Security Considerations . . . . . . . . . . . . . . . . . . .   9
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  10
     6.1.  Changes/additions to the IMAP4 capabilities registry  . .  10
   7.  Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  10
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  10
     8.1.  Normative References  . . . . . . . . . . . . . . . . . .  10
     8.2.  Informative References  . . . . . . . . . . . . . . . . .  11
   Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  11
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  11

Melnikov, et al.          Expires 4 August 2024                 [Page 2]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

1.  Introduction and Overview

   This document defines an extension to the Internet Message Access
   Protocol [RFC3501] for announcing a server limit on the number of
   messages that can be processed in a single
   FETCH/SEARCH/STORE/COPY/MOVE/APPEND/EXPUNGE command.  This extension
   is compatible with both IMAP4rev1 [RFC3501] and IMAP4rev2 [RFC9051].

2.  Document Conventions

   In protocol examples, this document uses a prefix of "C: " to denote
   lines sent by the client to the server, and "S: " for lines sent by
   the server to the client.  Lines prefixed with "// " are comments
   explaining the previous protocol line.  These prefixes and comments
   are not part of the protocol.  Lines without any of these prefixes
   are continuations of the previous line, and no line break is present
   in the protocol unless specifically mentioned.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

   Other capitalised words are IMAP key words [RFC3501][RFC9051] or key
   words from this document.

3.  The MESSAGELIMIT extension

   An IMAP server advertises support for the MESSAGELIMIT extension by
   including "MESSAGELIMIT=<limit>" capability in the CAPABILITY
   response/response code, where "<limit>" is a positive integer that
   conveys the maximum number of messages that can be processed in a
   single [UID] SEARCH/FETCH/STORE/COPY/MOVE/APPEND or UID EXPUNGE
   command.

   An IMAP server that only enforces message limit on [UID] COPY/APPEND
   commands would include the "SAVELIMIT=<limit>" capability (instead of
   the "MESSAGELIMIT=<limit>") in the CAPABILITY response/response code.

   The limit advertised in the MESSAGELIMIT or SAVELIMIT capability
   SHOULD NOT be lower than 1000 messages.

Melnikov, et al.          Expires 4 August 2024                 [Page 3]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

3.1.  Returning limits on the number of messages processed in a single
      SEARCH/FETCH/STORE/COPY/MOVE/APPEND/EXPUNGE command

   If a server implementation doesn't allow more than <N> messages to be
   operated on by a single COPY/UID COPY command, it MUST fail the
   command by returning a tagged NO response with the MESSAGELIMIT
   response code defined below.  If a server implementation doesn't
   allow more than <N> messages to be operated on by a single
   SEARCH/FETCH/STORE/MOVE/EXPUNGE command (or their UID variants), it
   MUST return the MESSAGELIMIT response code defined below:

   MESSAGELIMIT  The server doesn't allow more than <N> messages to be operated
         on by a single SEARCH/FETCH/STORE/COPY/MOVE command (or their
         UID variants).  The lowest processed UID is <LastUID>.  The
         client needs to repeat the operation for remaining messages, if
         required.

         The server doesn't allow more than <N> \Deleted messages to be
         operated on by a single UID EXPUNGE command.  The lowest
         processed UID is <LastUID>.  The client needs to repeat the
         operation for remaining messages, if required.

         Note that when the MESSAGELIMIT response code is returned, the
         server is REQUIRED to process messages from highest to lowest
         UIDs.

         Note that when the MESSAGELIMIT response code is similar to the
         LIMIT ([RFC9051]) response code, but it provides more details
         on the exact type of the limit and how to resume the command
         once the limit is exceeded.

         In the following example the <N> value is 1000 and the lowest
         processed UID <LastUID> is 23221.

 
           C: 03 FETCH 10000:14589 (UID FLAGS)
           S: * 14589 FETCH (FLAGS (\Seen) UID 25000)
           S: * 14588 FETCH (FLAGS (\Answered) UID 24998)
           S: ... further 997 fetch responses
           S: * 13590 FETCH (FLAGS () UID 23221)
           S: 03 OK [MESSAGELIMIT 1000 23221] FETCH completed with 1000 partial
               results

         In the following example the client searches for UNDELETED UIDs
         between 22000:25000.  The total number of searched messages
         (note, NOT the number of matched messages) exceeds the server's
         published 1000 messages limit.

Melnikov, et al.          Expires 4 August 2024                 [Page 4]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

           C: 04 UID SEARCH UID 22000:25000 UNDELETED
           S: * SEARCH 25000 24998 (... UIDs ...) 23221
           S: 04 OK [MESSAGELIMIT 1000 23221] SEARCH completed with 1000 partial results

         The following example demonstrates copy of messages with UIDs
         between 18000:21000.  The total message count exceeds the
         server's published 1000 messages limit.  As COPY/UID COPY needs
         to atomic (as per [RFC3501]/[RFC9051]), no messages are copied.

 
           C: 05 UID COPY 18000:21000 "Trash"
           S: 05 NO [MESSAGELIMIT 1000 20001] Too many messages to copy, try a smaller subset

         The following example shows MOVE of messages with UIDs between
         18000:21000.  The total message count exceeds the server's
         published 1000 messages limit.  (Unlike COPY/UID COPY, MOVE/UID
         MOVE don't need to be atomic.)  The client that wants to move
         all messages in the range and observes a MESSAGELIMIT response
         code, can repeat the UID MOVE command with the same parameter.
         (For the MOVE command, the message set parameter need to be
         updated before repeating the command.)  The client needs to
         keep doing this until the MESSAGELIMIT response is not returned
         (or until a tagged NO/BAD is returned).

 
           C: 06 UID MOVE 18000:21000 "Archive/2021/2021-12"
           S: * OK [COPYUID 1397597919 20001:21000 22363:23362] Some messages were not moved
           S: * 12336 EXPUNGE
           S: * 12335 EXPUNGE
           ...
           S: * 11337 EXPUNGE
           S: 06 OK [MESSAGELIMIT 1000 20001] MOVE completed for the last 1000 messages

         The following example shows update of flags for messages with
         UIDs between 18000:20000.  The total number of existing
         messages in the UID range exceeds the server's published 1000
         messages limit.  The client that wants to change flags for all
         messages in the range and observes a MESSAGELIMIT response
         code, can repeat the UID STORE command with the updated UID
         range that doesn't include the UID returned in the MESSAGELIMIT
         response code.  (For the STORE command, the message set
         parameter also need to be updated before repeating the
         command.)  The client needs to keep doing this until the
         MESSAGELIMIT response is not returned (or until a tagged NO/BAD
         is returned).

Melnikov, et al.          Expires 4 August 2024                 [Page 5]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

           C: 07 UID STORE 18000:20000 +FLAGS (\Seen)
           S: * 11215 FETCH (FLAGS (\Seen \Deleted) UID 20000)
           S: * 11214 FETCH (FLAGS (\Seen \Answered \Deleted) UID 19998)
           ...
           S: * 10216 FETCH (FLAGS (\Seen) UID 19578)
           S: 07 OK [MESSAGELIMIT 1000 19578] STORE completed for the last 1000 messages

         The following example shows removal of messages (using UID
         EXPUNGE) that have \Deleted flag set with UIDs between
         11000:13000.  The total message count of messages with \Deleted
         flag set exceeds the server's published 1000 messages limit.
         The client that wants to remove all messages marked as \Deleted
         in the range and observes a MESSAGELIMIT response code, can
         repeat the UID EXPUNGE command with the same parameter.  The
         client needs to keep doing this until the MESSAGELIMIT response
         is not returned (or until a tagged NO/BAD is returned).

 
           C: 08 UID EXPUNGE 11000:13000
           S: * 4306 EXPUNGE
           S: * 4305 EXPUNGE
           ...
           S: * 3307 EXPUNGE
           S: 08 OK [MESSAGELIMIT 1000 11627] UID EXPUNGE completed for the last 1000 messages

         The following example shows removal of messages (using EXPUNGE)
         that have \Deleted flag set.  Unlike UID EXPUNGE, the server
         MUST NOT impose any message limit when processing EXPUNGE.

 
           C: 09 EXPUNGE
           S: * 4306 EXPUNGE
           S: * 4305 EXPUNGE
           ...
           S: * 3307 EXPUNGE
           S: * 112 EXPUNGE
           S: 09 OK EXPUNGE completed

         Similarly, the server MUST NOT impose any message limit when
         processing a CLOSE or a STATUS UNSEEN command.

         The following example shows use of MESSAGELIMIT response code
         together with the PARTIAL [RFC9394] extension.  The total
         message count (as specified by the PARTIAL range) exceeds the
         server's published 1000 messages limit, so the server refuses
         to do any work in this case.

Melnikov, et al.          Expires 4 August 2024                 [Page 6]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

           C: 10 UID FETCH 22000:25000 (UID FLAGS MODSEQ) (PARTIAL -1:-1500)
           S: 10 NO [MESSAGELIMIT 1000] FETCH exceeds the maximum 1000 message limit

         Without the PARTIAL parameter the above UID FETCH can look like
         this:

 
           C: 10 UID FETCH 22000:25000 (UID FLAGS MODSEQ)
           S: * 12367 FETCH (FLAGS (\Seen \Deleted) UID 23007)
           S: * 12366 FETCH (FLAGS (\Seen \Answered \Deleted) UID 23114)
           ...
           S: * 13366 FETCH (FLAGS (\Seen) UID 24598)
           S: 10 OK [MESSAGELIMIT 1000 23007] FETCH exceeds the maximum 1000 message limit

   Note that when the server needs to return both EXPUNGEISSUED
   ([RFC9051]) and MESSAGELIMIT response codes, the former MUST be
   returned in the tagged OK response, while the latter MUST be returned
   in an untagged NO response.  The following example demonstrates that:

     C: 11 FETCH 10000:14589 (UID FLAGS)
     S: * 14589 FETCH (FLAGS (\Seen) UID 25000)
     S: * 14588 FETCH (FLAGS (\Answered) UID 24998)
     S: ... further 997 fetch responses
     S: * 13590 FETCH (FLAGS () UID 23221)
     S: * NO [MESSAGELIMIT 1000 23221] FETCH completed with 1000 partial
         results
     S: 11 OK [EXPUNGEISSUED] Some messages were also expunged

   When IMAP MULTIAPPEND [RFC3502] extension is also supported by the
   server, the message limit also applies to the APPEND command.

3.2.  UIDAFTER and UIDBEFORE SEARCH criteria

   The MESSAGELIMIT extension also defines 2 extra SEARCH keys: UIDAFTER
   and UIDBEFORE, which make it easier to convert a single UID to a
   range of UIDs.

   "UIDAFTER <uid>" - Messages that have a UID greater than the
   specified UID.  This is semantically the same as "UID <uid>+1:*".

   "UIDBEFORE <uid>" - Messages that have a UID less than the specified
   UID.  This is semantically the same as "UID 1:<uid>-1" (or if <uid>
   has the value 1, then the empty set).

Melnikov, et al.          Expires 4 August 2024                 [Page 7]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

   These 2 SEARCH keys are particularly useful when the SEARCHRES
   [RFC5182] extension is also supported, but they can be used without
   it.  For example, this allows a SEARCH that sets the "$" marker to be
   converted to a range of messages in a subsequent SEARCH, and both
   SEARCH requests can be pipelined.

     C: 12 UID SEARCH UIDAFTER 25000 UNDELETED
     S: * SEARCH 27800 27798 (... 250 UIDs ...) 25001
     S: 12 OK SEARCH completed

3.3.  Interaction with SORT and THREAD extensions

   Servers that advertise MESSAGELIMIT N will be unable to execute a
   THREAD [RFC5256] command in a mailbox with more than N messages.

   Servers that advertise MESSAGELIMIT N might be unable to execute a
   SORT [RFC5256] command in a mailbox with more than N messages, unless
   they maintain indices for different SORT orders they support.  In
   absence of such indeces server implementors will need to decide
   whether their server advertises SORT or MESSAGELIMIT capability.

3.4.  Interaction with SEARCHRES extension and IMAP4rev2

   Servers that support both MESSAGELIMIT and SEARCHRES [RFC5182]
   extensions MUST truncate SEARCH SAVE result stored in the $ variable
   when the SEARCH command succeeds, but the MESSAGELIMIT response code
   is returned.  For example, if the following SEARCH would have
   returned 1200 results in absence of MESSAGELIMIT, and the
   MESSAGELIMIT is 1000, only 1000 matching results will be saved in the
   $ variable:

  C: D0004 UID SEARCH RETURN (SAVE) SINCE 1-Jan-2004 NOT FROM "Smith" UID 22000:25000 UNDELETED
  S: D0004 OK [MESSAGELIMIT 1000 1179] SEARCH completed with 1000 partial results saved

3.5.  Effects of MESSAGELIMIT/SAVELIMIT extensions on non compliant
      clients

   A server that advertises the MESSAGELIMIT=N capability would have the
   following effect on clients that don't support this capability:

      Operations on a mailbox that has <= N messages are not affected.

      In a mailbox with more than N messages:

      -  An attempt to COPY/UID COPY more than N messages will always
         fail.

Melnikov, et al.          Expires 4 August 2024                 [Page 8]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

      -  EXPUNGE and CLOSE will always operate on the full mailbox, so
         they are not affected.

      -  Other commands like FETCH, SEARCH and MOVE will be effectively
         restricted to the last N messages of the mailbox.  In
         particular unextended SEARCHes intended for counting of
         messages with or without a particular set of flags would return
         incorrect counts.

4.  Formal syntax

   The following syntax specification uses the Augmented Backus-Naur
   Form (ABNF) notation as specified in [ABNF].

   Non-terminals referenced but not defined below are as defined by
   IMAP4 [RFC3501].

   Except as noted otherwise, all alphabetic characters are case-
   insensitive.  The use of upper or lower case characters to define
   token strings is for editorial clarity only.  Implementations MUST
   accept these strings in a case-insensitive fashion.

   capability          =/ "MESSAGELIMIT=" message-limit /
                          "SAVELIMIT=" message-limit
                          ;; <capability> from [RFC3501]

   message-limit       = nz-number

   resp-text-code      =/ "MESSAGELIMIT" SP message-limit [SP uniqueid]
       ;; No more than nz-number messages can be processed
       ;; by any command at a time. The last (lowest) processed
       ;; UID is uniqueid.
       ;; The last parameter is omitted, when not known.

5.  Security Considerations

   This document defines an additional IMAP4 capability.  As such, it
   does not change the underlying security considerations of [RFC3501]
   and IMAP4rev2 [RFC9051].

   This document defines an optimization that can both reduce the amount
   of work performed by the server, as well at the amount of data
   returned to the client.  Use of this extension is likely to cause the
   server and the client to use less memory than when the extension is
   not used.  However, as this is going to be new code in both the
   client and the server, rigorous testing of such code is required in
   order to avoid introducing of new implementation bugs.

Melnikov, et al.          Expires 4 August 2024                 [Page 9]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

6.  IANA Considerations

6.1.  Changes/additions to the IMAP4 capabilities registry

   IMAP4 capabilities are registered by publishing a standards track or
   IESG approved Informational or Experimental RFC.  The registry is
   currently located at:

      https://www.iana.org/assignments/imap4-capabilities

   IANA is requested to add registrations of "MESSAGELIMIT=" and
   "SAVELIMIT=" capabilities to this registry, both pointing to this
   document.

7.  Acknowledgments

   This document was motivated by Yahoo! team and their questions about
   best client practices for dealing with large mailboxes.

   Editor of this document would like to thank the following people who
   provided useful comments or participated in discussions of this
   document: Timo Sirainen, Barry Leiba, Ken Murchison and Arnt
   Gulbrandsen.

8.  References

8.1.  Normative References

   [ABNF]     Crocker, D., Ed. and P. Overell, Ed., "Augmented BNF for
              Syntax Specifications: ABNF", RFC 5234, January 2008,
              <https://www.rfc-editor.org/info/rfc5234>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC3501]  Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION
              4rev1", RFC 3501, DOI 10.17487/RFC3501, March 2003,
              <https://www.rfc-editor.org/info/rfc3501>.

   [RFC3502]  Crispin, M., "Internet Message Access Protocol (IMAP) -
              MULTIAPPEND Extension", RFC 3502, DOI 10.17487/RFC3502,
              March 2003, <https://www.rfc-editor.org/info/rfc3502>.

   [RFC5182]  Melnikov, A., "IMAP Extension for Referencing the Last
              SEARCH Result", RFC 5182, DOI 10.17487/RFC5182, March
              2008, <https://www.rfc-editor.org/info/rfc5182>.

Melnikov, et al.          Expires 4 August 2024                [Page 10]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

   [RFC5256]  Crispin, M. and K. Murchison, "Internet Message Access
              Protocol - SORT and THREAD Extensions", RFC 5256,
              DOI 10.17487/RFC5256, June 2008,
              <https://www.rfc-editor.org/info/rfc5256>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC9051]  Melnikov, A., Ed. and B. Leiba, Ed., "Internet Message
              Access Protocol (IMAP) - Version 4rev2", RFC 9051,
              DOI 10.17487/RFC9051, August 2021,
              <https://www.rfc-editor.org/info/rfc9051>.

8.2.  Informative References

   [RFC9394]  Melnikov, A., Achuthan, A. P., Nagulakonda, V., and L.
              Alves, "IMAP PARTIAL Extension for Paged SEARCH and
              FETCH", RFC 9394, DOI 10.17487/RFC9394, June 2023,
              <https://www.rfc-editor.org/info/rfc9394>.

Index

   M

      M

         MESSAGELIMIT (response code)
            Section 3.1, Paragraph 2.2.1

Authors' Addresses

   Alexey Melnikov
   Isode Limited
   Email: alexey.melnikov@isode.com
   URI:   https://www.isode.com

   Arun Prakash Achuthan
   Yahoo!
   Email: arunprakash@myyahoo.com

   Vikram Nagulakonda
   Yahoo!
   Email: nvikram_imap@yahoo.com

Melnikov, et al.          Expires 4 August 2024                [Page 11]
Internet-Draft              IMAP MESSAGELIMIT              February 2024

   Luis Alves
   Email: luis.alves@lafaspot.com

Melnikov, et al.          Expires 4 August 2024                [Page 12]