IMAP Paged SEARCH & FETCH Extension
draft-melnikov-imap-partial-00
This document is an Internet-Draft (I-D).
Anyone may submit an I-D to the IETF.
This I-D is not endorsed by the IETF and has no formal standing in the
IETF standards process.
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft whose latest revision state is "Replaced".
|
|
|---|---|---|---|
| Authors | Alexey Melnikov , ArunPrakash Achuthan , Vikram Nagulakonda , Luis Alves | ||
| Last updated | 2021-10-22 | ||
| Replaced by | draft-ietf-extra-imap-partial, RFC 9394 | ||
| RFC stream | (None) | ||
| Formats | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-melnikov-imap-partial-00
Network Working Group A. Melnikov
Internet-Draft Isode
Updates: 5267, 4731 (if approved) A. P. Achuthan
Intended status: Standards Track V. Nagulakonda
Expires: 25 April 2022 Yahoo!
L. Alves
Verizon
22 October 2021
IMAP Paged SEARCH & FETCH Extension
draft-melnikov-imap-partial-00
Abstract
The PARTIAL extension of the Internet Message Access Protocol (RFC
3501/RFC 9051) allows clients to limit the number of search results
returned, as well as to perform incremental (paged) searches. This
also helps servers to optimize resource usage when performing
searches.
This document extends PARTIAL SEARCH return option originally
specified in RFC 5267. It also clarifies some interactions between
RFC 5267 and RFC 4731/RFC 9051.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 25 April 2022.
Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved.
Melnikov, et al. Expires 25 April 2022 [Page 1]
Internet-Draft IMAP PARTIAL October 2021
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Simplified BSD License text
as described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Simplified BSD License.
This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other
than English.
Table of Contents
1. Introduction and Overview . . . . . . . . . . . . . . . . . . 2
2. Document Conventions . . . . . . . . . . . . . . . . . . . . 3
3. Incremental SEARCH and partial results . . . . . . . . . . . 3
4. Interaction between PARTIAL, MIN, MAX and SAVE SEARCH return
options . . . . . . . . . . . . . . . . . . . . . . . . . 5
5. Extension to FETCH . . . . . . . . . . . . . . . . . . . . . 6
6. Expressing limits on a number of messages used in
SEARCH/FETCH/STORE/COPY/MOVE . . . . . . . . . . . . . . 6
7. Formal syntax . . . . . . . . . . . . . . . . . . . . . . . . 7
8. Security Considerations . . . . . . . . . . . . . . . . . . . 8
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8
9.1. Changes/additions to the IMAP4 capabilities registry . . 8
10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 9
11. Normative References . . . . . . . . . . . . . . . . . . . . 9
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10
1. Introduction and Overview
This document defines an extension to the Internet Message Access
Protocol [RFC3501] for performing incremental searches and fetches.
This extension is compatible with both IMAP4rev1 [RFC3501] and
IMAP4rev2 [RFC9051].
Melnikov, et al. Expires 25 April 2022 [Page 2]
Internet-Draft IMAP PARTIAL October 2021
The PARTIAL extension of the Internet Message Access Protocol (RFC
3501/RFC 9051) allows clients to limit the number of search results
returned, as well as to perform incremental (paged) searches. This
also helps servers to optimize resource usage when performing
searches.
This document extends PARTIAL SEARCH return option originally
specified in RFC 5267.
2. Document Conventions
In protocol examples, this document uses a prefix of "C: " to denote
lines sent by the client to the server, and "S: " for lines sent by
the server to the client. Lines prefixed with "// " are comments
explaining the previous protocol line. These prefixes and comments
are not part of the protocol. Lines without any of these prefixes
are continuations of the previous line, and no line break is present
in the protocol unless specifically mentioned.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
Other capitalised words are IMAP keywords [RFC3501][RFC9051] or
keywords from this document.
3. Incremental SEARCH and partial results
The PARTIAL search return option causes the server to provide in an
ESEARCH response a subset of the results denoted by the sequence
range given as the mandatory argument. The first result (message
with the lowest matching UID) is 1; thus, the first 500 results would
be obtained by a return option of "PARTIAL 1:500", and the second 500
by "PARTIAL 501:1000". This intentionally mirrors message sequence
numbers.
It is also possible to direct the server to start SEARCH from the
latest matching (with the highest UID) message. This can be done by
prepeding "-" to the index. For example -1 is the last message, -2
is next to the last and so on. Using this syntax helps server
implementations to optimize their SEARCHes.
A single command MUST NOT contain more than one PARTIAL or ALL search
return option -- that is, either one PARTIAL, one ALL, or neither
PARTIAL nor ALL is allowed.
Melnikov, et al. Expires 25 April 2022 [Page 3]
Internet-Draft IMAP PARTIAL October 2021
For SEARCH results, the entire result list MUST be ordered in mailbox
order, that is, in UID or message sequence number order.
Where a PARTIAL search return option references results that do not
exist, by using a range which starts or ends higher (or lower) than
the current number of results, then the server returns the results
that are in the set. This yields a PARTIAL return data item that
has, as payload, the original range and a potentially missing set of
results that may be shorter than the extent of the range. If the
whole range references results that do not exist, a special value
"NIL" is returned by the server instead of the sequence set.
Clients need not request PARTIAL results in any particular order.
Because mailboxes may change, clients might wish to use PARTIAL in
combination with UPDATE (see [RFC5267] if the server also advertises
CONTEXT=SEARCH capability, especially if the intent is to walk a
large set of results; however, these return options do not interact
-- the UPDATE will provide notifications for all matching results.
// Let's assume that the A01 SEARCH without PARTIAL would return
// 23764 results.
C: A01 UID SEARCH RETURN (PARTIAL -1:-100) UNDELETED
UNKEYWORD $Junk
S: * ESEARCH (TAG "A01") UID PARTIAL (-1:-100 ...)
// 100 most recent results in set syntax elided.
S: A01 OK Completed.
// Let's assume that the A02 SEARCH without PARTIAL would return
// 23764 results.
C: A02 UID SEARCH RETURN (PARTIAL 23500:24000) UNDELETED
UNKEYWORD $Junk
C: A03 UID SEARCH RETURN (PARTIAL 1:500) UNDELETED
UNKEYWORD $Junk
C: A04 UID SEARCH RETURN (PARTIAL 24000:24500) UNDELETED
UNKEYWORD $Junk
S: * ESEARCH (TAG "A02") UID PARTIAL (23500:24000 ...)
// 264 results in set syntax elided,
// this spans the end of the results.
S: A02 OK Completed.
S: * ESEARCH (TAG "A03") UID PARTIAL (1:500 ...)
// 500 results in set syntax elided.
S: A03 OK Completed.
S: * ESEARCH (TAG "A04") UID PARTIAL (24000:24500 NIL)
// No results are present, this is beyond the end of the results.
S: A04 OK Completed.
Melnikov, et al. Expires 25 April 2022 [Page 4]
Internet-Draft IMAP PARTIAL October 2021
4. Interaction between PARTIAL, MIN, MAX and SAVE SEARCH return options
This section only applies if the server advertises PARTIAL IMAP
capability or CONTEXT=SEARCH [RFC5267], together with ESEARCH
[RFC4731] and/or IMAP4rev2"[RFC9051].
The SAVE result option doesn't change whether the server would return
items corresponding to PARTIAL SEARCH result options.
As specified in Section 3, it is an error to specify both PARTIAL and
ALL result options in the same SEARCH command.
When the SAVE result option is combined with the PARTIAL result
option, and none of MIN/MAX/COUNT result options is present, the
corresponding PARTIAL is returned, and the "$" marker would contain
all messages returned by the PARTIAL result option.
When the SAVE + PARTIAL result options are combined with the MIN or
the MAX result option, and the COUNT result option is absent, the
corresponding PARTIAL result and MIN/MAX is returned (if the search
result is not empty), and the "$" marker would contain all messages
returned by the PARTIAL result option + the corresponding MIN/MAX
message.
If the SAVE + PARTIAL result options are combined with both MIN and
MAX result options, and the COUNT result options is absent, the
PARTIAL, MIN and MAX are returned (if the search result is not
empty), and the "$" marker would contain all messages returned by the
PARTIAL result option plus MIN and MAX messages.
If the SAVE + PARTIAL result options are combined with the COUNT
result option, the PARTIAL and COUNT are returned, and the "$" marker
would always contain all messages found by the SEARCH or UID SEARCH
command.
The following table summarizes the additional requirement on ESEARCH
server implementations described in this section.
+==============================+=====================+
| Combination of Result option | "$" marker value |
+==============================+=====================+
| SAVE PARTIAL | PARTIAL |
+------------------------------+---------------------+
| SAVE PARTIAL MIN | PARTIAL & MIN |
+------------------------------+---------------------+
| SAVE PARTIAL MAX | PARTIAL & MAX |
+------------------------------+---------------------+
| SAVE PARTIAL MIN MAX | PARTIAL & MIN & MAX |
Melnikov, et al. Expires 25 April 2022 [Page 5]
Internet-Draft IMAP PARTIAL October 2021
+------------------------------+---------------------+
| SAVE PARTIAL COUNT [m] | all found messages |
+------------------------------+---------------------+
Table 1
where '[m]' means optional "MIN" and/or "MAX"
5. Extension to FETCH
The PARTIAL extension also extends the FETCH command with a PARTIAL
FETCH modifier. The PARTIAL FETCH modifier has the same syntax as
the PARTIAL SEARCH result option. Presence of the PARTIAL FETCH
modifier instructs the server to only return FETCH results for
messages in the specified range. It is useful when the sequence-set
(first) parameter to FETCH/UID FETCH command includes unknown number
of messages.
// Returning information for the last 3 messages in the UID range
C: 10 UID FETCH 25900:26600 (UID FLAGS) (PARTIAL -1:-3)
S: * 12888 FETCH (FLAGS (\Seen) UID 25996)
S: * 12889 FETCH (FLAGS (\Flagged \Answered) UID 25997)
S: * 12890 FETCH (FLAGS () UID 26600)
S: 10 OK FETCH completed
// Returning information for the first 5 messages in the UID range
C: 11 UID FETCH 25900:26600 (UID FLAGS) (PARTIAL 1:5)
S: * 12591 FETCH (FLAGS (\Seen) UID 25900)
S: * 12592 FETCH (FLAGS (\Flagged) UID 25902)
S: * 12593 FETCH (FLAGS (\Answered) UID 26310)
S: * 12594 FETCH (FLAGS () UID 26311)
S: * 12595 FETCH (FLAGS (\Answered) UID 26498)
S: 11 OK FETCH completed
6. Expressing limits on a number of messages used in
SEARCH/FETCH/STORE/COPY/MOVE
// Should this be a separate IMAP capability?
// Also, should this also affect SEARCH? If yes, do we need a way to
// specify SEARCH criterion for "all UIDs after" or "all UIDs before"
// a specific UID?
If a server implementation doesn't allow more than <N> messages to be
operated on by a single FETCH/STORE/COPY/MOVE command, it MUST return
the MESSAGELIMIT response code defined below:
MESSAGELIMIT The server doesn't allow more than <N> messages to be operated
Melnikov, et al. Expires 25 April 2022 [Page 6]
Internet-Draft IMAP PARTIAL October 2021
on by a single SEARCH/FETCH/STORE/COPY/MOVE command. The last
processed UID is <LastUID>. The client needs to repeat the
operation for remaining messages.
C: 03 FETCH 10000:14589 (UID FLAGS)
S: * 14589 FETCH (FLAGS (\Seen) UID 25000)
S: * 14588 FETCH (FLAGS (\Answered) UID 24998)
S: ... further 997 fetch responses
S: * 13590 FETCH (FLAGS () UID 23221)
S: 03 OK [MESSAGELIMIT 1000 23221] FETCH completed with 1000 partial
results
In the above example the <N> value is 1000 and the last
processed UID <LastUID> is 23221.
7. Formal syntax
The following syntax specification uses the Augmented Backus-Naur
Form (ABNF) notation as specified in [ABNF].
Non-terminals referenced but not defined below are as defined by
IMAP4 [RFC3501].
Except as noted otherwise, all alphabetic characters are case-
insensitive. The use of upper or lower case characters to define
token strings is for editorial clarity only. Implementations MUST
accept these strings in a case-insensitive fashion.
SP = <Defined in RFC 5234>
MINUS = "-"
capability =/ "PARTIAL"
;; <capability> from [RFC3501]
modifier-partial = "PARTIAL" SP partial-range
partial-range-first = nz-number ":" nz-number
;; Request to search from oldest (lowest UIDs) to
;; more recent messages.
;; A range 500:400 is the same as 400:500.
;; This is similar to <seq-range> from [RFC3501],
;; but cannot contain "*".
partial-range-last = MINUS nz-number ":" MINUS nz-number
;; Request to search from newest (highest UIDs) to
;; oldest messages.
Melnikov, et al. Expires 25 April 2022 [Page 7]
Internet-Draft IMAP PARTIAL October 2021
;; A range -500:-400 is the same as -400:-500.
partial-range = partial-range-first / partial-range-last
search-return-opt =/ modifier-partial
;; All conform to <search-return-opt>, from [IMAP-ABNF]/[RFC9051]
search-return-data =/ ret-data-partial
ret-data-partial = "PARTIAL"
SP "(" partial-range SP partial-results ")"
;; <partial-range> is the requested range.
partial-results = sequence-set / "NIL"
;; <sequence-set> from [RFC3501].
;; NIL indicates no results correspond to the requested range.
tagged-ext-simple =/ partial-range-last
fetch-modifier =/ modifier-partial
resp-text-code =/ "MESSAGELIMIT" SP nz-number SP uniqueid
;; No more than nz-number messages can be processed
;; by any command at a time. The last processed
;; UID is uniqueid.
8. Security Considerations
TBD.
9. IANA Considerations
9.1. Changes/additions to the IMAP4 capabilities registry
IMAP4 capabilities are registered by publishing a standards track or
IESG approved Informational or Experimental RFC. The registry is
currently located at:
https://www.iana.org/assignments/imap4-capabilities
IANA is requested to add definition of the PARTIAL extension to point
to this document.
Melnikov, et al. Expires 25 April 2022 [Page 8]
Internet-Draft IMAP PARTIAL October 2021
10. Acknowledgments
This document was motivated by Yahoo! team and their questions about
best client practices for dealing with large mailboxes.
This document uses lots of text from RFC 5267. Thus work of the RFC
5267 authors Dave Cridland and Curtis King is appreciated.
11. Normative References
[ABNF] Crocker, D., Ed. and P. Overell, Ed., "Augmented BNF for
Syntax Specifications: ABNF", RFC 5234, January 2008,
<https://www.rfc-editor.org/info/rfc5234>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION
4rev1", RFC 3501, DOI 10.17487/RFC3501, March 2003,
<https://www.rfc-editor.org/info/rfc3501>.
[RFC4731] Melnikov, A. and D. Cridland, "IMAP4 Extension to SEARCH
Command for Controlling What Kind of Information Is
Returned", RFC 4731, DOI 10.17487/RFC4731, November 2006,
<https://www.rfc-editor.org/info/rfc4731>.
[RFC5267] Cridland, D. and C. King, "Contexts for IMAP4", RFC 5267,
DOI 10.17487/RFC5267, July 2008,
<https://www.rfc-editor.org/info/rfc5267>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC9051] Melnikov, A., Ed. and B. Leiba, Ed., "Internet Message
Access Protocol (IMAP) - Version 4rev2", RFC 9051,
DOI 10.17487/RFC9051, August 2021,
<https://www.rfc-editor.org/info/rfc9051>.
Index
M
M
MESSAGELIMIT (response code)
Melnikov, et al. Expires 25 April 2022 [Page 9]
Internet-Draft IMAP PARTIAL October 2021
Section 6, Paragraph 3.2.1
Authors' Addresses
Alexey Melnikov
Isode Limited
Email: alexey.melnikov@isode.com
URI: https://www.isode.com
Arun Prakash Achuthan
Yahoo!
Email: arunprakash@myyahoo.com
Vikram Nagulakonda
Yahoo!
Email: nvikram_imap@yahoo.com
Luis Alves
Verizon Media
Email: luis.alves@lafaspot.com
Melnikov, et al. Expires 25 April 2022 [Page 10]