IMAP Extension for Referencing the Last SEARCH Result
draft-melnikov-imap-search-res-07
The information below is for an old version of the document that is already published as an RFC.
| Document | Type | RFC Internet-Draft (individual in app area) | |
|---|---|---|---|
| Author | Alexey Melnikov | ||
| Last updated | 2018-12-20 (Latest revision 2008-01-28) | ||
| Stream | Internet Engineering Task Force (IETF) | ||
| Formats | plain text htmlized pdfized bibtex | ||
| Stream | WG state | (None) | |
| Document shepherd | (None) | ||
| IESG | IESG state | RFC 5182 (Proposed Standard) | |
| Consensus boilerplate | Unknown | ||
| Telechat date | (None) | ||
| Responsible AD | Chris Newman | ||
| Send notices to | (None) |
draft-melnikov-imap-search-res-07
Document: draft-melnikov-imap-search-res-07.txt A. Melnikov
Intended status: Proposed Standard Isode Ltd.
Updates: RFC 3501 (if approved) January 26, 2008
Expires: July 2008
Internet-Draft
IMAP extension for referencing the last SEARCH result
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of
six months and may be updated, replaced, or obsoleted by
other documents at any time. It is inappropriate to use
Internet-Drafts as reference material or to cite them other
than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/1id-abstracts.html
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
A revised version of this draft document will be submitted to the RFC
editor as a Proposed Standard for the Internet Community. Discussion
and suggestions for improvement are requested, and should be sent to
the IMAPEXT Mailing list <ietf-imapext@imc.org>. Distribution of this
draft is unlimited.
Copyright Notice
Copyright (C) The IETF Trust (2008).
Abstract
Many IMAP clients use the result of a SEARCH command as the input to
perform another operation, for example fetching the found messages,
deleting them or copying them to another mailbox.
Melnikov Expires: July 2008 [Page 1]
INTERNET DRAFT Last SEARCH result reference January 2008
This can be achieved using standard IMAP operations described in RFC
3501, however this would be suboptimal: the server will send the list
of found messages to the client, after that the client will have to
parse the list, reformat it and send it back to the server. The
client can't pipeline the SEARCH command with the subsequent command,
and, as a result, the server might not be able to perform some
optimizations.
This document proposes an IMAP extension that allows a client to tell
a server to use the result of a SEARCH (or UID SEARCH) command as an
input to any subsequent command.
1. Conventions Used in this Document
In examples, "C:" indicates lines sent by a client that is connected
to a server. "S:" indicates lines sent by the server to the client.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [KEYWORDS].
Explanatory comments in examples are starting with // and are not
part of the protocol.
<<Other editorial comments/questions are shown like this.>>
2. Introduction and Overview
Many IMAP clients use the result of a SEARCH command as the input to
perform another operation, for example fetching the found messages,
deleting them or copying them to another mailbox.
This document proposes an IMAP extension that allows a client to tell
a server to use the result of a SEARCH (or UID SEARCH) command as an
input to any subsequent command.
The SEARCH result reference extension defines a new SEARCH result
option [IMAPABNF] "SAVE" that tells the server to remember the result
of the SEARCH or UID SEARCH command (as well as any command based on
SEARCH, e.g. SORT and THREAD [SORT]) and store it in an internal
variable that we will reference as the "search result variable". The
client can use the "$" marker to reference the content of this
internal variable. The "$" marker can be used instead of message
sequence or UID sequence in order to indicate that the server should
substitute it with the list of messages from the search result
variable. Thus the client can use the result of the latest
Melnikov Expires: July 2008 [Page 2]
INTERNET DRAFT Last SEARCH result reference January 2008
remembered SEARCH command as a parameter to another command. The
search result marker has several advantages:
* it avoids wasted bandwidth and associated delay;
* it allows the client to pipeline a SEARCH [IMAP4] command with a
subsequent FETCH/STORE/COPY/SEARCH [IMAP4] or UID EXPUNGE
[UIDPLUS] command;
* the client doesn't need to spend time reformatting the
result of a SEARCH command into a message set used in
the subsequent command;
* it allows the server to perform optimizations. For example if
the server can execute several pipelined commands in parallel
(or out of order), presence of the search result marker can
allow the server to decide which commands may or may not be
executed out of order.
In absence of any other SEARCH result option, the SAVE result option
also suppresses any SEARCH response that would have been otherwise
returned by the SEARCH command.
2.1. Normative description of the SEARCHRES extension
The SEARCH result reference extension described in this document is
present in any IMAP4 server implementation which returns "SEARCHRES"
as one of the supported capabilities in the CAPABILITY command
response. Any such server MUST also implement the [ESEARCH]
extension.
Upon successful completion of a SELECT or an EXAMINE command (after
the tagged OK response), the current search result variable is reset
to the empty sequence.
A successful SEARCH command with the SAVE result option sets the
value of the search result variable to the list of messages found in
the SEARCH command. For example, if no messages were found, the
search result variable will contain the empty list.
Any of the following SEARCH command MUST NOT change the search result
variable:
o A SEARCH command that caused the server to return the BAD tagged
response,
o a SEARCH command with no SAVE result option that caused the server
to return NO tagged response,
o a successful SEARCH command with no SAVE result option.
A SEARCH command with the SAVE result option that caused the server
to return the NO tagged response sets the value of the search result
Melnikov Expires: July 2008 [Page 3]
INTERNET DRAFT Last SEARCH result reference January 2008
variable to the empty sequence.
When a message listed in the search result variable is EXPUNGEd, it
is automatically removed from the list. Implementors are reminded
that if the server stores the list as a list of message numbers, it
MUST automatically adjust them when notifying the client about
expunged messages, as described in Section 7.4.1 of [IMAP4].
If the server decides to send a new UIDVALIDITY value while mailbox
is opened, this causes resetting of the search variable to the empty
list.
Note that even if the "$" marker contains the empty list of messages,
it must be treated by all commands accepting message sets as
parameters, as a valid, but non matching list of messages. For
example, the "FETCH $" command would return tagged OK response and no
FETCH responses. See also the Example # 5 below.
Note that even if the "$" marker contains the empty list of messages,
it must be treated as a valid but non matching list of messages, by
all commands that accept message sets as parameters.
Implementation note: server implementers should note that "$" can
reference IMAP message sequence or UID sequence, depending on context
where it is used. For example, the "$" marker can be set as a result
of a SEARCH (SAVE) command and used as a parameter to a UID FETCH
command (which accept UID sequence, not message sequence), or the "$"
marker can be set as a result of a UID SEARCH (SAVE) command and used
as a parameter to a FETCH command (which accept message sequence, not
UID sequence).
2.2. Examples
1) The following example demonstrates how the client can use the
result of a SEARCH command to FETCH headers of interesting
messages:
Example 1:
C: A282 SEARCH RETURN (SAVE) FLAGGED SINCE 1-Feb-1994
NOT FROM "Smith"
S: A282 OK SEARCH completed, result saved
C: A283 FETCH $ (UID INTERNALDATE FLAGS RFC822.HEADER)
S: * 2 FETCH (UID 14 ...
S: * 84 FETCH (UID 100 ...
S: * 882 FETCH (UID 1115 ...
S: A283 OK completed
The client can also pipeline the two commands:
Melnikov Expires: July 2008 [Page 4]
INTERNET DRAFT Last SEARCH result reference January 2008
Example 2:
C: A282 SEARCH RETURN (SAVE) FLAGGED SINCE 1-Feb-1994
NOT FROM "Smith"
C: A283 FETCH $ (UID INTERNALDATE FLAGS RFC822.HEADER)
S: A282 OK SEARCH completed
S: * 2 FETCH (UID 14 ...
S: * 84 FETCH (UID 100 ...
S: * 882 FETCH (UID 1115 ...
S: A283 OK completed
2) The following example demonstrates that the result of one SEARCH
command can be used as input to another SEARCH command:
Example 3:
C: A300 SEARCH RETURN (SAVE) SINCE 1-Jan-2004
NOT FROM "Smith"
S: A300 OK SEARCH completed
C: A301 UID SEARCH UID $ SMALLER 4096
S: * SEARCH 17 900 901
S: A301 OK completed
Note that the second command in Example 3 can be replaced with:
C: A301 UID SEARCH $ SMALLER 4096
and the result of the command would be the same.
3) The following example shows that the "$"
marker can be combined with other message numbers using the OR
SEARCH criterion.
Example 4:
C: P282 SEARCH RETURN (SAVE) SINCE 1-Feb-1994
NOT FROM "Smith"
S: P282 OK SEARCH completed
C: P283 SEARCH CHARSET UTF-8 (OR $ 1,3000:3021) TEXT {8}
C: YYYYYYYY
S: * SEARCH 882 1102 3003 3005 3006
S: P283 OK completed
Note: Since this document format is restricted to 7-bit ASCII
text, it is not possible to show actual UTF-8 data. The
"YYYYYYYY" is a placeholder for what would be 8 octets of 8-bit
data in an actual transaction.
4) The following example demonstrates that a failed SEARCH sets
the search result variable to the empty list.
Example 5:
C: B282 SEARCH RETURN (SAVE) SINCE 1-Feb-1994
Melnikov Expires: July 2008 [Page 5]
INTERNET DRAFT Last SEARCH result reference January 2008
NOT FROM "Smith"
S: B282 OK SEARCH completed
C: B283 SEARCH CHARSET KOI8-R (OR $ 1,3000:3021) TEXT {4}
C: XXXX
S: B283 NO [BADCHARSET UTF-8] KOI8-R is not supported
//After this command the saved result variable contains
//no messages. A client that wants to reissue the B283
//SEARCH command with another CHARSET would have to reissue
//the B282 command as well. One possible workaround for
//this is to include the desired CHARSET parameter
//in the earliest SEARCH RETURN (SAVE) command in a
//sequence of related SEARCH commands.
//A better approach might be to always use CHARSET UTF-8
//instead.
Note: Since this document format is restricted to 7-bit ASCII
text, it is not possible to show actual KOI8-R data. The "XXXX"
is a placeholder for what would be 4 octets of 8-bit data in an
actual transaction.
5) The following example demonstrates that it is not an error to
use the "$" marker when it contains no messages.
Example 6:
C: E282 SEARCH RETURN (SAVE) SINCE 28-Oct-2006
NOT FROM "Eric"
C: E283 COPY $ "Other Messages"
//The "$" contains no messages
S: E282 OK SEARCH completed
S: E283 OK COPY completed, nothing copied
2.3. Multiple Commands in Progress
Use of a SEARCH RETURN (SAVE) command followed by a command
using the "$" marker creates direct dependency between the two
commands. As directed by Section 5.5 of [IMAP4], a server MUST
execute the two commands in the order they were received. (A
server capable of out-of-order execution can in some cases
execute the two commands in parallel, for example if a SEARCH
RETURN (SAVE) is followed by SEARCH $, the search criteria from
the first command can be directly substituted into the second
command.)
A client supporting this extension MAY pipeline a SEARCH RETURN
(SAVE) command with one or more command using the "$" marker, as
long as this doesn't create an ambiguity described in Section
5.5 of [IMAP4].
Melnikov Expires: July 2008 [Page 6]
INTERNET DRAFT Last SEARCH result reference January 2008
Example 7:
C: F282 SEARCH RETURN (SAVE) KEYWORD $Junk
C: F283 COPY $ "Junk"
C: F284 STORE $ +FLAGS.Silent (\Deleted)
S: F282 OK SEARCH completed
S: F283 OK COPY completed
S: F284 OK STORE completed
Example 8:
C: G282 SEARCH RETURN (SAVE) KEYWORD $Junk
C: G283 SEARCH RETURN (ALL) SINCE 28-Oct-2006
FROM "Eric"
//The server can execute the two SEARCH commands
//in any order, as they don't have any dependency.
//Note that the second command is making use of
//the [ESEARCH] extension.
S: * ESEARCH (TAG "G283") ALL 3:15,27,29:103
S: G283 OK SEARCH completed
S: G282 OK SEARCH completed
The following example demonstrates that the result of the second
SEARCH always overrides the result of the first.
Example 9:
C: H282 SEARCH RETURN (SAVE) KEYWORD $Junk
C: H283 SEARCH RETURN (SAVE) SINCE 28-Oct-2006
FROM "Eric"
S: H283 OK SEARCH completed
S: H282 OK SEARCH completed
2.4. Interaction with ESEARCH extension
Servers that implement the extension defined in this document
are required to implement [ESEARCH] and to conform to additional
requirements listed in this section.
The SAVE result option doesn't change whether the server would
return items corresponding to MIN, MAX, ALL or COUNT [ESEARCH]
result options.
When the SAVE result option is combined with the MIN or MAX
[ESEARCH] result option, and none of the other ESEARCH result
options are present, the corresponding MIN/MAX is returned (if
the search result is not empty) but the "$" marker would contain
a single message as returned in the MIN/MAX return item.
If the SAVE result option is combined with both MIN and MAX
Melnikov Expires: July 2008 [Page 7]
INTERNET DRAFT Last SEARCH result reference January 2008
result options, and none of the other ESEARCH result options are
present, the "$" marker would contain one or two messages as
returned in the MIN/MAX return items.
If the SAVE result option is combined with ALL and/or COUNT
result option, the "$" marker would always contain all messages
found by the SEARCH or UID SEARCH command. (Note that the last
rule might affect ESEARCH implementations that optimize how
COUNT result is constructed.)
The following table summarizes the additional requirement on
ESEARCH server implementations described in this section.
+----------------+-------------------+
| Combination of | "$" marker value |
| Result option | |
+----------------+-------------------+
| SAVE MIN | MIN |
+----------------+-------------------+
| SAVE MAX | MAX |
+----------------+-------------------+
| SAVE MIN MAX | MIN & MAX |
+----------------+-------------------+
| SAVE * [m] | all found messages|
+----------------+-------------------+
where '*' means "ALL" and/or "COUNT"
'[m]' means optional "MIN" and/or "MAX"
The following example demonstrates behavioural difference
for different combinations of ESEARCH result options.
Explanatory comments are starting with // and are not
part of the protocol:
Example 10:
C: C282 SEARCH RETURN (ALL) SINCE 12-Feb-2006
NOT FROM "Smith"
S: * ESEARCH (TAG "C283") ALL 2,10:15,21
//$ value hasn't changed
S: C282 OK SEARCH completed
C: C283 SEARCH RETURN (ALL SAVE) SINCE 12-Feb-2006
NOT FROM "Smith"
S: * ESEARCH (TAG "C283") ALL 2,10:15,21
//$ value is 2,10:15,21
S: C283 OK SEARCH completed
Melnikov Expires: July 2008 [Page 8]
INTERNET DRAFT Last SEARCH result reference January 2008
C: C284 SEARCH RETURN (SAVE MIN) SINCE 12-Feb-2006
NOT FROM "Smith"
S: * ESEARCH (TAG "C284") MIN 2
//$ value is 2
S: C284 OK SEARCH completed
C: C285 SEARCH RETURN (MAX SAVE MIN) SINCE
12-Feb-2006 NOT FROM "Smith"
S: * ESEARCH (TAG "C285") MIN 2 MAX 21
//$ value is 2,21
S: C285 OK SEARCH completed
C: C286 SEARCH RETURN (MAX SAVE MIN COUNT)
SINCE 12-Feb-2006 NOT FROM "Smith"
S: * ESEARCH (TAG "C286") MIN 2 MAX 21 COUNT 8
//$ value is 2,10:15,21
S: C286 OK SEARCH completed
C: C286 SEARCH RETURN (ALL SAVE MIN) SINCE
12-Feb-2006 NOT FROM "Smith"
S: * ESEARCH (TAG "C286") MIN 2 ALL 2,10:15,21
//$ value is 2,10:15,21
S: C286 OK SEARCH completed
2.5. Refusing to save search results
In some cases, the server MAY refuse to save a SEARCH (SAVE)
result, for example if an internal limit on the number of saved
results is reached.
In this case, the server MUST return a tagged NO response
containing the NOTSAVED response code, and set the search result
variable to the empty sequence, as described in Section 2.1.
3. Formal Syntax
The following syntax specification uses the Augmented Backus-Naur
Form (ABNF) notation as specified in [ABNF]. Non-terminals referenced
but not defined below are as defined in [IMAP4] or [IMAPABNF].
Except as noted otherwise, all alphabetic characters are case-
insensitive. The use of upper or lower case characters to define
token strings is for editorial clarity only. Implementations MUST
accept these strings in a case-insensitive fashion.
capability =/ "SEARCHRES"
Melnikov Expires: July 2008 [Page 9]
INTERNET DRAFT Last SEARCH result reference January 2008
;; capability is defined in [IMAP4]
sequence-set =/ seq-last-command
;; extends sequence-set to allow for
;; "result of the last command" indicator.
seq-last-command = "$"
search-return-opt = "SAVE"
;; conforms to generic search-return-opt
;; syntax defined in [IMAPABNF]
resp-text-code =/ "NOTSAVED"
;; <resp-text-code> from [IMAP4]
4. Security Considerations
This extension requires the server to keep additional state, that may
be used to simplify Deny of Service attacks. In order to minimize
damage from such attacks server implementations MAY limit the number
of saved searches they allow across all connections at any given time
and return the tagged NO response containing the NOTSAVED response
code (see Section 2.5) to a SEARCH RETURN (SAVE) command when this
limit is exceeded.
Apart from that it is believed that this extension doesn't raise any
additional security concerns not already discussed in [IMAP4].
5. IANA Considerations
This document defines the "SEARCHRES" IMAP capability. IANA is
requested to add it to the IMAP4 Capabilities Registry, which is
currently located at:
http://www.iana.org/assignments/imap4-capabilities
6. Acknowledgments
The author would like to thank Mark Crispin, Cyrus Daboo and Curtis
King for reminding that this document has to be written, as well as
for comments and corrections received.
The author would also like to thank Dave Cridland, Mark Crispin,
Chris Newman, Dan Karp and Spencer Dawkins for comments and
corrections received.
Melnikov Expires: July 2008 [Page 10]
INTERNET DRAFT Last SEARCH result reference January 2008
Valuable comments, both in agreement and in dissent, were received
from Arnt Gulbrandsen.
7. References
7.1. Normative references
[KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[ABNF] Crocker, D., and Overell, P. "Augmented BNF for Syntax
Specifications: ABNF", RFC 4234, October 2005.
[IMAP4] Crispin, M., "Internet Message Access Protocol - Version
4rev1", RFC 3501, University of Washington, March 2003.
[IMAPABNF] Melnikov, A., and C. Daboo, "Collected extensions to IMAP4
ABNF", RFC 4466, April 2006.
[ESEARCH] Melnikov, A., and D. Cridland, "IMAP4 Extension to SEARCH
Command for Controlling What Kind of Information Is Returned", RFC
4731, November 2006.
7.2. Informative references
[UIDPLUS] Crispin, M., "Internet Message Access Protocol (IMAP) -
UIDPLUS extension", RFC 4315, December 2005.
[SORT] Crispin, M. and K. Murchison, "INTERNET MESSAGE ACCESS
PROTOCOL - SORT AND THREAD EXTENSIONS", work in progress, draft-ietf-
imapext-sort-XX.txt
8. Author's Addresses
Alexey Melnikov
Isode Ltd.
5 Castle Business Village,
36 Station Road,
Hampton, Middlesex,
TW12 2BX, United Kingdom
Email: Alexey.Melnikov@isode.com
9. Full Copyright Statement
Copyright (C) The IETF Trust (2008).
Melnikov Expires: July 2008 [Page 11]
INTERNET DRAFT Last SEARCH result reference January 2008
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is provided by the IETF
Administrative Support Activity (IASA).
10. Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Melnikov Expires: July 2008 [Page 12]