INTERNET-DRAFT S. Varshavchik
Expires Jan 12, 2000 Double Precision, Inc.
Jul 12, 1999
Variable Envelope Return Path SMTP Extension
draft-varshavchik-verp-smtpext-00.txt
Status Of This Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet- Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
1. Abstract
This document describes an extension to the SMTP service [1], called
Variable Envelope Return Path (VERP). The VERP extension implements
a way of automatically identifying undeliverable mail recipients,
even when non-delivery reports originate from mail systems that do
not implement delivery status notifications as specified in [2] and
[3].
2. Introduction
All E-mail software can expect to deal with undeliverable mail. [2]
and [3] implement a way for undeliverable mail to be handled in a
completely automatic fashion, without requiring manual intervention.
For example, mailing list managers can automatically identify
addresses that are no longer deliverable, and remove them from the
mailing list.
Although [2] and [3] are widely implemented, there are still a lot of
S. Varshavchik Expires Jan 12, 2000 [Page 1]
VERP SMTP Extension S. Varshavchik Jul 12, 2000
systems that do not use them. This translates into a non-trivial
amount of manual work, to identify undeliverable addresses and remove
them from the mailing list. Even when the percentage of
undeliverable addresses starts out rather small, over time they
accumulate to the point of requiring manual intervention.
VERPs represent an alternative way of handling non-delivery notices.
The advantage of VERPs is that they work every time, even when non-
delivery notices are not in the form specified by [2]. In a VERP,
the recipient address is encoded into a portion of the return path.
When undeliverable mail comes back, the mail software decodes the
return address and obtains the address responsible for the non-
delivery notice.
For example, mail sent by a mailing list manager to the address
<john@example.org> carries a return address of <mlist-return-
john=example.org@domain.com>. The mail system for domain.com knows
that all mail with the local address starting with "mlist-return-"
must go to the mailing list manager. The mailing list manager takes
the return address, retrieves the remaining portion of the local
address, "john=example.org", and determines that the undeliverable
address was <john@example.org>.
This does not rely on RFC 1894, and will work for all non-delivery
notices.
Unfortunately, VERPs have a known drawback when used with large
mailing lists: an individual copy of each message must be sent to
every individual recipient. It is no longer possible to conserve
network resources by transmitting only one copy of each message,
addressed to every recipient in the same domain (or a couple of
messages where the number of recipients in the same domain is very
large). A separate message must be sent to every recipient when
VERPs are used, because the recipient's address must to be encoded as
a part of the return path.
This document specifies an SMTP service extension that enables mail
systems to exchange message with variable envelope return paths,
without transmitting one message per recipient.
3. Framework for the VERP SMTP transport extension
This SMTP transport extension [1] is laid out as follows.
(1) The name of the SMTP transport extension defined here is
Variable Envelope Return Path.
(2) The EHLO keyword associated with this extension is VERP.
S. Varshavchik Expires Jan 12, 2000 [Page 2]
VERP SMTP Extension S. Varshavchik Jul 12, 2000
(3) The VERP EHLO keyword takes no parameters.
(4) One optional ESMTP keyword VERP is associated with the MAIL
FROM command. This parameter takes no values.
(5) No additional ESMTP verbs are defined by this extension.
(6) The next section specifies how support for this extension
affects the behavior of a server and client SMTP.
4. The VERP SMTP extension
When a VERP keyword is present in the MAIL FROM command, [4],
additional restrictions are imposed on the RFC 822 address [5],
specified by that MAIL FROM command, and on all RFC 822 addresses in
the subsequent RCPT TO commands that refer to the same message (that
is, until the next DATA, RSET, or QUIT command). The term "VERP
message" refers to any E-mail message whose MAIL FROM command
includes the VERP keyword. The term "VERP-compliant server" refers
to any E-mail server that supports the Variable Envelope Return Path
SMTP extension. When a VERP keyword is present in the MAIL FROM
command:
(1) The address specified by the MAIL FROM verb MUST contain at
least one @ character.
(2) The address in every RCPT TO verb referring to the same
message MUST contain at least one @ character.
(3) The domain portion of the address in the MAIL FROM and RCPT
TO verbs MUST be compliant with the definition of <domain> in
[6]. That is, it MUST contain only letters, digits, hyphens,
and periods. The domain portion is the portion of the
address that follows the last @ character,
4.1 Delivery failures
When a VERP-compliant server is unable to deliver a VERP message to
one or more of its recipients, the VERP server MUST do one of the
following:
1) Return an RFC 1891 delivery status notification to the return
address, or:
2) Transmit a separate non-delivery notice for each failed
recipient. The return address for each non-delivery notice
MUST be the address that's formed by applying the procedure
described in section 7 of this document to the return address
S. Varshavchik Expires Jan 12, 2000 [Page 3]
VERP SMTP Extension S. Varshavchik Jul 12, 2000
of the message and the failed recipient's address. If more
than one recipient failed, a separate notice MUST be sent for
each undeliverable address.
5. Final delivery
A VERP-compliant server may have locally-defined conventions which
records the return address in each message, for informational
purposes.
If this is the case, the recorded return address MUST be formed by
applying the procedure described in section 7 of this document to the
return address and the recipient's address.
6. Relaying
When a VERP-compliant server determines that a recipient of a VERP
message is not a local mailbox, and the message must be relayed to
another server, the VERP-compliant server MUST:
(1) If the VERP-compliant server's local policies require the
return and/or recipient addresses to be rewritten, any
rewritten addresses MUST have at least one @ character.
(2) If the VERP-compliant server determines that the remote
server is also a VERP compliant server, the VERP keyword MUST
be included in the MAIL FROM command used to relay the VERP
message to the remote server.
(3) If the remote server is not a VERP compliant server, The VERP
compliant server SHOULD send a separate copy of the VERP
message for every recipient, and the return address of each
message MUST be formed by applying the procedure described in
section 7 of this document to the original return address,
and the address of each recipient. Alternatively, the
message SHOULD NOT be returned as undeliverable. If it is,
the rules defined in section 4.1 MUST be applied.
This also applies if the SMTP-compliant server determines
that the VERP message is to be forwarded via some other
protocol to a non-SMTP gateway, unless the non-SMTP protocol
has equivalent features that are completely identical in
function to Variable Envelope Return Path SMTP service
extension (including any translations of E-mail addresses to
and from the non-RFC822 format).
This SMTP service extensions allows E-mail software to send a single
VERP message to all addresses the same mail domain, as long as all
S. Varshavchik Expires Jan 12, 2000 [Page 4]
VERP SMTP Extension S. Varshavchik Jul 12, 2000
mail servers used to deliver the message support the Variable
Envelope Return Path SMTP extension. When a VERP message reaches a
non VERP-compliant server, a separate message with a variable
envelope return path is generated for each recipient.
7. Variable envelope return path encoding
This encoding method starts with one return address and one recipient
address. As mentioned previously, both addresses MUST be valid
RFC822 addresses, [5], and MUST contain at least one @ character.
The portion of each address following the last @ character MUST be
compliant with [6].
Let "sdomain" represent the portion of the return address that
follows the last @ character.
Let "slocal" represent the portion of the return address that
precedes the last @ character.
Let "rdomain" represent the portion of the recipient address that
follows the last @ character.
Let "rlocal" represent the portion of the recipient address that
precedes the last @ character.
To encode the recipient address within the envelope sender address,
create an address of the following form:
slocal-encodedrlocal=rdomain@sdomain
Where "encodedrlocal" is formed by taking rlocal and encoding it as
follows:
1) Each @, :, %, !, and + character in rlocal is replaced by a
single '+' character followed by two uppercase hexadecimal
characters whose value is the ASCII code of the replaced
character.
2) All other characters are unchanged. Other characters MAY,
but SHOULD NOT be also encoded in the same fashion.
This can be represented using BNF as follows:
encodedverp: slocal "-" encodedrlocal "=" rdomain "@" sdomain
encodedrlocal: * (char-literal / char-encoded )
char-literal: any character valid in an RFC822 address [5],
S. Varshavchik Expires Jan 12, 2000 [Page 5]
VERP SMTP Extension S. Varshavchik Jul 12, 2000
except @, :, %, !, and +
char-encoded: "+" hexdigit hexdigit
hexdigit: ("0" / "1" / "2" / "3" / "4" / "5" / "6" / "7" / "8" /
"9" / "A" / "B" / "C" / "D" / "E" / "F" )
8. Variable envelope return path decoding
Non-delivery notices for VERP messages will be sent to either the
original address, <slocal@sdomain>, or to the VERP-encoded address,
<slocal-encodedrlocal=rdomain@sdomain>.
Messages sent to <slocal@sdomain> will be RFC 1891-compliant delivery
status notifications. These messages will be machine-readable, and
the mail software will be able to identify failed addresses from the
RFC 1891 delivery report. Non-delivery notices will also be sent to
the VERP-encoded address, and the mail software will be able to
reconstruct the failed address from the VERP-encoded address by
simply reversing the steps used in encoding:
1) Extracting encodedrlocal and rdomain from the recipient
address. There will be at least one = character in the
encoded portion of the return address. encodedrlocal is
everything up to the last = character. Everything following
the last = character is rdomain.
2) Replacing all occurrences of "+" followed by two hexadecimal
digits in encodedrlocal with the equivalent ASCII character.
3) Using the decoded rlocal, @, then rdomain.
9. Examples
Suppose that a VERP-compliant server named "example.com" receives a
message with the following SMTP conversation (for brevity, non-
relevant headers have been omitted):
250 example.com ESMTP
EHLO domain.com
250-example.com ESMTP
250-SIZE
250-DSN
250-VERP
250 HELP
MAIL FROM:<itny-out@domain.com> VERP SIZE=100
250 Ok
RCPT TO:<alex@example.com>
S. Varshavchik Expires Jan 12, 2000 [Page 6]
VERP SMTP Extension S. Varshavchik Jul 12, 2000
250 Ok
RCPT TO:<node42!ann@old.example.com>
250 Ok
RCPT TO:<tom@old.example.com>
250 Ok
RCPT TO:<lisa@new.example.com>
250 Ok
RCPT TO:<dave+priority@new.example.com>
250 Ok
DATA
250 Ok
From: "John" <john@domain.com>
Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST)
Subject: Meeting canceled.
Today's 2pm meeting has been rescheduled for tomorrow, 9am, due
to a scheduling conflict.
.
The message is delivered to the local mailbox for <alex@example.com>.
The message looks like this:
Return-Path: <itny-out-alex=example.com@domain.com>
From: "John" <john@domain.com>
Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST)
Subject: Meeting canceled.
Today's 2pm meeting has been rescheduled for tomorrow, 9am, due
to a scheduling conflict.
The VERP-compliant server at example.com connects to the mail server
for old.example.com. old.example.com does not support the Variable
Envelope Return Path extension. Therefore, old.example.com receives
two messages. The SMTP conversation for the first message is as
follows:
250 old.example.com ESMTP
EHLO example.com
250-old.example.com ESMTP
250-SIZE
250-DSN
250 HELP
MAIL FROM:<itny-out-node42+21ann=old.example.com@domain.com>
250 Ok
RCPT TO:<node42!ann@old.example.com>
250 Ok
DATA
250 Ok
S. Varshavchik Expires Jan 12, 2000 [Page 7]
VERP SMTP Extension S. Varshavchik Jul 12, 2000
From: "John" <john@domain.com>
Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST)
Subject: Meeting canceled.
Today's 2pm meeting has been rescheduled for tomorrow, 9am, due
to a scheduling conflict.
.
The SMTP conversation for the second message is as follows:
MAIL FROM:<itny-out-tom=old.example.com@domain.com>
250 Ok
RCPT TO:<tom@old.example.com>
250 Ok
DATA
250 Ok
From: "John" <john@domain.com>
Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST)
Subject: Meeting canceled.
Today's 2pm meeting has been rescheduled for tomorrow, 9am, due
to a scheduling conflict.
.
example.com connects to new.example.com and determines that
new.example.com runs a modern ESMTP server that supports the VERP
keyword. The SMTP conversation then goes like this:
250 new.example.com ESMTP
EHLO example.com
250-new.example.com ESMTP
250-SIZE
250-DSN
250-VERP
250 HELP
MAIL FROM:<itny-out@domain.com> VERP SIZE=100
250 Ok
RCPT TO:<lisa@new.example.com>
250 Ok
RCPT TO:<dave+priority@new.example.com>
250 Ok
DATA
250 Ok
From: "John" <john@domain.com>
Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST)
Subject: Meeting canceled.
Today's 2pm meeting has been rescheduled for tomorrow, 9am, due
S. Varshavchik Expires Jan 12, 2000 [Page 8]
VERP SMTP Extension S. Varshavchik Jul 12, 2000
to a scheduling conflict.
.
10. Security concerns
All the usual security considerations applicable to SMTP are also
applicable to this extension. Relay of VERP messages to non-VERP
servers requires a single message with many recipients to be exploded
into many messages with one recipient. In all cases, however, there
will never be any additional overhead beyond the resources that are
required when variable envelope return paths are manually implemented
by the mail sender, instead of using the VERP SMTP extension.
Mail systems which support the VERP extension SHOULD have adequate
security measures, including blocks against unauthorized access and
relaying.
S. Varshavchik Expires Jan 12, 2000 [Page 9]
VERP SMTP Extension S. Varshavchik Jul 12, 2000
11. References
[1] Klensin, J., Freed, N., Rose, M., Stefferud, E., Crocker, D.
"SMTP Service Extensions", RFC 1425, United Nations
University, Innosoft International, Inc., Dover Beach
Consulting, Inc., Network Management Associates, Inc., The
Branch Office, February 1993
[2] Moore, K., and G. Vaudreuil, "An Extensible Message Format
for Delivery Status Notifications", RFC 1894, University of
Tennessee, Octel Network Services, January 1996.
[3] Moore, K. "SMTP Service Extension for Delivery Status
Notifications", RFC 1891, University of Tennessee, January
1996.
[4] Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821,
USC/Information Sciences Institute, August 1982.
[5] Crocker, D., "Standard for the Format of ARPA Internet Text
Messages", STD 11, RFC 822, UDEL, August 1982.
[6] Mockapetris, P., "Domain Names - Implementation and
Specification", RFC 1035, ISI, November 1987
12. Author's address
Sam Varshavchik
Double Precision, Inc.
PO Box 668
Greenwood Lake, NY 10925
<mrsam@concentric.net>
S. Varshavchik Expires Jan 12, 2000 [Page 10]