Network Working Group J. Vinocur
INTERNET DRAFT Cornell University
Document: draft-ietf-nntpext-streaming-00.txt
June 2003
NNTP Extension for Streaming Feeds
Status of this memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC 2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as "work in
progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.html.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Copyright Notice
Copyright (C) The Internet Society (2002). All Rights Reserved.
Abstract
This memo defines an extension to the Network News Transport
Protocol [NNTP] to provide asynchronous transfer of articles. This
allows servers to transfer articles to other servers with much
greater efficiency.
Table of Contents
1. Introduction ............................................. 2
1.1. Conventions Used in this Document ................... 2
2. Advertising Capabilities with the Extensions Mechanism ... 3
3. MODE STREAM Command ...................................... 3
Vinocur Expires December 2003 [Page 1]
Internet Draft NNTP Extension for Streaming Feeds June 2003
3.1. Usage ............................................... 3
3.2. Description ......................................... 3
3.3. Examples ............................................ 4
4. CHECK Command ............................................ 4
4.1. Usage ............................................... 4
4.2. Description ......................................... 4
4.3. Examples ............................................ 5
5. TAKETHIS Command ......................................... 5
5.1. Usage ............................................... 5
5.2. Description ......................................... 5
5.3. Examples ............................................ 6
6. Augmented BNF Syntax for Streaming ....................... 7
7. Security Considerations .................................. 7
8. Acknowledgements ......................................... 7
9. Normative References ..................................... 7
10. Informative References .................................. 7
11. Author's Address ........................................ 7
1. Introduction
According to the NNTP specification [NNTP], a peer uses the IHAVE
command to query whether a server wants a particular article.
Because the IHAVE command cannot be pipelined, the need to stop and
wait for the remote end's response greatly restricts the throughput
that can be achieved. The alternative method of server-to-server
article transfer described in this document permits a more
consistent use of network bandwidth.
This document formalizes extensions previously described [NNTP-
COMMON] in accordance with the NNTP extension mechanism [NNTP].
1.1. Conventions Used in this Document
The key words "REQUIRED", "MUST", "MUST NOT", "SHOULD", "SHOULD
NOT", "MAY", and "OPTIONAL" in this document are to be interpreted
as described in "Key words for use in RFCs to Indicate Requirement
Levels" [KEYWORDS].
This document assumes you are familiar with NNTP [NNTP]. In
general, the connections described below are from one server to
another (i.e., feeding articles to a peer), but we will continue to
use "client" to mean the initiator of the NNTP connection, and
"server" to mean the other endpoint.
A variety of response codes are applicable to all NNTP commands
(see section 3.2 of [NNTP]). Clients MUST be prepared for the
commands defined in this document to return these generic response
codes when appropriate, even if they are not explicitly mentioned
Vinocur Expires December 2003 [Page 2]
Internet Draft NNTP Extension for Streaming Feeds June 2003
below.
In the examples, commands from the client are indicated with [C],
and responses from the server are indicated with [S].
2. Advertising Capabilities with the Extensions Mechanism
The "LIST EXTENSIONS" command, documented in section 8 of [NNTP],
provides a mechanism for clients to discover what extensions are
available.
A server supporting the streaming commands described in this
document will advertise the "STREAMING" capability in response to
the "LIST EXTENSIONS" command.
Example:
[C] LIST EXTENSIONS
[S] 202 Extensions supported:
[S] HDR
[S] STREAMING
[S] .
3. MODE STREAM Command
3.1. Usage
This command MUST NOT be pipelined.
Syntax
MODE STREAM
Responses
203 Streaming permitted
500 Unknown command [1]
501 Unknown MODE option [1]
501 Command syntax error
502 Permission denied
[1] Note the distinction in section 3.2.1 of [NNTP] between the
responses for MODE being entirely unimplemented, and MODE being
implemented but without the MODE STREAM variant.
3.2. Description
A client issues the MODE STREAM command to indicate that it wishes
to use the CHECK and TAKETHIS commands instead of, or in addition
to, IHAVE. The client MUST NOT send any additional commands on the
socket until after it has received the server response to the
Vinocur Expires December 2003 [Page 3]
Internet Draft NNTP Extension for Streaming Feeds June 2003
command; this command MUST NOT be pipelined as per section 3.2 of
[NNTP].
3.3. Examples
Example of a client requesting the ability to stream articles from
a server which does not support this extension:
[C] MODE STREAM
[S] 501 Unknown MODE variant
Example of a client successfully requesting the ability to stream
articles:
[C] MODE STREAM
[S] 203 Streaming permitted
4. CHECK Command
4.1. Usage
Syntax
CHECK message-id
Responses
238 message-id Send article to be transferred
400 Service temporarily unavailable
431 message-id Transfer not possible; try again later
438 message-id Article not wanted
480 Permission denied
The 238, 431, and 438 responses MUST contain only the response
code followed by the message-id of the article in question;
free-form text is not permitted.
Parameters
message-id = Article message-id
4.2. Description
A client issues the CHECK command to discover if the server would
like for the article with the specified message-id to be sent using
the TAKETHIS command. This command MAY be pipelined; in
particular, clients often issue many CHECK commands in sequence and
use the responses to construct a list of articles to be sent with
the TAKETHIS command.
The use of the CHECK command for streaming is optional, as
described in section 5.2 below.
Vinocur Expires December 2003 [Page 4]
Internet Draft NNTP Extension for Streaming Feeds June 2003
4.3. Examples
Example of a client checking whether the server would like a number
of articles and getting a mixture of responses:
[C] CHECK <i.am.an.article.you.will.want@example.com>
[S] 238 <i.am.an.article.you.will.want@example.com>
[C] CHECK <i.am.an.article.you.have@example.com>
[S] 438 <i.am.an.article.you.have@example.com>
[C] CHECK <i.am.an.article.you.defer@example.com>
[S] 431 <i.am.an.article.you.defer@example.com>
The same example, but demonstrating streaming:
[C] CHECK <i.am.an.article.you.will.want@example.com>
[C] CHECK <i.am.an.article.you.have@example.com>
[C] CHECK <i.am.an.article.you.defer@example.com>
[S] 238 <i.am.an.article.you.will.want@example.com>
[S] 438 <i.am.an.article.you.have@example.com>
[S] 431 <i.am.an.article.you.defer@example.com>
5. TAKETHIS Command
5.1. Usage
Syntax
TAKETHIS message-id
[the entire article MUST be sent at this point, without prompting]
Responses
239 message-id Article transferred OK
400 Service temporarily unavailable
439 message-id Transfer failed
480 Permission denied
XXX: Are those all the responses used in current practice?
The 239 and 439 responses MUST contain only the response code
followed by the message-id of the article in question; free-form
text is not permitted.
Parameters
message-id = Article message-id
5.2. Description
A client issues the TAKETHIS command to indicate that the article
with the specified message-id will be sent immediately following
the CRLF at the end of the TAKETHIS command. The entire article,
headers and body, is sent just as with IHAVE; articles sent using
Vinocur Expires December 2003 [Page 5]
Internet Draft NNTP Extension for Streaming Feeds June 2003
this command MUST obey the format described for IHAVE (see section
6.3.2 of [NNTP]). This command MAY be pipelined; in particular,
clients often issue many TAKETHIS command in sequence to reduce the
time spent not transferring data.
The client MAY send articles via TAKETHIS without first querying
the server with CHECK. The client SHOULD NOT send every article in
this fashion unless explicitly configured to do so by the site
administrator based on out-of-band information. However, the
client MAY use an adaptive strategy where it initially sends CHECK
commands for all articles, but switches to using TAKETHIS without
CHECK if most articles are being accepted (over 95% acceptance may
be a reasonable metric in some configurations). If the client uses
such a strategy, it SHOULD also switch back to using CHECK on all
articles if the acceptance rate ever falls much below the
threshold.
5.3. Examples
Example of successfully sending an article to another site:
[C] TAKETHIS <i.am.an.article.you.will.want@example.com>
[C] Path: pathost!demo!somewhere!not-for-mail
[C] From: "Demo User" <nobody@example.com>
[C] Newsgroups: misc.test
[C] Subject: I am just a test article
[C] Date: 6 Oct 1998 04:38:40 -0500
[C] Organization: An Example Com, San Jose, CA
[C] Message-ID: <i.am.an.article.you.will.want@example.com>
[C]
[C] This is just a test article.
[C] .
[S] 239 <i.am.an.article.you.will.want@example.com>
Example of sending an article to a site that already has it, or
does not want it, or where it does not transfer correctly:
[C] TAKETHIS <i.am.an.article@example.com>
[C] Path: pathost!demo!somewhere!not-for-mail
[C] From: "Demo User" <nobody@example.com>
[C] Newsgroups: misc.test
[C] Subject: I am just a test article
[C] Date: 6 Oct 1998 04:38:40 -0500
[C] Organization: An Example Com, San Jose, CA
[C] Message-ID: <i.am.an.article@example.com>
[C]
[C] This is just a test article.
[C] .
[S] 439 <i.am.an.article@example.com>
Vinocur Expires December 2003 [Page 6]
Internet Draft NNTP Extension for Streaming Feeds June 2003
6. Augmented BNF Syntax for Streaming
This amends the formal syntax for NNTP [NNTP] to add the commands
described above. The syntax is defined using ABNF [ABNF],
including the core rules from section 6 of [ABNF].
command /= mode-stream-command /
check-command /
takethis-command
mode-stream-command = "MODE" WS "STREAM
check-command = "CHECK" WS message-id
takethis-command = "TAKETHIS" WS message-id
; WS and message-id are defined in section 13 of [NNTP]
7. Security Considerations
No new security considerations are introduced by this extension,
beyond those already described in the core specification [NNTP].
8. Acknowledgements
This document is based heavily on the relevant sections of RFC 2980
[NNTP-COMMON], by Stan Barber.
9. Normative References
[ABNF] Crocker, D., Overell, P., "Augmented BNF for Syntax
Specifications: ABNF", RFC 2234, November 1997.
[KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", RFC 2119, Harvard University, March 1997.
[NNTP] Feather, C., "Network News Transport Protocol"
(draft-ietf-nntpext-base-18.txt).
10. Informative References
[NNTP-COMMON] Barber, S., "Common NNTP Extensions", RFC 2980,
Academ Consulting Services, October 2000.
11. Author's Address
Jeffrey M. Vinocur
Department of Computer Science
Upson Hall
Cornell University
Ithaca, NY 14853
Vinocur Expires December 2003 [Page 7]
Internet Draft NNTP Extension for Streaming Feeds June 2003
EMail: vinocur@cs.cornell.edu
Full Copyright Statement
Copyright (C) The Internet Society (2002). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain
it or assist in its implementation may be prepared, copied,
published and distributed, in whole or in part, without restriction
of any kind, provided that the above copyright notice and this
paragraph are included on all such copies and derivative works.
However, this document itself may not be modified in any way, such
as by removing the copyright notice or references to the Internet
Society or other Internet organizations, except as needed for the
purpose of developing Internet standards in which case the
procedures for copyrights defined in the Internet Standards process
must be followed, or as required to translate it into languages
other than English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on
an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Vinocur Expires December 2003 [Page 8]
