INTERNET-DRAFT                    |                 F. (Lalo) Martins
December 1996                     |    Independent software developer
Expires: 03/jul/96                |               Free Software Union

            Internet Discussion Forum Protocol (IDFP)
                   draft-martins-forums-01.txt


Status of this Document

   This document is an Internet-Draft.  Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups.  Note that other groups may also distribute
working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time.  It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as ``work in progress.''

   To learn the current status of any Internet-Draft, please check
the ``1id-abstracts.txt'' listing contained in the Internet-Drafts
Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe),
munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or
ftp.isi.edu (US West Coast).


Abstract

   This document presents a new alternative way of implementing
discussion forums other than NNTP and mailing lists. It is largely
based on standard e-mail except for retrieval. The purpose of this
new protocol is to atenuate the bandwidth and net resources needed
by either NNTP and mailing lists.

   Based on implementation experience by me and Kovisoft, the
remote moderator capabilities have been rewriten (in fact this was
because I conceived a better way).


Introduction

   With the growth of the Internet, its two means of discussion
(one-to-many) communications have grown with it reaching levels such
that they can safely be assumed to represent a good portion of all
network traffic.

   When these protocols were created, yers ago, there weren't enought
users to make bandwidth an issue; instead people wanted a distributed
model (NNTP) to allow for low speeds. Now, holding thousands of
copies of the newsgroups in servers all over the world has proven to
be a waste of resources, since we could easily access them on one, or
at most, for the top groups, ten hosts through the world.



Martins                                                      [Page 1]


Draft                     Internet Forums                     July 96

   The need for the newsgroups to be copied by this much hosts has
made the last years see an unthinkable explosion of mailing lists. As
they are a good substitute for newsgroups as long as postings don't
have to be stored in lots of places, they are inherently closed (if
someone is interested on them, it is necessary to subscribe) and
aren't by definition stored anywhere - archives have to be made "by
hand" by someone, usually the list mantainer.

   Also, mailing lists waste a lot of bandwidth since subscribers
don't always read all postings - unread postings have to be
transmitted via SMTP[1], usually trough many hosts, as well as those
wich are actually read.

   This documment presents a mid-term between newsgroups and mailing
lists, called Internet Forums. These are treated like mailing lists
half of their lives; someone posts a standard mail message to them
just like posting to a mailing list, and messages are carried to the
Forum host just like any mail message.

   Once they are in the Forum host, they become similar to newsgroups
in that they're not redistributed, but instead kept and archived for
later reading by anyone through a retrieval protocol similar to
POP (Post Office Protocol)[3].



Table of Contents

1.Description of the protocol ..................................... 3
                 (brief discussion about this all)
2.Message retrieval ............................................... 4
     (description of the core of IDFP: the retrieval subsystem)
  2.1.Starting the session ......................................4
  2.2.Listing messages ..........................................5
  2.3.Retrieving messages .......................................6
  2.4.Ending the session ........................................6
  2.5.Example session ...........................................6
3.Forum operators ................................................. 8
                  (remotely maintaining a forum)
4.Moderating forums ............................................... 9
        (instructions on how to implement moderated forums)
5.Inclusion and exclusion lists .................................. 10
               (limiting the postings to a forum)
6.Mirroring ...................................................... 10
       (achieving some of NNTP functionality trough mirroring)
7.The Message-number 0 ........................................... 10
             (an useful convention to help new users)
8.Security considerations ........................................ 11
                    (what NOT to expect from IDFP)
a.References ..................................................... 11
b.Author's Address ............................................... 11



Martins                                                      [Page 2]


Draft                     Internet Forums                     July 96

1.Description of the protocol

   The Internet Discussion Forum Protocol presents methods for
accessing Internet Discussion Forums, and to implement moderated and
mirrored Forums.

   The method for submitting messages is SMTP (Simple Mail Transfer
Protocol) and is beyond the scope of this document - refer to RFC
821[1]. The method for storage is up to the implementor and is beyond
the scope of this document. The format of the messages is MIME and is
beyond the scope of this document - refer to RFC 1522[4].

   IDFP is intended to be:

   1: simple, so that hackers, people who are experimenting with the
system or otherwise brave people can access it directly with any
telnet tool;

   2: similar to POP (Post Office Protocol)[3], so that POP
programmers can easily start doing Forum clients;

   3: not too space-consumming so that it fulfills its primary goal
of freeing bandwidth and so that people can easily read it over slow
modem connections.

   Forums have an standard "e-mail" address trough wich they receive
their messages. As any e-mail address, it is composed of the Forum
name (internal host identification), an at "@" sign and the host
name. For example, if we would copy the newsgroup comp.os.linux.misc
to a forum at, say, forums.linux.org (wich would be a bad idea BTW),
its address would be "comp.os.linux.misc@forums.linux.org". Note
that this address is somewhat big; forum addresses are restricted in
size just like any address.





















Martins                                                      [Page 3]


Draft                     Internet Forums                     July 96

2.Message Retrieval

   The method for retrieving Internet Forum messages is designed to
be simple and similar to POP[3]. Its description is divided in four
stages, with the second and third being optional (you can connect,
list and close, connect, retrieve and close or even, for any reason,
connect and close). There is just one command for each stage, and
commands are not case-sensitive (retr is the same as RETR or Retr).

   IDFP uses the response model from POP, issuing just one "ok" and
one "err" message. This is intended for simplicity, a goal POP
successfully reached. But while POP issues "+OK"and "-ERR" messages,
wich must always be uppercase, IDFP messages are always lowercase
"+ok" and "-err" to add a layer of differentiation - thus on
connection any user agent can tell if the host is POP or IDFP just by
looking at the welcome message.

   IDFP also adds a "-cri" response to notify critical errors. A
critical error is defined as any error wich causes closing of the
session by the server.

   Also, there are no provisions for listing the forums on a server.
To read a forum, the user has to already have its address.


2.1.Starting the session

   Before the session can start, the computer that keeps the messages
(from now on referred to as the host or server) is left "listening"
on the well-known TCP port (number 549). The computer interested on
receiving the messages (from now on referred to as the client)
connects via TCP to this port of the server to start the session.

   As long as the connection is stablished the host should respond
with a line starting with "+ok". This message shouldn't exceed one
line.

   The client is expected to reply with the keyword "user" and the
full e-mail address of the user. This is for log purposes only. It
is also possible that the host use this information to restrict
multiple connections by the same user, but this is an elective
behaviour.

   Once the user is identifyed, the server should again respond with
"+ok" and start the session, entering a state in wich it is ready for
all the other IDFP commands.

   If there is an error anytime before identification, such as the
user being already connected (and multiple sessions not being
permitted), the user not being welcome (for reasons up to the host's
mantainer) or any internal server error, the server should respond
with "-cri" and close the connection.


Martins                                                      [Page 4]


Draft                     Internet Forums                     July 96

2.2.Listing messages

   The command for listing messages is "list". There are two
parameters, of wich the first is required and the second is optional.
The first parameter is the name of the forum, without the "@" and the
hostname. The second is the number of the first message to list.

   Each message in a forum must have an unique number. If old
messages are deleted, newer ones should retain their number and
further posts shouldn't reuse numbers. So if an user agent has
already listed up to "500", it is safe to assume that any new
messages can be listed with "list forumname 501".

   The server should respond with, first, "+ok", then a blank line,
then the listing. Entries in the listing are separated by a blank
line. The listing ends on a line composed of a period "." by itself.

   If there are deleted messages in the specified range (after the
number specified by the client as "first" and before the last
message in the forum) they should just be skipped - thus, if in the
forum there is the message 0 (see section 7) and then the next one is
230, the server should just list them skipping the missing ones.

   Each entry in the listing is composed of part of the MIME headers
for the respective headers. The first line is the internal message
number. Note this line isn't part of or incorporated in the header.
The format is: "Message-number: xxxxxx", according to RFC 822[3]
format for headers.

   Then follows the "Subject", "From", and "Date" message headers.

   Example listing:

------------------------------------
list foobar 237
+ok last message in foobar is 239

Message-number: 237
Subject: Best foobar on the net?
From: john.doe@mib.mil
Date: Sun, 28 Jul 1996 17:18:19 -0700

Message-number: 238
Subject: Re: fresh fish from foo freezer
From: jose.da.silva@provedor.com.br
Date: Sun, 28 Jul 1996 18:00:25 -0300

Message-number: 239
Subject: Re: Best foobar on the net?
From: dante@crystal.net
Date: Sun, 28 Jul 1996 19:33:06 -0600

.

Martins                                                      [Page 5]


Draft                     Internet Forums                     July 96

2.3.Retrieving messages

   The retrieval command is "retr". Its two paramenters are the forum
name and the message number, both required. The server response is
similar to the POP response for the POP RETR comand, except the
"+OK" message should be replaced by "+ok".

   The host first issues a "+ok" message; then a blank line; then the
message, in standard MIME format, ending with a line with a period
"." by itself. As in POP, lines of the actual message wich start with
a period should be preceded with another period.


2.4.Ending the session

   The session is closed when the client issues the "quit" command.
The server is only permitted to close the session after receiving a
"quit" command and issuing a "+ok" response, or if any internal error
occurs and after issuing a "-cri" message.


2.5.Example session

   The lines starting with "%" are entered by the client; please note
there is not an actual "%" in the session.

   Note also that sending blank lines after responses is an optional
but recommended behaviour of the server.


























Martins                                                      [Page 6]


Draft                     Internet Forums                     July 96

+ok Foonet Internet Forum server ready at foonet.org

%user lalo@webcom.com
+ok be welcome

%list foobar 240
-err no new messages

%list enfoozer.devel 79
+ok 3 messages (1522 octets)

Message-number: 79
Subject: Re: fresh fish from foo freezer
From: jose.da.silva@provedor.com.br
Date: Sun, 28 Jul 1996 18:00:25 -0300

Message-number: 80
Subject: Re: fresh fish from foo freezer
From: dante@crystal.net
Date: Sun, 28 Jul 1996 22:16:00 -0600

Message-number: 81
Subject: Should we add a deffozer?
From: john.doe@mib.mil
Date: Mon, 29 Jul 1996 02:00:22 -0700

.

%retr 81
+ok 300 octets

Received: from mib.mil by foonet Mon, 29 Jul 1996 02:05:08 -0700
Received: from localhost by mib.mil with SMTP (1.37.109.15/16.2) \
id AA234939381; Mon, 29 Jul 1996 02:00:30 -0700
Date: Mon, 29 Jul 1996 02:00:22 -0700
Message-Id: <31F8FFE6.8BC@mib.mil>
From: John Doe <john.doe@mib.mil>
Organization: Men In Black
Mime-Version: 1.0
To: enfoozer.devel@foonet.org
Subject: Should we add a deffozer?
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=us-ascii
Content-Length: 300

   I think a good enffozer tool also has to have a deffozer included.
Comments?

John Doe - Man in Black (john.doe@mib.mil, http://www.mib.mil/doe)
.

%quit
+ok foonet.org forums server closing connection

Martins                                                      [Page 7]


Draft                     Internet Forums                     July 96

3.Forum operators

   Eventually, messages do have to be deleted (usually). The easiest
way of doing that is locally, by someone working on the same LAN as
the server, trough direct editing of the forum files. However, in big
or commercial forum servers, there may be need for remote operators,
and the protocol allows for that.

   Note that the features described in this section are optional. A
Forum server can be considered an implementation of IDFP and not
implement remote operators. However, implementing remote operators in
a different way than described here makes the server incompatible.

   The remote forum operation adds three new commands: "oper", "pass"
and "dele". The "oper" command may be issued anytime after user
identification ("user") and before logoff ("quit") and, when
accepted, gives the user permission to issue operator and moderator
commands on one given forum.

   The syntax is: "oper", followed by the forum name, followed by the
operator password. An user can have operator permissions for more
than one forum at the same time, by issuing more than one "oper"
commands in the same session. Of course, if the password given is
incorrect, the user is not granted operator permission.

   The "pass" command changes the moderator password for a forum. The
user has to already have moderator permission (have already issued an
"oper" command for that forum). The "pass" command takes only one
parameter, which is the new password. Changing a password does not
have any other effect and does not revoke current permissions.

   Finally, the "dele" command operates in a similar way to the POP
command of the same name. The only differences are, the user has to
have operator permissions and have to specify the forum name. Of
course there is one more difference, but an implementation difference
- in forums, "dele" does not change message numbers as said above
(if message 3 is deleted, message 4 will still be message 4).

   The syntax for "dele" is: "dele" forumname number ... : the first
parameter is the name of a forum for which the user has operator
permission, and the second is the number of the message to be
deleted. Different from the POP command, more than one message
number can be specified in a same command.











Martins                                                      [Page 8]


Draft                     Internet Forums                     July 96

4.Moderating forums

   Sometimes there is need to disable postings to a forum, except
from one person (usually tha mantainer) called the Moderator. To do
so, once a message for the forum arrives at the host it is stored in
a separate message "queue" that can only be operated by the forum
moderator. This feature is also optional; however, if remote
moderators are supported, they have to be supported the way they are
described here, and remote operators have to be supported also
because this protocol only supports moderating by the forum operator.

   Forum servers that implement moderated forums have four extra
moderator commands: "que", "spy", "acpt" and "delq". All of them can
only be issued by an user with operator permission to the forum (one
that have already successfully issued an "oper" command for that
forum).

   The "que" command is similar to "list", but it lists the queue
instead of the real forum messages. The syntax is the same, and the
format for the server reply is also the same. The only issue to be
noted is that queue message numbers are not fixed; when the session
ends, the numbers are lost and then, in the next session, the first
message in the queue will be message 1, the next will be 2 etc.

   The "spy" command is identical to the "retr" command, but it
retrieves a message from the queue instead of from the forum.

   The "delq" is also similar to the "dele" command, but it deletes a
message from the queue, not from the forum. Note the number is not
freed until the session is closed; a subsequent "que" or "spy" on a
deleted message will notice the message is deleted. In the "que"
response, the "From", "Subject" and "Date" fields will be ommited,
and the word "DELETED" (all uppercase) will be appended to the
"Message-number" pseudo-header. Example:

Message-number: 237 DELETED

   A "spy" on a deleted queue message will result in a "-err" message
just like a "retr" on a forum deleted message would.

   Finally, the "acpt" command moves a message from the queue to the
forum. The message is marked as deleted in the queue, and added to
the forum with a new number assigned. The server response should
start with "+ok" and end with the number the message was assigned in
the forum - something like "+ok message was assigned number 372".

   The optional "acc0" command acts like "acpt", but the message is
assigned the number 0 (replacing the previous message with that
number). Details on the purpose of this command are in section 7.







Martins                                                      [Page 9]


Draft                     Internet Forums                     July 96

5.Inclusion and exclusion lists

   An implementor may choose to have inclusion or exclusion lists to
limit the "from" addresses accepted for messages. As this is not
apparent in the IDFP session, it can be implemented in any way the
implementor likes. Unaccepted messages can be returned, discarded, or
forwarded to the operator.

   Note this is not a good way of implementing moderated lists like
the "*.announce" newsgroups where only a limited range of people can
post; anyone can fake a "from:" header! The right way to do that is
trough a combination of both an inclusion (or exclusion) list AND
a moderated forum - so only mail from permitted users would go to the
queue, and others will be (discarded|returned|forwarded to the oper).

   In any instance, NO messages to a moderated forum should go
directly to the forum bypassing the queue.



6.Mirroring

   Very popular forums can result in cluttered hosts, and the
consequence is a slow or even refused connection, especially if the
client is located far from the host. Fortunately, as this also
happens to the web and to ftp sites, the solution has already been
thinked out for us. It is called Mirroring, and consists on copying a
site - or, in this case, forum.

   This is very easy to implement - the mirroring server only has to
act as a client and retrieve messages from the main site in fixed
intervals of time.

   It is also possible to send messages to the mirror forums. These
messages should be just forwarded to the main site.



7.The Message-number 0

   A recommended convention is that, in any forum, the message with
the Message-number 0 should be a message from the operator to new
users, describing the forum, what are its purposes and posting
conventions, and welcoming the user. Of course, the message 0 should
never be deleted, but the operator can substitute it for a new one
with the "acc0" moderator command in a moderated forum. In a non-
moderated one, another provision may be made - like the existence of
another e-mail address to put messages in the moderating queue. Of
course, this is optional also (something kind of like a BCP).





Martins                                                     [Page 10]


Draft                     Internet Forums                     July 96

8.Security considerations

   Inherently, Internet Forums are not secure. Anything even remotely
private shouldn't be on a public forum. The only security issues
addressed by this document are concerning authentication of operator
permission, in sections 3 and 4.

   A certain degree of privacity can be achieved using standard tools
for electonic mail encription, if something must be read by a
restricted but large group and nobody else. Since Forum messages are
MIME messages, almost everything that applies to electronic mail
security applies also to Forums.








Appendix A.References

[1]  J. Postel, "Simple Mail Transfer Protocol", 01/aug/1982, RFC821

[2]  D. Crocker, "Standard for the format of ARPA Internet text
     messages", 13/aug/1982, RFC 822

[3]  J. Myers, M. Rose, "Post Office Protocol - Version 3",
     14/may/1996, RFC1939

[4]  N. Borenstein, N. Freed, "MIME  (Multipurpose Internet Mail
     Extensions) Part One:  Mechanisms for Specifying and Describing
     the Format of Internet Message Bodies", 09/23/1993, RFC1521


Appendix B.Author's Address

Fernando Mauro Martins - aka Lalo

postal: Rua Realengo, 268 - cep 05451-030 - Sao Paulo - Brazil

phone: +55-11-263 7358 (home and operating base)

now how to REALLY contact me:

http://webcom.com/lalo        |         mailto:lalo@webcom.com

Now I'm on the IETF mailing list, and that is, probably, the best
place to post comments so other people can see your comments and
comment them in turn.




Martins                                                     [Page 11]