Easily Parsed LIST Format (EPLF) INTERNET-DRAFT draft-bernstein-eplf-02.txt (expires 1 August 1997) This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as ``work in progress.'' To learn the current status of any Internet-Draft, please check the ``1id-abstracts.txt'' listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). Status of this memo This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Distribution of this memo is unlimited. Abstract The File Transfer Protocol (FTP) supports two commands that list files: NLST and LIST. The NLST response is easy to parse but provides very little information. The LIST response provides more information, but in a format that varies from system to system. The most common LIST formats are undocumented and impossible to parse reliably. This document defines Easily Parsed LIST Format (EPLF), a format for the LIST response that is usable by humans yet easy for programs to handle. This format is supported by anonftpd, a secure FTP server. One visible advantage of EPLF is that a browser can easily display dates in the viewer's time zone and native language. EPLF also makes it straightforward for an indexing program to automatically traverse an FTP area and for a mirroring program to avoid downloading the same file twice. Easily Parsed LIST Format (EPLF) D. J. Bernstein, djb@pobox.com 19970201 1. Introduction The File Transfer Protocol (FTP) supports two commands that list files: NLST and LIST. The NLST response is easy to parse but provides very little information. The LIST response provides more information, but in a format that varies from system to system. The most common LIST formats are undocumented and impossible to parse reliably. This document defines Easily Parsed LIST Format (EPLF), a format for the LIST response that is usable by humans yet easy for programs to handle. This format is supported by anonftpd, a secure FTP server. One visible advantage of EPLF is that a browser can easily display dates in the viewer's time zone and native language. EPLF also makes it straightforward for an indexing program to automatically traverse an FTP area and for a mirroring program to avoid downloading the same file twice. EPLF also corrects a design flaw in FTP's handling of LIST arguments. An EPLF server must respond to ``LIST filename'' with information about that file and no others, even if that file is a directory. A client that wants an EPLF list of the contents of a directory must first CWD to that directory. A client that merely wants a list of file names in a different directory may use NLST. In this document, a string of 8-bit bytes may be written in two different forms: as a series of hexadecimal numbers between angle brackets, or as a sequence of ASCII characters between double quotes. For example, <68 65 6c 6c 6f 20 77 6f 72 6c 64 21> is a string of length 12; it is the same as the string "hello world!". 2. Format An EPLF response to LIST is a series of lines, each line specifying a different file. Each line begins with "+", continues with a series of facts about the file, and ends with <09> followed by the file name. Each fact is zero or more bytes of information, terminated by "," and not containing <09>. There are several possible facts, each of which appears at most once, in any order: "r" If this file name is supplied in a RETR command, the RETR should succeed. The server must supply this fact unless it is aware of file type problems, permission problems, or other reasons that RETR will fail. The presence of "r" does not guarantee success: for example, the file may be removed or renamed, or the RETR may suffer a temporary failure. "/" If this file name is supplied in a CWD command, the CWD should succeed. As with "r", the server must supply this fact unless it is aware of reasons that CWD will fail. The presence of "/" does not guarantee success. "i"[ident] This file has identifier [ident]. [ident] is a sequence of bytes not including "," or <09>. If two files on the same FTP server (not necessarily in the same LIST response) have the same [ident], those files have the same contents; a successful RETR of each file should produce the same results, and a successful CWD to each file should lead to the same working directory. (Under UNIX, for example, [dev].[ino] could be used as [ident], where [dev] and [ino] are the device number and inode number of the file.) "s"[size] The size of this file is [size]. [size] is a sequence of ASCII digits specifying a number. If the file is retrieved in TYPE I and is not modified, it will contain exactly [size] bytes. This fact should not be supplied if "r" is not supplied. "m"[time] This file was last modified at [time]. [time] is a sequence of ASCII digits specifying a number of seconds, real time, since the beginning of 1970 GMT. This fact cannot be used for files modified before 1970 GMT. Further facts may be defined in the future. Pieces of the fact-space beginning with "x" will be parcelled out to organizations that would like to define their own facts. Facts beginning with "X" are reserved for experimental use. All facts other than "/" and "r" are optional. Any statement of adherence to EPLF by a server FTP implementation must include a list of facts supported by that implementation other than "/" and "r". The server is under no obligation to ensure that LISTs in different directories produce disjoint lists of targets. For example, some servers may list a special ".." name that refers to the parent directory, or a "/" name that refers to the top directory. To avoid loops, a client attempting to traverse the FTP area must notice that the identifiers of these directories are the same as identifiers of directories already traversed. The server is also under no obligation to list all possible targets of RETR or CWD in a LIST command. Some servers may avoid listing special names such as ".." or "/". A client that wishes to return to a directory must use PWD and record the reply rather than relying on any useful meaning of CDUP, CWD .., or CWD /. Operating systems support a wide variety of means for obtaining the contents of a file from its name. For example, many systems support symbolic links: if ONE is a link to TWO, any reference to ONE is first replaced by a reference to TWO. Such information is irrelevant to FTP and is not displayed by any of the above facts. (Under UNIX this means that the server should use stat(), not lstat().) Servers are permitted to use arbitrary characters in file names, except for <0a> and <0d>. Beware that the characters <00>, <09>, <20>, and <ff> cause all sorts of trouble, ranging from inadequacies in the syntax of FTP commands to misinterpretation by some clients. 3. Examples Here is a typical EPLF response: "+i8388621.48594,m825718503,r,s280," <09> "djb.html" <0d 0a> "+i8388621.50690,m824255907,/," <09> "514" <0d 0a> "+i8388621.48598,m824253270,r,s612," <09> "514.html" <0d 0a> A typical EPLF-ignorant client will show the response to the user: ftp> dir 200 Okay. 150 I'm looking through the directory. Trying to connect... +i8388621.48594,m825718503,r,s280, djb.html +i8388621.50690,m824255907,/, 514 +i8388621.48598,m824253270,r,s612, 514.html 226 Finished transferring 127 bytes. ftp> A more sophisticated client (in the Pacific timezone) might instead display the following human-readable listing: Tue Feb 13 15:58:27 1996 514/ 612 bytes Tue Feb 13 15:14:30 1996 514.html 280 bytes Fri Mar 1 14:15:03 1996 djb.html 4. Sample code The following C function takes a pointer to a string containing one line of an EPLF response. It assumes that the original response did not contain <00>, and that the trailing <0d 0a> has been replaced by <00>. It returns a pointer to the filename, or 0 if the line does not appear to be an EPLF response. char *eplf_name(line) char *line; { if (*line != 43) return 0; while (*line) if (*line++ == 9) return line; return 0; } The following C function takes a pointer as above, and prints a human-readable listing as shown in section 3. It assumes that the local character set is ASCII, that file modification times fit into a local time_t, and that file sizes fit into a local unsigned long. It also assumes that time_t is interpreted as a number of seconds since the beginning of 1970 GMT. (A more portable function could use mktime() to discover the time_t representation of 1970 GMT.) Note that its output is not machine-readable, since the file name might contain the local newline sequence. #include <time.h> int eplf_readable(line) char *line; { int flagcwd = 0; time_t when = 0; int flagsize = 0; unsigned long size; if (*line++ != '+') return 0; while (*line) switch (*line) { case '\t': if (flagsize) printf("%10lu bytes ",size); else printf(" "); if (when) printf("%24.24s",ctime(&when)); else printf(" "); printf(" %s%s\n",line + 1,flagcwd ? "/" : ""); return 1; case 's': flagsize = 1; size = 0; while (*++line && (*line != ',')) size = size * 10 + (*line - '0'); break; case 'm': while (*++line && (*line != ',')) when = when * 10 + (*line - '0'); break; case '/': flagcwd = 1; default: while (*line) if (*line++ == ',') break; } return 0; } 5. Acknowledgments Thanks to Scott Schwartz for pointing out that "i"[ident] was originally overspecified. Thanks to Benjamin Riefenstahl for several helpful suggestions.