FTPEXT Working Group                     Internet Engineering Task Force
INTERNET-DRAFT                                                 D. Borman
<draft-ietf-ftpext-restart-00.txt>        Berkeley Software Design, Inc.
Expires 21 September 1997                                       R. Adams
                                                                   UUNET
                                                           21 March 1997


                 REST Command and Extensions to FTP

Status of this memo

   This document is an Internet-Draft.  Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF), its areas,
   and its working groups.  Note that other groups may also distribute
   working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet- Drafts as reference
   material or to cite them other than as ``work in progress.''

   To view the entire list of current Internet-Drafts, please check
   the "1id-abstracts.txt" listing contained in the Internet-Drafts
   Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
   (Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au
   (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu
   (US West Coast).

1. Overview

   This memo describes changes to FTP [1], which were proposed by Rick
   Adams in May of 1989, to allow the RESTART mechanism to be used with
   STREAM mode transfers.  Along with this, two new optional commands
   are added, MODIFICATION TIME (MDTM) and SIZE OF FILE (SIZE).  All
   these changes have been implemented, and are in use today in many
   production FTP implementations.

2. Motivation

   The FTP specification has three modes of data transfer, Stream, Block
   and Compressed.  To avoid having to resend the entire file if the
   file is only partially transferred, both sides need some way to be
   able to agree on where in the data stream to restart the data
   transfer.

   In Block and Compressed mode, the data that is transferred over the
   data connection is formatted, allowing the imbedding of restart
   markers   into the data stream.  The sending side can send a restart
   marker with whatever information it needs to be able to restart a
   file transfer at that point.  The receiving side can keep a list of
   these restart markers, and corrolate them with how the file is being
   saved.  To restart the file transfer, the receiver just sends across
   that last restart marker, and both sides know how to resume the data
   transfer.  Note that there are some flaws in the description of the
   restart mechanism in RFC 959, see RFC 1123 [2] for the corrections.

Borman & Adams            Expires 21 September 1997            [Page  1]

INTERNET-DRAFT     REST Command and Extensions to FTP      21 March 1997


2.1 Restarting in STREAM Mode

   In Stream mode, the data that is transferred over the data connection
   is just a stream of unformatted bytes of data, hence explicit restart
   markers cannot be inserted into the data stream.  For this reason,
   the FTP specification does not provide the ability to do restarts in
   Stream mode.  However, there is not really a need to have explicit
   restart markers in this case, as restart markers can be implied by
   the byte offset into the data stream.

   If the data representation type is IMAGE, for many systems the file
   will be stored exactly in the same format as it is sent across the
   data connection.  It is very easy for the receiver to determine how
   much data was previously sent, and notify the sender the byte offset
   where the transfer should be restarted.  If the User FTP is doing the
   retrieve, it will directly calculate the restart marker, and send
   that information in the RESTart command.  However, if the User FTP is
   sending the file, it needs to be able to determine how much data was
   previously sent.  A new FTP command is needed to get this
   information.

2.2 Determining File Size

   A new FTP command, SIZE OF FILE (SIZE), is used to get the size of a
   file on the remote side.  If the data representation type is not
   IMAGE, then how the file is stored may not match how it will look
   when transferred over the wire.  It is because of this that the
   implied restart marker is a byte offset into the data stream, and the
   SIZE of a file may change depending on what is the current mode of
   transfer.  The receiver will need to read the partially transferred
   file, do any appropriate conversion, and count the number of bytes
   that would be generated when sending the file.  This is the byte
   offset into the data stream, and is what is conveyed to the sender.
   The sender will need to read the file, do the appropriate conversion,
   and skip throw away as much of the data stream as the sender
   requested.  At this point, both sender and receiver will be at the
   same point in the data stream, and the transfer of the file can be
   continued.

   When doing a restart of a file transfer, it is very possible that the
   file may have been modified on the senders side between when the file
   was   originally transferred, and when the restart is being done.  In
   this case, it would be better to resend the entire file.  The FTP
   protocol does not provide a mechanism to automatically detect this
   situation.  However, with the addition of one new FTP command, this
   can be done.

2.3 RESTarting and Source File Modification

   The new FTP command, MODIFICATION TIME (MDTM), can be used to
   determine when the file on the remote side of the connection was last
   modified.  When attempting to restart a RETRieve, if the User FTP
   does a MDTM command, it can check and see if the modification time of

Borman & Adams            Expires 21 September 1997            [Page  2]

INTERNET-DRAFT     REST Command and Extensions to FTP      21 March 1997

   the source file is more recent than the modification time of the
   partially transferred file.  If it is, then most likely the source
   file has changed and it would be unsafe to restart in the middle of
   the file transfer.  When attempting to restart a STORe, the User FTP
   can do a MDTM command to find out the modification time of the
   partially transferred file.  If it is older than the modification
   time of the file that is about to be STORed, then most likely the
   source file has changed and it would be unsafe to restart in the
   middle of the file transfer.

   Note that this is applicable to any RESTart attempt, regardless of
   the mode of the file transfer.

3. Description

   This section describes the specific changes that need to be made to
   the FTP specification to add the changes described in Section 2.
   Note that the FTP specification is modified by RFC 1123 [2],
   specifically RFC 959 has errors in its description of the Restart
   mechanism.

3.1 MODIFICATION TIME

   In Section 4.1.3, "FTP SERVICE COMMANDS", insert:

       MODIFICATION TIME (MDTM)

          This command returns the time the file named in its argument
          field was last modified. The time is returned in ISO 3307
          "Representation of the Time of Day" format.  This format is
          YYYYMMDDHHmmSS or YYYYMMDDHHmmSS.xxx, where

              YYYY    is the year
              MM      is the month (01-12)
              DD      is the day of the month (01-31)
              HH      is the hour of the day (00-23)
              mm      is the minute of the hour (00-59)
              SS      is the second of the hour (00-59)
              xxx     if present, is a fractional second and may be
                      any length

          Time is expressed in UTC (GMT), not local time.

3.2 SIZE OF FILE (SIZE)

   In Section 4.1.3, "FTP SERVICE COMMANDS", insert:

      SIZE OF FILE (SIZE)

          This command returns the size of the file named in its
          argument field.  The value returned is in a format suitable
          for use with the RESTART (REST) command. This format will
          change depending on the current MODE and TYPE of the
          connection.  E.g. For mode stream and type image, it would be

Borman & Adams            Expires 21 September 1997            [Page  3]

INTERNET-DRAFT     REST Command and Extensions to FTP      21 March 1997

          a count of the number of octets that would be transferred with
          the RETRIEVE (RETR) command. This command is normally used in
          conjunction with the RESTART (REST) command.

3.3 ERROR RECOVER AND RESTART

   In Section 3.5, "ERROR RECOVERY AND RESTART", second paragraph,
   replace:

       The restart procedure is defined only for the block and
       compressed mode of data transfer. It requires the sender...

   with:

       The restart procedure is defined for the block, compressed and
       stream mode of data transfer. Block and compressed mode require
       the sender...

   At the end of Section 3.5, "ERROR RECOVERY AND RESTART", insert:

       STREAM MODE transfers with FILE STRUcture may be restarted even
       though no restart marker has been transferred in addition to the
       data itself.  This is done by using the SIZE command followed by
       the RESTART (REST) command.

       When using TYPE ASCII or IMAGE, the SIZE command will return the
       number of bytes that would actually be transferred if the file
       were to be sent between the two systems. I.e. with type IMAGE,
       the SIZE normally would be the number of octets in the file. With
       type ASCII, the SIZE would be the number of characters in the
       file INCLUDING any characters that would be inserted during the
       CR-LF expansion.

5. References

   [1] Postel, J., and Reynolds, J., "FILE TRANSFER PROTOCOL (FTP)", RFC
       959, ISI, October 1985

   [2] Braden, R., editor, "Requirements for Internet Hosts --
       Application and Support", RFC 1123, October 1989


Author's Address

   David A. Borman
   Berkeley Software Design, Inc.
   4719 Weston Hills Drive
   Eagan, MN 55123
   Phone: (612) 405-8194
   Email: dab@bsdi.com

   Rick Adams
   UUNET Technologies, Inc.
   3110 Fairview Park Drive, Suite 570

Borman & Adams            Expires 21 September 1997            [Page  4]

INTERNET-DRAFT     REST Command and Extensions to FTP      21 March 1997

   Falls Church, VA 22042 USA
   Phone: +1 703 204 8000
   Email: rick@uunet.uu.net



















































Borman & Adams            Expires 21 September 1997            [Page  5]