Internet Engineering Task Force A. Ford
Internet-Draft Roke Manor Research
Intended status: Experimental M. Handley
Expires: January 7, 2010 University College London
July 6, 2009
HTTP Extensions for Simultaneous Download from Multiple Mirrors
draft-ford-http-multi-server-00
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on January 7, 2010.
Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents in effect on the date of
publication of this document (http://trustee.ietf.org/license-info).
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document.
Abstract
This document describes an extension to HTTP by which servers can
automatically inform clients of mirrors of resources. Clients can
Ford & Handley Expires January 7, 2010 [Page 1]
Internet-Draft Multi-server HTTP July 2009
then simultaneously request segments of the resource from different
servers, enhancing both network and server utilisation, download
speeds, and thus user experience.
Table of Contents
1. Introduction and Motivation . . . . . . . . . . . . . . . . . 3
1.1. Operation Overview . . . . . . . . . . . . . . . . . . . . 3
1.2. Requirements Language . . . . . . . . . . . . . . . . . . 4
2. HTTP Extension Headers . . . . . . . . . . . . . . . . . . . . 4
2.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2. Requests . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2.1. X-Multiserver-Version . . . . . . . . . . . . . . . . 5
2.2.2. X-If-Checksum-Match . . . . . . . . . . . . . . . . . 5
2.3. Responses . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3.1. X-Multiserver-Version . . . . . . . . . . . . . . . . 6
2.3.2. X-Checksum . . . . . . . . . . . . . . . . . . . . . . 6
2.3.3. X-Mirrors . . . . . . . . . . . . . . . . . . . . . . 6
3. Example Operation . . . . . . . . . . . . . . . . . . . . . . 7
4. Full Interaction Specification . . . . . . . . . . . . . . . . 8
4.1. Error Handling . . . . . . . . . . . . . . . . . . . . . . 8
4.1.1. Checksum Failure . . . . . . . . . . . . . . . . . . . 8
5. Alternative Approaches . . . . . . . . . . . . . . . . . . . . 9
6. Heuristics and Optimizations . . . . . . . . . . . . . . . . . 9
7. Managing Server Load . . . . . . . . . . . . . . . . . . . . . 10
8. Security Considerations . . . . . . . . . . . . . . . . . . . 10
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 10
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11
11.1. Normative References . . . . . . . . . . . . . . . . . . . 11
11.2. Informative References . . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 11
Ford & Handley Expires January 7, 2010 [Page 2]
Internet-Draft Multi-server HTTP July 2009
1. Introduction and Motivation
Mirrored HTTP servers are regularly used for software downloads,
whereby copies of data to be downloaded are duplicated on many
servers distributed around the Internet. Users are encouraged to
manually choose a nearby mirror from which to download. This is
intended to increase both throughput and resilience, and reduce load
on individual servers. Manual mirror choice rarely works well; users
do not wish to make a choice, but if they are not forced to, then the
default server takes a disproportionate share of the load. Even when
they are forced to choose, they rarely have enough information to
choose the server that will provide the best performance.
Some popular sites automate this process using DNS load balancing,
both to approximately balance load between servers, and to direct
clients to nearby servers with the hope that this improves
throughput. Indeed, DNS load balancing can balance long-term server
load fairly effectively, but it is less effective at delivering the
best throughput to users when the bottleneck is not the server but
the network.
This document specifies an alternative mechanism by which the benefit
of mirrors can be automatically and more efficiently realised. These
benefits are achieved using a number of extensions to HTTP which
allow the discovery of mirrors, the verification of the integrity of
files on each mirror, and the simultaneous downloading of chunks from
multiple mirrors. The use of this mechanism allows greater
efficiency in resource utilisation in the Internet as a whole,
balances server utilization, even on short timescales, and enhances
user experience through faster downloads.
1.1. Operation Overview
In an HTTP request a client will first declare that it conforms to
multi-server HTTP extensions. The capable server that wishes to take
advantage of mirroring for the file requested will respond in its
headers with a list of mirrors for the given file, and it will also
return a checksum for the file.
Multi-server HTTP extensions require the client and server support
persistent connections. A server responding with a mirror list uses
chunked encoding for the response to permit the client to request
ranges of bytes from the file from different servers. It is normally
up to the client to decide on the ranges requested from each mirror.
However to avoid wasted time when talking to the initial server, this
server starts to return data immediately, and it is up to the server
to determine this initial chunk size.
Ford & Handley Expires January 7, 2010 [Page 3]
Internet-Draft Multi-server HTTP July 2009
The server will make an estimate of a sensible initial chunk size,
and immediately respond with this amount of the file (a Content-
Range: header declares how much is sent in this chunk). The
mechanism for determining the initial chunk size is up to the server
implementation. However some suggestions include: it could be
determined by a bounded proportion of the file size, or based on the
RTT of the initial handshake, or based on cached information on
previous throughput to the same client, or based on current
throughput achieved by the server, or a combination of all of these.
We will discuss these heuristics in more detail below
It is then up to the client to work out appropriate scheduling for
retrieving further chunks of this file from the original server and
the mirrors. The initial server will continue sending its own
scheduled chunks until a GET request is received containing the
standard HTTP Range: header, and at this point the server defers to
the client's requests. It is up to the client to choose how many
servers to use, and what size chunks to request from each. An
advanced client is expected to adapt to relative speeds of servers
and utilise the bandwidth most effectively, and to pipeline range
requests to these servers to avoid idle time on the connections. The
client will also include a new checksum verification header to ensure
the file on the mirror is the same as from the initial server.
1.2. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
2. HTTP Extension Headers
2.1. Overview
The additional functionality required at the HTTP level can be broken
down as follows:
o Ensure client and server operate same version of Multi-Server HTTP
o Retrieve and verify a checksum for a resource
o Identify alternatives servers that provide access to the same
resource
o Retrieve segments of a resource
From these requirements, the additional headers required can be
Ford & Handley Expires January 7, 2010 [Page 4]
Internet-Draft Multi-server HTTP July 2009
derived, and are described in the following sections.
NOTE WELL: The names of the extension headers given below are
TEMPORARY and FOR DISCUSSION ONLY. These will evolve to meet
[RFC2774] if appropriate.
The formal definitions below make use of the conventions defined in
Section 2.2 of [RFC2616].
2.2. Requests
2.2.1. X-Multiserver-Version
This is used in both request and response to indicate that the host
is has multi-server HTTP capability and what version it operates.
For this specification, this version field MUST read "0.1".
Formally,
X-Multiserver-Version =
"X-Multiserver-Version" ":" 1*DIGIT "." 1*DIGIT
Example:
X-Multiserver-Version: 0.1
2.2.2. X-If-Checksum-Match
This option is used in a request as a conditional, where the request
is only valid if the checksum of the resource matches that specified
in this header.
The checksum is that already provided by the X-Checksum header in the
response (see below).
Two checksum functions are provided in this specification, and a
compliant implementation MUST implement both. These are MD5
[RFC1321] and SHA-256 [FIPS180-3].
Formally,
checksum = quoted-string
sum-type = "MD5" | "SHA-256"
X-If-Checksum-Match =
"X-If-Checksum-Match" ":" 1 sum-type 1 checksum
Example:
Ford & Handley Expires January 7, 2010 [Page 5]
Internet-Draft Multi-server HTTP July 2009
X-If-Checksum-Match: MD5 \
"b1946ac92492d2347c6235b4d2611184"
2.3. Responses
2.3.1. X-Multiserver-Version
As for a request, this handshake ensures both hosts will speak the
same protocol.
2.3.2. X-Checksum
This option is presented by the server to provide a checksum of the
whole file, which MUST be used in the X-If-Checksum-Match option to
other mirrors. It SHOULD be used by a client to verify the file once
all segments have been downloaded.
Formally,
X-Checksum = "X-Checksum" ":" 1 sum-type 1 checksum
Example:
X-Checksum: MD5 \
"b1946ac92492d2347c6235b4d2611184"
2.3.3. X-Mirrors
This option lists the orginal file, the cache lifetime in seconds,
and the list of all mirrors available for the requested URI. This
header once per mirror. If the original file ends in a slash ("/"),
it indicates that the mirrors will provide all resources under this
directory. If so, then all the mirrors MUST also end in a slash. If
it does not specify a directory, but instead ends in a file name,
then this declares that the mirror exists only for this file. Any
further requests for different files must first go to this server to
acquire a mirror list, if it exists.
The cache lifetime is the time which this mirrored information can be
cached for, before it SHOULD be considered stale and SHOULD not be
used. Caching the mirror list is most important for mirrors of
directories, where small files can be requested directly from the
mirrors without needing to request the first chunk from the initial
server.
Formally:
X-Mirrors="X-Mirrors" ":" filename cachetime 1*absoluteURI
Ford & Handley Expires January 7, 2010 [Page 6]
Internet-Draft Multi-server HTTP July 2009
Example:
X-Mirrors: /wibble/download.zip 3600 \
http://www.example2.com/wibble/download.zip \
http://www.example3.com/wibble/download.zip
3. Example Operation
So far, this document has defined the functional components that
enable multi-server HTTP. This section will draw out an example of
how these fit together, and utilising existing HTTP/1.1 features.
First, a Multi-Server HTTP client will connect to a server and
request a file, while declaring itself multi-server HTTP capable:
GET /wibble/download.zip HTTP/1.1
Host: www.example.com
X-Multiserver-Version: 0.1
The server will respond with details about this resource, and begin
delivering some of the file:
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 10240
Content-Type: application/zip
Content-Range: bytes 1-10240/2025121
X-Multiserver-Version: 0.1
X-Checksum: MD5 "d6862c992a3d6736ad678cc865dee67f"
X-Mirrors: /wibble/download.zip 3600 \
http://www.example2.com/wibble/download.zip \
http://www.example3.com/wibble/download.zip
...data...
The client will then continue requesting more blocks from this
server:
GET /wibble/download.zip HTTP/1.1
Host: www.example.com
X-Multiserver-Version: 0.1
Range: 10241-20480
The server will then respond with further blocks of the file:
Ford & Handley Expires January 7, 2010 [Page 7]
Internet-Draft Multi-server HTTP July 2009
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 10240
Content-Type: application/zip
Content-Range: bytes 10241-20480/2025121
X-Multiserver-Version: 0.1
...data...
It will then also connect to the mirrors and request more blocks,
such as the following:
GET /wibble/download.zip HTTP/1.1
Host: www.example2.com
X-Multiserver-Version: 0.1
X-If-Checksum-Match: MD5 "d6862c992a3d6736ad678cc865dee67f"
Range: 20481-30720
Assuming the checksum matches correctly, the server will respond as
for a normal multiserver HTTP continuation request from the original
server.
When finished, the client can close all its connections.
4. Full Interaction Specification
TBD. This section will define permitted HTTP status codes, behaviour
upon errors, etc.
4.1. Error Handling
Multi-server HTTP introduces a number of failure modes that do not
exist in conventional HTTP.
4.1.1. Checksum Failure
It should not be the case that the data checksum fails when the file
has finished downloading, as the mirrors MUST NOT return data if it
does not correspond to the version of the file specified in the
checksum. However, if this occurs due to bugs or hardware data
corruption, there is no way to determine which chunks are errored.
It would be possible to add mechanism to check with the original
server to determine what the checksum would be for a range, but this
would add extra complexity for a case that is expected to be
extremely rare. In the case of a checksum failure, a client SHOULD
re-request any chunks received from mirrors from the original server,
but recalculate the file checksum periodically to determine if the
Ford & Handley Expires January 7, 2010 [Page 8]
Internet-Draft Multi-server HTTP July 2009
errored chunk has now been replaced.
5. Alternative Approaches
It is debatable whether ETags could be used in place of checksums,
along with the If-Match header in requests. We elected not to follow
this as the definition of an ETag is that it must be unique to a
resource, however in this case we are accessing multiple resources
(URIs) even though they should be the same data. We do not feel
comfortable using ETags as this would require the semantics of the
field to be changed.
6. Heuristics and Optimizations
The performance of Multipath HTTP depends heavily on several factors:
o The number of servers used simultaneously.
o The ability to pipeline sufficient or sufficiently large range
requests to each server so as to avoid connections going idle.
o The ability to pipeline sufficiently few or sufficiently small
range requests to servers so that all the servers finish their
final chunks simultaneously.
o The ability to switch between mirrors dynamically so as to use the
fastest mirrors at any moment in time
Obviously we do not want to use too many simultaneous connections, or
other traffic sharing a bottleneck link will be starved. But at the
same time, good performance requires that the client can
simultaneously download from at least one fast mirror while exploring
whether any other mirror is faster. Based on laboratory experiments,
we suggest a good default number of simultaneous connections is
probably four, with three of these being used for the best three
mirrors found so far, and one being used to evaluate whether any
other mirror might offer better performance.
The size of chunks chosen by the client should be sufficiently large
that the chunk request headers and reponse headers represent neglible
overhead, and sufficiently large that they can be pipelined
effectively without needing a very high rate of chunk requests. At
the same time, the amount of time wasted waiting for the last chunk
to download from the last server after all the other servers have
finished should be minimized. Thus we currently recommend that a
chunk size of at least 10KBytes should be used. If the file being
Ford & Handley Expires January 7, 2010 [Page 9]
Internet-Draft Multi-server HTTP July 2009
transfered is very large, or the download speed very high, this can
be increased to perhaps 1MByte. As network bandwidths increase, we
expect these numbers to increase appropriately, so that the time to
transfer a chunk remains significantly larger than the latency of
requesting a chunk from a server.
7. Managing Server Load
A goal of mirroring is to manage server load, not just network
bandwidth. Thus server operators need simple mechanisms they can use
to handle overload conditions gracefully and balance load. Multi-
server HTTP provides spreads load automatically across heterogeneous
servers so that the fastest servers get to serve the most data.
However, if a server is approaching an overload condition, Multi-
server HTTP can increase the overall load a little due to the
additional processing of chunk requests. Thus, a way for such a
server to discard load is desirable. The easiest way to shed load is
to use two ports for HTTP; one is used for the initial requests to a
server, and the second is used to handle requests for subsequent
chunks redirected from other servers. This second port would then be
explicitly reported in the URLs specified in the X-Mirrors option. A
server that is approaching overload can continue to report mirrors in
initial requests sent to it (thus directing load to other mirrors),
but simply stop listening on the second port. This avoids it having
to handle requests redirected to it from other servers until its load
returns to a more manageable level. In the event that all the
mirrors are overloaded, this effectively falls back to conventional
HTTP, with only requests sent to the original port being serviced.
8. Security Considerations
At present, we do not believe there should be any new security issues
with this approach. The use of checksums will ensure that the
resource on every server is the same, and the final result can also
be verified. One concern could be regarding whether we should permit
redirects to be used as responses on a mirror request. Although this
is deviating from the mirrored resource as originally specified by
the initial server, the use of checksums will allow the final file to
be verified.
9. Acknowledgements
This work builds upon work undertaken by Javier Vela Diago, who also
provided validation of the benefits of this approach. It is based
loosely on the BitTorrent algorithm, but adapted to the HTTP client/
Ford & Handley Expires January 7, 2010 [Page 10]
Internet-Draft Multi-server HTTP July 2009
server architecture.
The authors are supported by Trilogy
(http://www.trilogy-project.org), a research project (ICT-216372)
partially funded by the European Community under its Seventh
Framework Program. The views expressed here are those of the
author(s) only. The European Commission is not liable for any use
that may be made of the information in this document.
10. IANA Considerations
None.
11. References
11.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
11.2. Informative References
[FIPS180-3]
National Institute of Standards and Technology (NIST),
"FIPS 180-3 Secure Hash Standard (SHS)", October 2008, <ht
tp://csrc.nist.gov/publications/fips/fips180-3/
fips180-3_final.pdf>.
[RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321,
April 1992.
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
[RFC2774] Nielsen, H., Leach, P., and S. Lawrence, "An HTTP
Extension Framework", RFC 2774, February 2000.
Ford & Handley Expires January 7, 2010 [Page 11]
Internet-Draft Multi-server HTTP July 2009
Authors' Addresses
Alan Ford
Roke Manor Research
Old Salisbury Lane
Romsey, Hampshire SO51 0ZN
UK
Phone: +44 1794 833 465
Email: alan.ford@roke.co.uk
Mark Handley
University College London
Gower Street
London WC1E 6BT
UK
Email: m.handley@cs.ucl.ac.uk
Ford & Handley Expires January 7, 2010 [Page 12]