HTTP Working Group Jeffrey Mogul, DECWRL
Internet-Draft Ari Luotonen, Netscape
Expires: 28 September 1997 13 March 1997
Problem with HTTP/1.1 Warning header, and proposed fix
draft-ietf-http-warning-00.txt
STATUS OF THIS MEMO
This document is an Internet-Draft. Internet-Drafts are
working documents of the Internet Engineering Task Force
(IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of
six months and may be updated, replaced, or obsoleted by
other documents at any time. It is inappropriate to use
Internet-Drafts as reference material or to cite them other
than as "work in progress."
To learn the current status of any Internet-Draft, please
check the "1id-abstracts.txt" listing contained in the
Internet-Drafts Shadow Directories on ftp.is.co.za
(Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific
Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US
West Coast).
Distribution of this document is unlimited. Please send
comments to the HTTP working group at
<http-wg@cuckoo.hpl.hp.com>. Discussions of the working
group are archived at
<URL:http://www.ics.uci.edu/pub/ietf/http/>. General
discussions about HTTP and the applications which use HTTP
should take place on the <www-talk@w3.org> mailing list.
ABSTRACT
The current HTTP/1.1 (RFC2068) specification introduces a
new "Warning" header, meant to carry status information
about a request that cannot or should not be carried by the
response status code. The existing specification for the
interaction between Warning and HTTP caches is faulty, in
that it may allow incorrect results after cache validation
operations. This document identifies two separate (but
related) problems, and proposes revisions of the HTTP/1.1
specification to solve these problems.
Mogul, Luotonen [Page 1]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
TABLE OF CONTENTS
1 Introduction 2
2 Description of the problems 3
2.1 Proxy P2 implements HTTP/1.1 4
2.2 Proxy P2 implements HTTP/1.0 5
3 Proposed solutions 5
3.1 Solution to the ambiguity of future Warning codes 5
3.2 Solution to the HTTP/1.0 proxy problem 6
3.3 Editorial issues 7
3.3.1 RFC2068 Section 13.1.2: Warnings 7
3.3.2 RFC2068 Section 13.5.3 Combining Headers 8
3.3.3 RFC2068 Section 14.45 Warning 9
4 Security Considerations 12
5 Revision history 12
5.1 draft-ietf-http-warning-00.txt 12
6 Acknowledgements 12
7 References 12
8 Authors' addresses 12
1 Introduction
The HTTP/1.1 specification (RFC2068) [1] introduces a new response
header named "Warning", described this way (RFC2068 section 14.45):
The Warning response-header field is used to carry additional
information about the status of a response which may not be
reflected by the response status code. This information is
typically, though not exclusively, used to warn about a
possible lack of semantic transparency from caching
operations.
The specification also says:
A cache MUST NOT delete any Warning header that it received
with a response. However, if a cache successfully validates a
cache entry, it SHOULD remove any Warning headers previously
attached to that entry except as specified for specific
Warning codes. It MUST then add any Warning headers received
in the validating response.
while RFC2068 section 13.5.3 states, regarding what a cache does when
receiving a 304 response:
The end-to-end headers stored in the cache entry are used for
the constructed response, except that any end-to-end headers
provided in the 304 response MUST replace the corresponding
headers from the cache entry.
Note that there is an implicit disagreement about whether existing
Mogul, Luotonen [Page 2]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
Warning headers are associated with a cache entry after it is
revalidated.
Also, RFC2068 section 13.1.2 states:
Warnings are always cachable, because they never weaken the
transparency of a response. This means that warnings can be
passed to HTTP/1.0 caches without danger; such caches will
simply pass the warning along as an entity-header in the
response.
This statement turns out to be somewhat erroneous.
The (implicit) goal behind the design of the Warning mechanism was
that when an HTTP/1.1 client receives a message with a Warning
header, that header should accurately reflect the status of the
message. However, since the issuance of RFC2068, it has been pointed
out that the current specification can lead to Warning headers being
attached to messages that should not have them, because of the
interaction between the Warning specification and other
specifications for HTTP/1.1 caches.
Another goal of the Warning design is that when a Warning is properly
attached to a response, the response should be delivered to any
end-client with the Warning intact. This should be true even if the
path includes one or more HTTP/1.0 proxies.
2 Description of the problems
The two known problems both involve a form of the following scenario.
Suppose that the end-client is connected to the origin server via a
path with two or more caching proxies:
Origin HTTP/1.1 HTTP/1.1
Server ------ Proxy P1 ------ Proxy P2 ------ Client
Also suppose that proxy P1 has a cached response for resource D in
its cache, but proxy P2 does not. Also suppose that proxy P1 is
configured to provide stale responses in some situations (e.g.,
because of network bandwidth constraints).
Now the end-client requests resource D, via proxy P2. Proxy P2 does
not have a cache entry for D, so it forwards the request to proxy P1.
Proxy P1 does have a stale cache entry for D, but decides to provide
the response without validating the cache entry (i.e., without
sending a conditional GET to the origin server.) So proxy P1 sends a
response containing its cache entry for D, and attaches
Warning: 10 P1 "Response is stale"
Mogul, Luotonen [Page 3]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
P2 receives this response, forwards it to the end-client, and also
stores the response in its own cache. So far, there is no problem.
At a later time some client of proxy P2 (perhaps the same one as
before) requests resource D again from that proxy. This time, both
proxies decide to validate their cached responses (since both
responses are stale). So proxy P2 forwards a conditional GET to
proxy P1, which forwards a conditional GET to the origin server.
Suppose that the origin server sends a 304 (Not Modified) response to
proxy P1, which forwards it to proxy P2.
At this point, the "Response is stale" Warning stored in the cache
entry at P2 is clearly not correct; the response has been
successfully revalidated. The correct behavior in this case would be
for P2 to remove the warning from its cache entry, and to ensure that
it is not attached to the response it forwards to the end-client.
Given this scenario, there are two different situations, depending on
whether P2 implements HTTP/1.1 or higher (i.e., it understands the
Warning header), or HTTP/1.0 or lower (i.e., it does not understand
the warning header).
2.1 Proxy P2 implements HTTP/1.1
The HTTP/1.1 specification already hints at a solution for the case
where the proxy (or any other caching agent) in question actually
implements the Warning header. RFC2068 section 14.45 says:
A cache MUST NOT delete any Warning header that it received
with a response. However, if a cache successfully validates a
cache entry, it SHOULD remove any Warning headers previously
attached to that entry except as specified for specific
Warning codes. It MUST then add any Warning headers received
in the validating response.
Implicit in this is that there are two categories of Warnings:
1. Those that describe the freshness or revalidation status
of the response, and so must be deleted after a successful
revalidation.
2. Those that describe some aspect of the entity body or
entity headers that is not rectified by a revalidation;
for example, a lossy compression of the entity body.
These Warnings cannot be deleted after a revalidation.
Also implicit here is that a cache can reliably tell the difference
between Warnings that should be deleted upon revalidation, and
Warnings that should be retained even with revalidation. The problem
here is that the set of Warning codes might be expanded beyond the
set listed in the HTTP/1.1 specification.
Mogul, Luotonen [Page 4]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
For example, if an HTTP/1.1 cache receives
Warning: 37 "P1" "My hovercraft is full of eels"
that cache would not be able to tell if the Warning should be deleted
or retained after a revalidation of the response.
2.2 Proxy P2 implements HTTP/1.0
If the proxy implements HTTP/1.0 (or lower), then it does not
understand the Warning header at all. Therefore, it will always
retain the Warning header in its cache entry, even after
revalidation, and will pass this on to its clients.
This is not a problem for HTTP/1.0 clients (which ignore Warning),
but HTTP/1.1 clients of proxy P2 could receive spurious Warning
headers for the indefinite future.
3 Proposed solutions
This section proposes independent solutions to the two problems
described in section 2. These solutions, together, are consistent
with the two goals stated for Warnings in section 1. (These goals
would prevent the adoption of several other proposed solutions, such
as changing the Expires header when forwarding a response to an
HTTP/1.0 client, or deleting the Warning header in this case.)
3.1 Solution to the ambiguity of future Warning codes
Section 2.1 shows that HTTP/1.1 caches do not have unambiguous
information about whether a Warning should be retained or deleted
upon revalidation of a cached response.
This can be solved by adding explicit information to the Warning
values, so that the retain/delete choice is encoded in a way that
does not require a cache to understand the full set of Warning codes.
Several protocols, including FTP [2] and HTTP itself, use three-digit
status codes where the first digit conveys information about the
success or failure of an operation; this allows some extensibility of
the set of status codes without breaking existing implementations.
The same general technique can be used for the Warning code:
- Expand the warning code from two digits to three digits
- Use the leading digit to divide the Warnings into those
that refer to the freshness (validation state) of a
response, and must be deleted upon revalidation, and those
that refer to other aspects of the response, and must be
retained after revalidation.
Mogul, Luotonen [Page 5]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
- Redefine the existing Warning codes according to this
scheme
- Update the language in other parts of the specification to
be consistent with the new scheme.
This approach allows the agent that initially attaches a Warning
code, and so presumably knows the desired semantics, to explicitly
indicate whether it should be retained or deleted after revalidation.
It entirely removes the need for a recipient to make a non-trivial
decision.
Section 3.3.3 provides a revised specification for Warning.
3.2 Solution to the HTTP/1.0 proxy problem
The solution to the problem described in section 2.2 is more
difficult, because HTTP/1.0 caches cannot be expected to protect
against incorrect caching of received Warning headers.
Since HTTP/1.0 agents ignore Warning entirely, the specific problem
is to prevent an HTTP/1.1 recipient from interpreting a Warning
header that has incorrectly been associated with a response. If such
an incorrect association has been made, this is because an HTTP/1.0
cache first received a response with an appropriate Warning header,
then revalidated the response but failed to delete the Warning
header.
In other words, an HTTP/1.1 recipient of a Warning needs a way to
determine whether it came from the most recent validation attempt for
a cache entry, or whether it came with an earlier response.
Every HTTP response carries a Date header indicating when it was
generated. It appears that when an HTTP/1.0 cache forwards a request
and receives a 304 (Not Modified) response, the response that it
forwards to its own client carries the (more recent) Date of the 304
response, not the Date that was associated with the original (cached)
response.
---------
NOTE: We need to verify that this is indeed the way that
HTTP/1.0 caches behave!
---------
If Warning headers were to carry an (optional) warning-date value,
duplicating the value of the Date header associated with the response
when the Warning was first attached to it, then an HTTP/1.1 recipient
could compare this warning-date to the Date header in the received
response. If these timestamps are the same, then (within the
1-second resolution of the Date header) the recipient can be sure
that the Warning was not left over from an earlier response.
Mogul, Luotonen [Page 6]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
If the warning-date is earlier than the Date, however, then the
Warning had to have been created for an earlier response. If, in
this case, the Warning code indicates that the Warning should have
been deleted upon revalidation, then the recipient can safely delete
it.
---------
NOTE: The warning-date should never be later than the Date, if
I understand things correctly. If this ever happens, it
implies that some cache is not updating the Date of a cached
response with the Date of a subsequent 304 response. This
should never happen with HTTP/1.1 proxies, but it's not clear
if this could ever happen with an HTTP/1.0 proxy.
---------
Therefore, a solution to the problem of HTTP/1.0 caches and Warnings
is for
- HTTP/1.1 proxies that create or forward a Warning to copy
the Date value into an optional field of the Warning, if
and only if the message is being sent to an HTTP/1.0 or
lower recipient.
- HTTP/1.1 caches that receive an HTTP/1.0 response with a
Warning to delete that Warning if its warning-date does not
match the Date in the message, and if the Warning code
indicates that it should be deleted upon revalidation.
Section 3.3.3 provides a revised specification for Warning, which
includes this optional warning-date field.
3.3 Editorial issues
This section describes the editorial changes to RFC2068 that are
required to implement both of the proposed modifications, and to
eliminate contradictory language regarding the Warning header.
3.3.1 RFC2068 Section 13.1.2: Warnings
RFC2068 section 13.1.2 states:
Warnings are always cachable, because they never weaken the
transparency of a response. This means that warnings can be
passed to HTTP/1.0 caches without danger; such caches will
simply pass the warning along as an entity-header in the
response.
Warnings are assigned numbers between 0 and 99. This
specification defines the code numbers and meanings of each
currently assigned warnings, allowing a client or cache to
take automated action in some (but not all) cases.
This passaged must be revised as follows:
Mogul, Luotonen [Page 7]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
Warnings come in two categories:
1. Those that describe the freshness or revalidation
status of the response, and so MUST be deleted after a
successful revalidation (see section 13.3 for a
definition of revalidation).
2. Those that describe some aspect of the entity body or
entity headers that is not rectified by a
revalidation; for example, a lossy compression of the
entity body. These Warnings MUST NOT be deleted after
a successful revalidation.
Warnings are assigned 3-digit code numbers. The first digit
indicates whether the Warning must or must not be deleted
from a cached response after it is successfully revalidated.
This specification defines the code numbers and meanings of
each currently assigned warnings, allowing a client or cache
to take automated action in some (but not all) cases.
HTTP/1.0 caches will cache all Warnings, without deleting the
ones in the first category. Warnings that are passed to
HTTP/1.0 caches carry an extra warning-date field, which
prevents a future HTTP/1.1 recipient from believing an
erroneously cached Warning.
3.3.2 RFC2068 Section 13.5.3 Combining Headers
RFC2068 section 13.5.3 states:
When a cache makes a validating request to a server, and the
server provides a 304 (Not Modified) response, the cache must
construct a response to send to the requesting client. The
cache uses the entity-body stored in the cache entry as the
entity-body of this outgoing response. The end-to-end headers
stored in the cache entry are used for the constructed
response, except that any end-to-end headers provided in the
304 response MUST replace the corresponding headers from the
cache entry. Unless the cache decides to remove the cache
entry, it MUST also replace the end-to-end headers stored
with the cache entry with corresponding headers received in
the incoming response.
In other words, the set of end-to-end headers received in the
incoming response overrides all corresponding end-to-end
headers stored with the cache entry. The cache may add
Warning headers (see section 14.45) to this set.
This must be revised to say:
When a cache makes a validating request to a server, and the
server provides a 304 (Not Modified) response, the cache must
Mogul, Luotonen [Page 8]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
construct a response to send to the requesting client. The
cache uses the entity-body stored in the cache entry as the
entity-body of this outgoing response. The end-to-end headers
stored in the cache entry are used for the constructed
response, except that
- any stored Warning headers with warn-code 1XX (see
section 14.45) are deleted from the cache entry and the
forwarded response.
- any stored Warning headers with warn-code 2XX are
retained in the cache entry and the forwarded response.
- any end-to-end headers provided in the 304 response
MUST replace the corresponding headers from the cache
entry.
Unless the cache decides to remove the cache entry, it MUST
also replace the end-to-end headers stored with the cache
entry with corresponding headers received in the incoming
response.
In other words, the set of end-to-end headers received in the
incoming response overrides all corresponding end-to-end
headers stored with the cache entry (except for stored
Warning headers with warn-code 1XX, which are deleted even if
not overridden).
3.3.3 RFC2068 Section 14.45 Warning
The entire RFC2068 Section 14.45 (Warning) is replaced as follows:
The Warning response-header field is used to carry additional
information about the status of a response which may not be reflected
by the response status code. This information is typically, though
not exclusively, used to warn about a possible lack of semantic
transparency from caching operations.
Warning headers are sent with responses using:
Warning = "Warning" ":" 1#warning-value
warning-value = warn-code SP warn-agent SP warn-text
[SP warn-date]
warn-code = 3DIGIT
warn-agent = ( host [ ":" port ] ) | pseudonym
; the name or pseudonym of the server adding
; the Warning header, for use in debugging
warn-text = quoted-string
warn-date = <"> HTTP-date <">
Mogul, Luotonen [Page 9]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
---------
NOTE: The warn-date syntax requires the quotes because of the
comma in the HTTP-date syntax. The comma serves to separate
the wkday (day of week) token from the rest of the date, and
since the wkday is completely redundant, we could save seven
bytes (including the two commas) by definining a different date
format for use here. We could also eliminate the "GMT" from
this format, since it is mandatory for HTTP-Date.
---------
A response may carry more than one Warning header.
The warn-text should be in a natural language and character set that
is most likely to be intelligible to the human user receiving the
response. This decision may be based on any available knowledge,
such as the location of the cache or user, the Accept-Language field
in a request, the Content-Language field in a response, etc. The
default language is English and the default character set is
ISO-8859-1.
If a character set other than ISO-8859-1 is used, it MUST be encoded
in the warn-text using the method described in RFC 1522 [14]. Any
server or cache may add Warning headers to a response. New Warning
headers should be added after any existing Warning headers. A cache
MUST NOT delete any Warning header that it received with a response.
However, if a cache successfully validates a cache entry, it SHOULD
remove any Warning headers previously attached to that entry except
as specified for specific Warning codes. It MUST then add any Warning
headers received in the validating response. In other words, Warning
headers are those that would be attached to the most recent relevant
response.
When multiple Warning headers are attached to a response, the user
agent SHOULD display as many of them as possible, in the order that
they appear in the response. If it is not possible to display all of
the warnings, the user agent should follow these heuristics:
- Warnings that appear early in the response take priority
over those appearing later in the response.
- Warnings in the user's preferred character set take
priority over warnings in other character sets but with
identical warn-codes and warn-agents.
Systems that generate multiple Warning headers should order them with
this user agent behavior in mind.
The warn-code consists of three digits. The first digit indicates
whether the Warning MUST or MUST NOT be deleted from a stored cache
entry after a successful revalidation:
Mogul, Luotonen [Page 10]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
1XX Warnings that describe the freshness or revalidation
status of the response, and so MUST be deleted after
a successful revalidation.
2XX Warnings that describe some aspect of the entity body
or entity headers that is not rectified by a
revalidation, and which MUST NOT be deleted after a
successful revalidation.
This is a list of the currently-defined warn-codes, each with a
recommended warn-text in English, and a description of its meaning.
110 Response is stale
MUST be included whenever the returned response is
stale.
111 Revalidation failed
MUST be included if a cache returns a stale response
because an attempt to revalidate the response failed,
due to an inability to reach the server.
112 Disconnected operation
SHOULD be included if the cache is intentionally
disconnected from the rest of the network for a
period of time.
113 Heuristic expiration
MUST be included if the cache heuristically chose a
freshness lifetime greater than 24 hours and the
response's age is greater than 24 hours.
199 Miscellaneous warning
The warning text may include arbitrary information to
be presented to a human user, or logged. A system
receiving this warning MUST NOT take any automated
action.
214 Transformation applied
MUST be added by an intermediate cache or proxy if it
applies any transformation changing the
content-coding (as specified in the Content-Encoding
header) or media-type (as specified in the
Content-Type header) of the response, unless this
Warning code already appears in the response.
299 Miscellaneous persistent warning
The warning text may include arbitrary information to
be presented to a human user, or logged. A system
receiving this warning MUST NOT take any automated
action.
Mogul, Luotonen [Page 11]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
If an implementation sends a response with one or more Warning
headers to a client whose version is HTTP/1.0 or lower, then the
sender MUST include a warn-date in each warning-value.
If an implementation receives a response with a warning-value that
includes a warn-date, and that warn-date is different from the Date
value in the response, then that warning-value MUST be deleted from
the message before storing, forwarding, or using it. If all of the
warning-values are deleted for this reason, the Warning header MUST
be deleted as well.
4 Security Considerations
No known security implications beyond those listed in RFC2068.
5 Revision history
Minor clarifications, and grammar and spelling corrections, are not
listed here.
5.1 draft-ietf-http-warning-00.txt
Initial draft.
6 Acknowledgements
We gratefully acknowledge the constructive comments received from Roy
Fielding, Koen Holtman, and Ben Laurie.
7 References
1. Roy T. Fielding, Jim Gettys, Jeffrey C. Mogul, Henrik Frystyk
Nielsen, and Tim Berners-Lee. Hypertext Transfer Protocol --
HTTP/1.1. RFC 2068, HTTP Working Group, January, 1997.
2. Jon Postel and Joyce Reynolds. File Transfer Protocol (FTP).
RFC 959, Network Working Group, October, 1985.
8 Authors' addresses
Jeffrey C. Mogul
Western Research Laboratory
Digital Equipment Corporation
250 University Avenue
Palo Alto, California, 94305, U.S.A.
Email: mogul@wrl.dec.com
Phone: 1 415 617 3304 (email preferred)
Mogul, Luotonen [Page 12]
Internet-Draft HTTP Warning Header 13 March 1997 17:10
Ari Luotonen
Netscape Communications Corp.
501 East Middlefield Road
Mountain View, CA 94043
Email: ari@netscape.com
Mogul, Luotonen [Page 13]
