Internet Draft Ingrid Melve
Expires: December 1999 UNINETT
Informational Gary Tomlinson
WREC Working Group Novell
Ian Cooper
Mirror Image Internet
June, 25 1999
Internet Web Replication and Caching Taxonomy
draft-ietf-wrec-taxonomy-01.txt
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet- Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Abstract
This memo specifies standard terminology and the current taxonomy of
web replication and caching infrastructure deployed today. It
introduces standard concepts and protocols uses today within this
application domain. Currently deployed solutions employing this
technologies are presented to establish a standard taxonomy.
Research issues and HTTP proxy caching known problems are covered in
two accompanying document, and are not part of this document. This
document presents open protocols and points to published RFCs for
each protocol.
Melve, Tomlinson, Cooper [Page 1]
Replication and Caching Taxonomy June 25, 1999
Contents
1. Introduction
2. Terminology
3. Distributed Relationships
4. Client to Replica Communication
5. Inter-Replica Communication
6. Client to Proxy Configuration
7. Inter-Cache Communication
8. Network Element Communication
9. Security Considerations
10. Acknowledgements
11. References
12. Authors' Addresses
1. Introduction
Since its introduction in 1990, the World-Wide Web has evolved from a
simple client server model into a sophisticated distributed
architecture. This evolution has been driven largely due to the
scaling problems associated with exponential growth. Distinct
paradigms and solutions have emerged to satisfy specific
requirements. Two core infrastructural components being employed to
meet the demands of this growth are replication and caching. In man
cases, there is a need for web caches and replicated services to be
able to coexist.
There are many protocols, both open and proprietary, employed in web
replication and caching today. A majority of the open protocols
include DNS[21], CacheDigest[16], CARP[9], HTTP[6], ICP[10], PAC[7],
SOCKS[19], TPACT[22], WPAD[8], and WCCP[18]. Additional protocols are
being planned to address emerging solution requirements.
This memo specifies standard terminology and the current taxonomy of
web replication and caching infrastructure deployed in the Internet
today. The principal goal of this document is to establish a common
understanding and reference point of this application domain.
We also expect that this document will be used in the creation of a
standard architectural framework for efficient, reliable, and
predictable service in a web which includes both replicas and caches.
2. Terminology
Where possible, existing definitions [5, 6] have been used in this
document. Additional terminology has been agreed upon and defined in
this document. All of the terminology used in this document is
considered to be standardized with respect to IETF WREC working group
Melve, Tomlinson, Cooper [Page 2]
Replication and Caching Taxonomy June 25, 1999
RFCs.
In this document a number of terms are used to refer to the roles
played by participants in, and objects of, the HTTP communication.
The following definitions are used in the HTTP/1.1 specification [6].
However, these definitions may have come to have differing meaning
within the Web caching community. In those cases, additional
clarification is given.:
client
An application program that establishes connections for the
purpose of sending requests.
user agent
The client which initiates a request. These are often
browsers, editors, spiders (web-traversing robots), or
other end user tools.
server
An application program that accepts connections in order to
service requests by sending back responses. Any given
program may be capable of being both a client and a server;
our use of these terms refers only to the role being
performed by the program for a particular connection,
rather than to the program's capabilities in
general. Likewise, any server may act as an origin server,
proxy, gateway, or tunnel, switching behavior based on the
nature of each request.
origin server
The server on which a given resource resides or is to be
created.
[Ed note; IAN: The following is subtly different from the
definition given in HTTP/1.1. (Should we now
revert to the definition in HTTP/1.1 and document
the difference?) As a community we must be
careful about which type of "transparent proxy" is
being discussed.]
proxy
An intermediary system which acts as both a server and a
client for the purpose of making requests on behalf of
other clients. Requests are serviced internally or by
passing them on, with possible translation, to other
servers. A proxy MUST implement both the client and server
requirements of this specification. A "transparent proxy"
is a proxy that does not modify the request or response
beyond what is required for proxy authentication and
Melve, Tomlinson, Cooper [Page 3]
Replication and Caching Taxonomy June 25, 1999
identification. A "non-transparent proxy" is a proxy that
modifies the request or response in order to provide some
added service to the user agent, such as group annotation
services, media type transformation, protocol reduction,
or anonymity filtering. Except where either transparent or
non-transparent behavior is explicitly stated, the HTTP
proxy requirements apply to both types of proxies.
Note: The term "transparent proxy" given in [6] has different
meaning within the Web caching community. Further
unspecified references in this document (including the
following paragraph) are to the Web caching community
definition, which is given later.
The condition requiring implementation of both server and
client requirements of HTTP/1.1 is only appropriate for a
non-transparent proxy.
[Ed note; IAN: The following is also subtly different from
HTTP/1.1. Should also consider comments from Joe
Touch on whether we should distinguish types of
tunnels.]
tunnel
An intermediary system which is acting as a blind relay
between two connections. Once active, a tunnel is not
considered a party to the HTTP communication, though the
tunnel may have been initiated by an HTTP request. The
tunnel ceases to exist when both ends of the relayed
connections are closed.
[Ed note; IAN: The following has been slightly modified from
HTTP/1.1 to consider server load. Need to consider
comment from Joe Touch regarding clarification of
not using a cache when tunnelling.]
cache
A program's local store of response messages and the
subsystem that controls its message storage, retrieval, and
deletion. A cache stores cacheable responses in order to
reduce the response time, server load and network
bandwidth consumption on future, equivalent requests. Any
client or server may include a cache, though a cache
cannot be used by a server while it is acting as a tunnel.
[Ed note; IAN: The following has been edited from RFC2616 to
reference that document.]
cacheable
Melve, Tomlinson, Cooper [Page 4]
Replication and Caching Taxonomy June 25, 1999
A response is cacheable if a cache is allowed to store a
copy of the response message for use in answering
subsequent requests. The rules for determining the
cacheability of HTTP responses are defined in section 13
of [6]. Even if a resource is cacheable, there may be
additional constraints on whether a cache can use the
cached copy for a particular request.
To these we add the following:
authoritative reference
the owner of data; content production system; possibly an
origin server
content consumer
the user or system that makes requests of an origin server
(which may in turn be handled by a proxy).
caching proxy
A proxy with a cache, acting as server to clients, and
a client to servers
origin server accelerator
an application of a caching proxy where the proxy is
placed closer to the origin server than to the content
consumers in order to off-load the handling of cacheable
responses from the server; also as a means to reduce
traffic within the server's network.
surrogate
[Ed note; IAN: need a definition.]
network element
router or switch
[Ed note; IAN: This term probably needs a better name.]
browser
a special instance of a user agent that acts as a content
presentation device for content consumer
cluster
a tightly coupled set of devices acting together to share
load
reverse proxy
An intermediary system which acts as both a server and a
client for the purpose of serving requests on behalf of
Melve, Tomlinson, Cooper [Page 5]
Replication and Caching Taxonomy June 25, 1999
origin servers. Requests are serviced internally or by
passing them on to the origin server they are representing.
A reverse proxy must interpret and, if necessary, rewrite a
request message before forwarding it. Reverse proxies are
often used as server-side portals through network firewalls
and as helper applications for off loading requests from
origin servers.
[Ed note; IAN: leaving this as a placeholder until we can
work out proxies/reverse proxies/surrogates
and accelerators]
The following definitions are added to describe caching device
topology:
user agent cache
the cache within the user agent program
local caching proxy
the caching proxy a user agent connects to
[Ed note; IAN: should this be renamed 'primary proxy'?]
intermediate caching proxy
seen from the content consumer's view, all caches
participating in the caching mesh that are not the user
agent's local caching proxy
cache server
a server to requests made by local and upper level caching
proxies, but which does not act as a proxy
cache array
diffused array
cache cluster
a cluster of caching proxies, acting logically as one
service and partitioning the URL name space across the
array
caching mesh
a loosely coupled set of co-operating proxy- or caching-
servers, or clusters, acting independently but sharing
cacheable content between themselves using inter-cache
communication protocols (see Section 7)
Moves to insert proxies into the network in a manner such at the
content consumer is unaware of their presence has created a set of
Melve, Tomlinson, Cooper [Page 6]
Replication and Caching Taxonomy June 25, 1999
terms whose definitions may not be consistent with other uses. This
section references prior definitions but also gives their meaning in
the realm of Web caching.
[Ed note; IAN: snooping, redirection, interception - need to
clarify if we only need the first two]
traffic redirection
redirection of traffic from a user agent or network
element to a specific proxy, used to deploy Web-caching
without the need to manually reconfigure individual user
agents, or to force the use of a proxy where such use
would not otherwise occur
network traffic snooping
the examination of network traffic within a network
element to determine whether it should be redirected
transparent proxy (additional definition)
the term "transparent proxy" is defined in [6] (and quoted
above). However, in the realm of Web caching, this has
come to define a proxy which receives traffic as a result
of network traffic snooping. The term typically
describes the use of a proxy and the additional systems
which performing network traffic snooping. The use of
the proxy is transparent to the client. Transparent
proxies are used to remove the need for configuration of
clients to use a proxy.
proxy discovery
this describes the discovery and configuration for use of
a proxy in an environment where the content consumer may
be unaware of the proxy's existence. The use of the proxy
is transparent to the content consumer, but not to the
client.
[Ed note; IAN: should we consider the ability of proxies
to discover each other? Would this be
better titled as "transparent proxy
configuration"?]
The following terms describe the roles of servers and caches in the
realm of caching and replication:
[Ed note; IAN: This section needs significant work]
temporal domain, sparse working set cache
a subset of the content from one or more origin servers,
Melve, Tomlinson, Cooper [Page 7]
Replication and Caching Taxonomy June 25, 1999
stored temporarily and collected from requests made by
content consumers
persistent domain
a collection of origin servers maintaining a persistent
data set from the authoritative reference
replica origin server
origin server storing a persistent replica of a data set
stored at the authoritative reference
3. Distributed System Relationships
[Ed note; GARY: Consider eliminating this big picture, its doesn't
capture all of the relationships and is difficult to communicate]
Diagram of the components that make up a web replication and caching
infrastructure, with communication between the components.
------------------ ----------------- ------------------
| Replica Origin |-----| Master Origin |-----| Replica Origin |
| Server | | Server | | Server |
------------------ ----------------- ------------------
\ | /
\ | /
-----------------------------------------
| Client to
----------------- Replica Server
| Top-Level |
| Caching Proxy |
-----------------
/ \ Inter Cache
/ \ Communication
----------------- -----------------
| Upper-Level |-----------| Upper-Level |
| Caching Proxy | | Caching Proxy |
----------------- -----------------
/ Inter Cache \
/ Communication \ Inter Cache
/ \ Communication
/ \
/ ------------------ \
/ ------------------| \
Melve, Tomlinson, Cooper [Page 8]
Replication and Caching Taxonomy June 25, 1999
----------------- ----------------- || -----------------
| First Level |-----| Caching Proxy | |-----| First Level |
| Caching Proxy | | Array |-- | Caching Proxy |
----------------- ----------------- -----------------
| Client to |
| Proxy Cache | Cache to Network Element
------------- ------------
| Client | | Network |
------------- | Element |
------------
|
|
------------
| Client |
------------
3.1 Replication Relationships
[Ed note; describe the replication system relationship domain]
3.1.1 Client to Replica
[Ed note; recast this as relationship not the definition which
follows in section 4] Client to Replica: cooperation and
communication between clients (both browser/user agents and proxy
caches) and replica origin servers. Used to discover optimal replica
proximity.
Persistent Domain
Complete Idem-Potent Set Replication
------------------ ----------------- ------------------
| Replica Origin | | Master Origin | | Replica Origin |
| Server | | Server | | Server |
------------------ ----------------- ------------------
\ | /
\ | /
-----------------------------------------
| Client to
----------------- Replica Server
| Client |
| |
-----------------
3.1.2 Inter-Replica
Melve, Tomlinson, Cooper [Page 9]
Replication and Caching Taxonomy June 25, 1999
[Ed note; recast this as relationship not the definition which
follows in section 5] Inter-Replica: cooperation and communication
between replica origin servers. Used in replicating data sets
between origin servers.
Persistent Domain
Complete Idem-Potent Set Replication
------------------ ----------------- ------------------
| Replica Origin |-----| Master Origin |-----| Replica Origin |
| Server | | Server | | Server |
------------------ ----------------- ------------------
3.2 Caching Relationships
[Ed note; describe the caching system relationship domain]
3.2.1 Client to Proxy
[Ed note; recast this as relationship not the definition which
follows in section 6] Client to Proxy: configuration, cooperation and
communication between end user clients (browsers and applications)
and a caching proxy.
Temporal Domain
Sparse Working Set Cache
----------------- ----------------- -----------------
| First Level | | First Level | | First Level |
| Caching Proxy | | Caching Proxy | | Caching Proxy |
----------------- ----------------- -----------------
\ | /
\ | /
-----------------------------------------
|
-----------------
| Client |
-----------------
3.2.2 Reverse Proxy to Origin Server
[Ed note; describe the accelerator relationship]
3.2.2 Inter-Cache
[Ed note; recast this as relationship not the definition which
follows in section 7] Inter-Cache: cooperation and communication
between caching proxies.
Melve, Tomlinson, Cooper [Page 10]
Replication and Caching Taxonomy June 25, 1999
Temporal Domain
Sparse Working Set Cache
-----------------
| Top-Level |
| Caching Proxy |
-----------------
/ \
/ \
----------------- -----------------
| Upper-Level |-----------| Upper-Level |
| Caching Proxy | | Caching Proxy |
----------------- -----------------
/ \ / \
/ \ / \
/ \ / \
/ \ / \
/ \ / \
/ \ / \
----------------- ----------------- -----------------
| First Level |-----| First Level |-------| First Level |
| Caching Proxy | | Caching Proxy | | Caching Proxy |
----------------- ----------------- -----------------
Network Element to Caching Proxy
[Ed note; recast this as relationship not the definition which
follows in section 8] Network Element to Proxy Cache: cooperation and
communication between caching proxy and network elements. Examples
include routes and switches. Generally used for transparent caching
and/or diffused arrays.
Temporal Domain
Sparse Working Set Cache
----------------- ----------------- -----------------
| Caching Proxy | | Caching Proxy | | Caching Proxy |
| Array | | Array | | Array |
----------------- ----------------- -----------------
\ | /
\ | /
-----------------------------------------
|
--------------
| Network |
| Element |
--------------
|
|
------------
Melve, Tomlinson, Cooper [Page 11]
Replication and Caching Taxonomy June 25, 1999
| Client |
------------
Caching Proxies with Transparency
[Ed note: Currently contains citations from NetApp document, need
rewording to avoid specific products and concentrate on generic
properties. Explain network elements and NATs and other ways
interception may happen. Intro to usage and "normal" setup.]
Reference [1,2,3,4] for introduction to caching proxies with
transparency.
The goal of intercepting web traffic is to provide a transparent web
proxy, thus avoiding the hassle of individually configuring each
client.
Transparency means that the user does not need to be aware of the
proxy.
The origin server see connections coming from the proxy, not from the
individual end user. Authentication based on client IP address do not
work if there is a transparent proxy cache in the way to the web
server.
A web cache is said to be transparent if clients can access the cache
without the need to configure their browsers, using either a proxy
auto-configuration URL or a manual proxy setting. Transparent caches
appear as a seamless part of the network infrastructure, rather than
a set of discrete proxy servers, and function much like a transparent
firewall. Many ISPs and carriers desire transparent caches because it
lets them retrofit their network with caching without action at the
client. However, when deployed transparently, a web cache must be as
fail-safe and scalable as the rest of the network. [2]
A transparent cache acts much like a gateway or firewall -- it
effectively sits between the users and the network. The advantage of
transparent caching is that it eliminates the need to configure
browsers to use caching. Another strength (and sometimes a weakness)
is that it is impossible to bypass caching. [2]
Conceptually, transparency works by modifying the TCP/IP stack of a
cache so that it operates in "promiscuous mode" and effectively binds
itself to all possible IP addresses. [2]
We need to give a far more abstract definition which includes the way
that router and switch redirection, and within-router action,
operate.
Melve, Tomlinson, Cooper [Page 12]
Replication and Caching Taxonomy June 25, 1999
Comment on some of the problems:
* limited number of ports which can be captured
* due to "unexpected" data on other ports
(or even on well known ports), as experienced by setting up
various services on port 80
* well known problems with use of HTTP for transport [20]
Out-of-path Transparent Caching Proxies
An Out-of-path Transparent Caching Proxy performs the same proxy and
caching functions as a Transparent Caching Proxy and is similarly
transparent to the client. However it does not lie on the forwarding
path between a client and a server and does not perform web traffic
interception. Instead it relies upon a redirecting network element in
the path between client and server to intercept and redirect web
traffic to it. One advantage of this method of transparent caching is
that in the case of cache failure the network element can, providing
it monitors the state of the caches, revert to forwarding web traffic
direct to the server. It is also possible for the network element to
distribute the web traffic load across a group of caches. This method
of transparent caching generally requires a protocol to be run
between the redirecting network element and the cache or caches.
4. Client to Replica Communication
This section describes the cooperation and communication between
clients (both user agents and proxy caches) and replica origin web
servers. Used to discover a optimal web origin server replica for a
web client to establish service with. Optimality is a policy based
decision, often based upon proximity, but may be based on other
criteria such as load.
4.1 Navigation Hyperlinks
Authoritative reference:
This memo.
Description:
The simplest of client to replica communication
mechanisms. This utilizes hyperlink URL's embedded in web
pages that point to the mirror sites. The human user
manually selects the link of the replica origin server
they wish to use.
Security:
Relies on the protocol security associated with the URL
scheme.
Melve, Tomlinson, Cooper [Page 13]
Replication and Caching Taxonomy June 25, 1999
Deployment:
Probably the most commonly deployed client to replica
communication mechanism. Ubiquitous interoperability
with humans.
Submitter:
Document editors.
4.2 URL Redirection
Authoritative reference:
This memo.
Description:
A simple and commonly used mechanism to connect web
clients with origin server replicas is to use URL
redirection. Clients are redirected to a optimal web
server replica via the use of the HTTP [6] protocol
response code 307 Temporary Redirect. A web client
establishes HTTP communication with one of the web server
replicas. The initially contacted replica origin web
server can either choose to accept the service or redirect
the client to the proper replica. Refer to section 10.3.8
in HTTP/1.1 RFC2616 for information on HTTP response code
307.
Security:
Relies entirely upon HTTP security.
Deployment:
Observed at a number of large web sites. Extent of usage
in the Internet is unknown at this time.
Submitter:
Document editors.
4.3 DNS Redirection [21]
Authoritative reference:
Load balancing: RFC1794 DNS Support for Load Balancing
Proximity: This memo
[Ed note; it would have been nice to cite SONAR, but draft has
expired]
Description:
The Domain Name Service (DNS) provides a more
Melve, Tomlinson, Cooper [Page 14]
Replication and Caching Taxonomy June 25, 1999
sophisticated client to replica communication mechanism.
This is accomplished by DNS servers that implement order
of addresses based upon quality of service policies. When
a web client resolves the name of a web server, the
enhanced DNS server orders the IP addresses of the web
server starting with the most optimal replica and ending
with the least optimal replica.
Security:
Relies entirely upon DNS security.
Deployment:
Observed at a number of large web sites and large ISP web
hosted services. Extent of usage in the Internet is
unknown at this time.
Submitter:
Document editors.
5. Inter-Replica Communication
This section describes the cooperation and communication between
replica origin servers. Used in replicating data sets between origin
servers.
5.1 Batch Driven Mirror Replication
Authoritative reference:
This memo.
Description:
In this model, the replica web server to be updated
initiates communication with a master origin web server.
The communication is established at intervals based upon
queued transactions which are scheduled for deferred
processing. The scheduling mechanism policies vary, but
generally are reoccuring at a specified time. Once
communication is established, data sets are copied to the
initiating replica web server.
Security:
Relies upon the protocol being used to transfer the data
set. FTP and RDIST are the most common protocols observed.
Deployment:
Very common for mirror synchronization in the Internet.
Melve, Tomlinson, Cooper [Page 15]
Replication and Caching Taxonomy June 25, 1999
Submitter:
Document editors.
5.2 Demand Driven Mirror Replication
Authoritative reference:
This memo.
Description:
In this model, the replica web server acquires the content
as needed due to demand. This is generally done by web
server accelerators (reverse proxy) operating as origin
server replicas. When a web client requests a URL that is
not in the data set or the replica origin server, the
replica server attempts to acquire it from a master origin
server and forwarded on to the requesting web client.
Security:
Relies upon the protocol being used to transfer the URLs.
FTP, Gopher, HTTP and ICP are the most common protocols
observed.
Deployment:
Observed at several large web sites. Extent of usage in
the Internet is unknown at this time.
Submitter:
Document editors.
5.3 Synchronized Replication
Authoritative reference:
This memo. [Ed note; there is no IETF protocol specified at
this time. The editors are aware of at least
two open source protocols, AFS and CODA, along
with one expired IETF draft
<draft-leach-cifs-v1-spec-01.txt> and one
proprietary protocol Novell NRS; none of which
can be considered an authoritative reference]
Description:
In this model, the replicated origin servers cooperate
using synchronized strategies and specialized replica
protocols to keep the replica data sets coherent.
Synchronization strategies range from tightly coherent (a
few minutes) to loosely coherent (a few or more hours).
Updates occur between replicas based upon the
Melve, Tomlinson, Cooper [Page 16]
Replication and Caching Taxonomy June 25, 1999
synchronization time constraints of the coherency model
employed and are generally in the form of deltas only.
Security:
All of the known protocols utilize strong cryptographic key
exchange methods, which are either based upon the Kerberos
shared secret model or the public/private key RSA model.
Deployment:
Observed at a few sites, primarily at university campuses.
Submitter:
Document editors.
6. Client to Proxy Configuration
This section describes the configuration, cooperation and
communication between end user clients (browsers and applications) a
proxy.
6.1 Manual Proxy Configuration
Authoritative reference:
This memo.
Description:
Each user needs to configure its web client by typing in
information pertaining to proxied protocols and local
policies.
Security:
The potential for doing wrong is high, as each user
individually sets preferences.
Deployment:
Widely deployed, used in all current browsers. Most
browsers support other options as well.
Submitter:
Document editors.
6.2 Proxy Auto Configuration (PAC) [7]
[Ed note: Does it really need to be submitted for Informational RFC?]
Authoritative reference:
Melve, Tomlinson, Cooper [Page 17]
Replication and Caching Taxonomy June 25, 1999
No RFC published, no Internet-Draft
Navigator Proxy Auto-Config File Format. Available from
http://home.netscape.com/eng/mozilla/2.0/
relnotes/demo/proxy-live.html
Description:
A JavaScript page on a web server hands out information on
where to find proxies. Clients need to point at the URL of
this page. No bootstrap mechanism, manual configuration
necessary.
Manual configuration is made easier by centralizing the
script to one URL.
Security:
Common policy per organization possible. Does still require
manual configuration. PAC is better than "manual proxy
configuration" because with PAC administrators can update
the proxy configuration without user intervention.
Interoperability of PAC files is not as good as wanted,
since more popular browsers have slightly different
interpretation of the script, and this may lead to
undesired effects.
Deployment:
Implemented in most web clients.
Submitter:
Document editors.
6.3 Cache Array Routing Protocol (CARP) v1.0 [9]
[Ed note: Current draft expired. A new draft must submitted and this
section completed for this protocol to be considered in the Taxonomy]
Authoritative reference:
Expired Internet-Draft draft-vinod-carp-v1-03.txt
Work in progress.
Description:
Clients may use CARP directly as a hash function based
proxy selection mechanism. They need to be configured with
the location of the cluster information.
Security:
Melve, Tomlinson, Cooper [Page 18]
Replication and Caching Taxonomy June 25, 1999
Deployment:
Submitter:
6.4 Web Proxy Auto-Discovery Protocol (WPAD) [8]
Authoritative reference:
Internet Draft <draft-ietf-wrec-wpad-00.txt>
[Ed note; I-D submission anticipated by 6/25/99]
Work in progress.
Description:
WPAD uses a collection of pre-existing Internet resource
discovery mechanisms to perform web proxy auto-discovery.
The only goal of WPAD is to locate the PAC URL. WPAD does
not specify which proxies will be used. WPAD gets you to
the PAC URL, and the PAC script chooses the proxies for
you.
The WPAD protocol specifies the following:
+ how to use each mechanism for the specific purpose of
web proxy auto-discovery
+ the order in which the mechanisms should be performed
+ the minimal set of mechanisms which must be attempted
by a WPAD compliant web client
The resource discovery mechanisms utilized by WPAD are as
follows:
+ Dynamic Host Configuration Protocol DHCP
+ Service Location Protocol SLP
+ "Well Known Aliases" using DNS A records
+ DNS SRV records
+ "service: URLs" in DNS TXT records
Security:
Relies upon DNS and HTTP security.
Deployment:
Implemented in web clients and caching proxy servers. More
than two independent implementations.
Submitter:
Melve, Tomlinson, Cooper [Page 19]
Replication and Caching Taxonomy June 25, 1999
Josh Cohen, Microsoft, joshco@microsoft.com
7. Inter-Cache Communication
[Ed note: INGRID. Review and chase submissions (push Duane)]
This section describes the cooperation and communication between
caching proxies.
7.1 Internet Cache Protocol (ICP) [10, 11, 12, 13, 14]
Authoritative reference:
RFC 2186 Internet Cache Protocol (ICP), version 2
Description:
ICP is used by caches to query other caches about web
objects, to see if a web object is present at the other
cache.
ICP uses UDP. Since UDP is unreliable, an estimate of
network congestion and availability may be calculated
by ICP loss. This rudimentary loss measurement does,
together with round trip times provide a load balancing
method for caches.
Security:
ICP does not convey information about HTTP headers
associated with a web object. HTTP headers may include
access control and cache directives, Since caches ask for
objects, and then download the objects using HTTP, false
cache hits may occur (object present in cache, but not
accessible for sibling cache is one example).
ICP suffer from all the security problems of UDP.
Deployment:
Widely deployed. Most current cache implementations support
ICP in one form or the other.
Submitter:
Document editors.
7.2 Hyper Text Caching Protocol (HTCP/0.0) [15]
[Ed note: Current draft expired. A new draft must submitted for this
protocol to be considered in the Taxonomy. Based upon reviewers
Melve, Tomlinson, Cooper [Page 20]
Replication and Caching Taxonomy June 25, 1999
comments, the editors would like to drop this protocol from current
Taxonomy consideration, due to its experimental nature]
Authoritative reference:
Expired Internet Draft draft-vixie-htcp-proto-03.txt,
Work in Progress
Description:
HTCP is a protocol for discovering HTTP caches and cached
data, managing sets of HTTP caches, and monitoring cache
activity.
HTCP includes HTTP headers, while ICPv2 does not. HTTP
headers are vital information for web proxy caches.
Security:
Optionally uses the MD5 shared secret authentication.
Lack of authentication option make protocol subject to
attack.
Deployment:
Implemented in caching proxies (two independent
implementations)
Submitter:
Document editors.
7.3 Cache Array Routing Protocol (CARP) v1.0 [9]
[Ed note: Current draft expired. A new draft must submitted and this
section completed for this protocol to be considered in the Taxonomy]
Authoritative reference:
Work in Progress: Internet-Draft draft-vinod-carp-v1-03.txt
Description:
CARP is a hashing function for dividing URL-space among a
cluster of proxy caches. Included in CARP is the definition
of a Proxy Array Membership Table, and ways to download
this information.
An HTTP client agent (either a proxy server or a client
browser) which implements CARP v1.0 can allocate and
intelligently route requests for the correct URLs to any
member of the Proxy Array. Due to the resulting sorting of
requests through these proxies, duplication of cache
contents is eliminated and global cache hit rates may be
improved.
Melve, Tomlinson, Cooper [Page 21]
Replication and Caching Taxonomy June 25, 1999
Security:
Deployment:
Implemented in caching proxy servers. More than two
independent implementations.
Submitter:
7.4 Cache Digest [16]
[Ed note: Does it really need to be submitted for Informational RFC?]
Authoritative reference:
No RFC published, no Internet-Draft
Cache Digest specification
http://squid.nlanr.net/Squid/CacheDigest/
cache-digest-v5.txt
Squid Digest FAQ entry
http://squid.nlanr.net/Squid/FAQ/FAQ-16.html
Description:
Cache Digests are a response to the problems of latency
and congestion associated with previous inter-cache
communications mechanisms such as the Internet Cache
Protocol (ICP) [10, 11] and the HyperText Cache Protocol
[15]. Unlike most of these protocols, Cache Digests
support peering between cache servers without a
request-response exchange taking place. Instead, a summary
of the contents of the server (the Digest) is fetched by
other servers which peer with it. Using Cache Digests it
is possible to determine with a relatively high degree of
accuracy whether a given URL is cached by a particular
server.
Cache Digests are both an exchange protocol and a data
format [16a,16b].
Security:
If the contents of a Digest is sensitive, it should be
protected from access by The Wrong People. Any methods
which would normally be applied to secure an HTTP
connection can be applied to Cache Digests.
A 'Trojan horse' attack is currently possible in a cache
mesh: Cache A can build a fake peer Digest for cache B and
serve it to B's peers if requested. This way A can direct
traffic toward/from B. The impact of this problem is
Melve, Tomlinson, Cooper [Page 22]
Replication and Caching Taxonomy June 25, 1999
minimized by the 'pull' model of transferring Cache
Digests from one server to another.
Cache Digests provide knowledge about peer cache content
on a URL level. Hence, they do not dictate a particular
level of policy management and can be used to implement
various policies on any level (user, organization, etc.).
Deployment:
Cache Digests are supported in Squid; several commercial
vendors are looking into Digest support.
Cache Meshes:
+ NLANR Mesh
+ TF-CACHE mesh (European Academic networks)
Submitter:
Alex Rousskov, NLANR, rousskov@nlanr.net
7.5 Cache Pre-filling [23]
Authoritative reference:
Internet Draft <draft-lovric-francetelecom-
satellites-00.txt>
Work in progress.
Description:
Cache pre-filling is a push-caching implementation. It is
particularly well adapted to IP-multicast networks because
it allows preselected URLs to be inserted in one single
time within all the caches that belong to the targeted
multicast group. Different implementations of cache
pre-filling already exist, especially in satellite
contexts. However, there is still no standard for this
kind of push-caching and vendors propose solutions either
based on dedicated equipments or public domain caches
extended with a pre-filling module.
Security:
Relies on the inter cache protocols being employed.
Deployment:
Observed in two commercial content distribution service
providers.
Submitter:
Ivan Lovric, France Telecom,
ivan.lovric@cnet.francetelecom.fr
Melve, Tomlinson, Cooper [Page 23]
Replication and Caching Taxonomy June 25, 1999
8. Network Element Communication
This section describes the cooperation and communication between
caching proxy and network elements. Examples include routers and
switches. Generally used for transparent caching and/or diffused
arrays.
8.1 Web Cache Coordination Protocol (WCCP)
Authoritative reference:
Internet Draft <draft-ietf-wrec-web-pro-00.txt> [18]
Work in progress.
Description:
WCCP V1 runs between a router functioning as a redirecting
network element and out-of-path transparent caching
proxies. The protocol allows one or more caching proxies
to register themselves with a single router to receive
redirected web traffic. It also allows one of the proxies,
the designated proxy, to dictate to the router how
redirected web traffic is distributed across the caching
proxies.
Security:
WCCP V1 has no security features.
Deployment:
Network elements: WCCP V1 is deployed on a wide range of
Cisco routers.
Caching proxies: WCCP V1 is deployed on a number of
vendors' caches.
Submitter:
David Forster, CISCO, dforster@cisco.com
8.2 Transparent Proxy Agent Control Protocol (TPACT)
Authoritative reference: [Ed note; anticipated submission]
Internet Draft <draft-ietf-wrec-tpact-00.txt> [22] [Ed
note; I-D submission anticipated by 6/25/99]
Work in progress.
Description:
TPACT runs between a network elements (router or switch)
functioning as a redirecting network element and
out-of-path transparent caching proxies. The protocol
Melve, Tomlinson, Cooper [Page 24]
Replication and Caching Taxonomy June 25, 1999
allows one or more caching proxies to register themselves
with a single network element to receive redirected web
traffic. All of the participating caching proxies operate
as a quorum in the diectating of web traffic distribution
across the group.
Security:
MD5 is optionally employed for authentication. Sequence
numbers are employed as security against replay attacks.
Deployment:
Network elements: TPACT is prototyped and being evaluated
on multiple vendor L4 switches.
Caching proxies: TPACT is prototyped and being evaluated
on multiple vendor caches.
Submitter:
John Martin, Network Appliance, jmartin@netapp.com
8.3 SOCKS [19]
Authoritative reference:
RFC1928 SOCKS Protocol Version 5
Description:
SOCKS is primarily used as a proxy cache to firewall
protocol. Although, firewalls don't conform to the
narrowly defined network element definition of routers and
switches, they are a integral part of the network
infrastructure. When used in conjunction with a firewall,
SOCKS provides a authenticated tunnel between the proxy
cache and the firewall.
Security:
A extensive framework provides for multiple authentication
methods. Currently, SSL, CHAP, DES, 3DES are known to be
available.
Deployment:
SOCKS is been widely deployed in the Internet.
Submitter:
Document editors.
9. Security Considerations
Melve, Tomlinson, Cooper [Page 25]
Replication and Caching Taxonomy June 25, 1999
[Ed note: INGRID. Send to list, more information needed]
Information on security in each protocol is provided in the
description of the protocol, and in the accompanying RFC for each
protocol.
Refer to section 15 in HTTP/1.1bis draft-ietf-http-v11-spec-
rev-06.txt
Man in the middle attacks
Refer to HTTP/1.1bis, chapter 15.7
HTTP proxies are men-in-the-middle, the perfect place for a man-in-
the-middle-attack.
Denial of service
Individual protocols
See documentation for each protocol for discussion of security
issues.
Trusted parties
You need to trust your proxy.
Stupid configuration
It is quite easy to have a stupid configuration which will harm
service for end users.
Privacy
Logs from proxies need to be kept secure, as they provide information
about users and end user patterns. A proxy log is even more
sensitive than a web server log, as all requests from the user
population goes through the proxy. Logs from replication servers may
need to be amalgamated to get aggregated statistics from a service,
transporting logs across borders may have legal implications. Log
handling is restricted by law in some countries.
Requirements for object security and privacy are the same in a web
replication and caching system as it is in the Internet at large.
The only reliable solution is strong cryptography. End to end
encryption does not necessarily make objects cacheable, as is the
Melve, Tomlinson, Cooper [Page 26]
Replication and Caching Taxonomy June 25, 1999
case of SSL encrypted web sessions.
Communication
Transient copies
The legislative forces of the world are still out on the question of
transient copies, like those kept in replication and caching system,
being legal. Legal implications of replication and caching is
subject to local law.
10. Acknowledgements
[Ed note: No decision made on authors list. Submitters of individual
entries are acknowledged in the text. Need to sort out how to give
credits where they are due.]
David Forster, Cisco, dforster@cisco.com provided info on Out-of-path
Transparent Caching Proxies.
Alex Rousskov, David Forster, Josh Cohen and John Martin for protocol
information.
John Dilley, Ivan Lovric and Joe Touch for terminology and taxonomy
information.
David Forster, Josh Cohen, Henrik Nordstrom and Patrick McManus for
their help in defining proxy transparency.
11. References
[1] Duane Wessels. Squid FAQ: Transparent Caching/Proxying.
National Laboratory for Applied Network Research. Available from:
http://squid.nlanr.net/Squid/FAQ/FAQ-17.html
[2] Peter Danzig and Karl L. Swartz. Transparent, Scalable, Fail-
Safe Web Caching. Network Appliance, Inc. Available from
http://www.netapp.com/technology/level3/3033.html
[3] Bert Williams. Transparent Web Caching Solutions. Alteon
Networks. Available from Transparent Web Caching Solutions
[4] Tony Hain. Architectural Implications of NAT. Internet
Architecture Board. Internet Draft (Work in Progress). Available from
ftp://ftp.nordu.net/internet-drafts/draft-iab-nat-implications-02.txt
Melve, Tomlinson, Cooper [Page 27]
Replication and Caching Taxonomy June 25, 1999
[5] Ingrid Melve, Lars Slettjord, Ton Verschuren, Henny Bekker,
Technical report European Union RE1004-M4.3 "Web caching
architecture"
[6] Fielding, et al. Hypertext Transfer Protocol -- HTTP/1.1. IETF
RFC2616. Available from http://www.rfc-editor.org/rfc/rfc2616.txt
[7] Netscape, Inc. Navigator Proxy Auto-Config File Format.
Available from
http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/proxy-
live.html
[8] Paul Gauthier, J. Cohen, Martin Dunsmuir and Charles Perkins.
The Web Proxy Auto-Discovery Protocol. Internet Draft. Available from
http://www.ietf.org/internet-drafts/draft-ietf-wrec-wpad-00.txt
[9] Vinod Valloppillil and Keith W. Ross. Cache Array Routing
Protocol. Internet Draft (Work in Progress) Available from
ftp://ftp.nordu.net/internet-drafts/draft-vinod-carp-v1-03.txt
[10] D. Wessels and K. Claffy. Internet Cache Protocol (ICP), version
2. 'RFC2186. Available from ftp://ftp.nordu.net/rfc/rfc2186.txt
[11] D. Wessels and K. Claffy. Application of Internet Cache Protocol
(ICP), version 2, RFC2187. Available from
ftp://ftp.nordu.net/rfc/rfc2187.txt
[12] Ivan Lovric. Internet Cache Protocol Extension Internet Draft
(Work in Progress) Available from ftp://ftp.nordu.net/internet-
drafts/draft-lovric-icp-ext-01.txt
[13] Duane Wessels. ICP Home Page, National Laboratory for Applied
Research. Available from [52]http://ircache.nlanr.net/Cache/ICP/
[14] University of Southern California. Internet Cache Protocol
Specification 1.4. Available from
http://excalibur.usc.edu/icpdoc/icp.html
[15] Paul Vixie and Duane Wessels. Hyper Text Caching Protocol
(HTCP/0.0). Internet Draft (Work in Progress) Available from
ftp://ftp.nordu.net/internet-drafts/draft-vixie-htcp-proto-03.txt
[16] Alex Rouskov and Duane Wessels. Cache Digests. National
Laboratory for Applied Network Research. Available from [16a] Cache
Digest specification http://squid.nlanr.net/Squid/CacheDigest/cache-
digest-v5.txt [16b] Squid Digest FAQ entry
http://squid.nlanr.net/Squid/FAQ/FAQ-16.html
Melve, Tomlinson, Cooper [Page 28]
Replication and Caching Taxonomy June 25, 1999
[17] Berners-Lee, et al. Hypertext Transfer Protocol -- HTTP/1.0 IETF
RFC1945 Available from http://www.rfc-editor.org/rfc/rfc1945.txt
[18] Cisco Web Cache Coordination Protocol V1.0. Internet Draft.
Available from http://www.ietf.org/internet-drafts/draft-ietf-wrec-
web-pro-00.txt
[19] Leech, et al. SOCKS Protocol Version 5, RFC1928 Available from
http://www.rfc-editor.org/rfc/rfc1928.txt
[20] Keith Moore, On the use of HTTP as a Substrate for Other
Protocols. Internet Draft (Work in Progress) Available from
ftp://ftp.nordu.net/internet-drafts/draft-iesg-using-http-00.txt
[21] Brisco, T. DNS Support for Load Balancing. RFC1794. Available
from http://www.rfc-editor.org/rfc/rfc1794.txt
[22] Cerpa, et al. Transparent Proxy Agent Control Protocol.
Internet Draft. Available from http://www.ietf.org/internet-
drafts/draft-ietf-wrec-tpact-00.txt
[23] Goutard, et al. Pre-filling a cache - A satellite overview.
Internet Draft. Available from http://www.ietf.org/internet-drafts/
draft-lovric-francetelecom-satellites-00.txt
12. Authors' Addresses
Ingrid Melve
UNINETT
Tempeveien 22, Trondheim, NORWAY
Phone: +47 73 55 79 07
Email: Ingrid.Melve@uninett.no
Gary Tomlinson
Novell, Inc.
122 East 1700 South
Provo, Utah 84606 USA
Phone: +1 801 861 7021
Email: garyt@novell.com
Ian Cooper
Mirror Image Internet, Inc.
18 Commerce Way, Suite 4800
Woburn, MA 01801 USA
Phone: +1 800 353 2923
Email: ian@mirror-image.com
Melve, Tomlinson, Cooper [Page 29]
