James Kempf
Internet Draft Rob Austein
Document: draft-iab-e2e-futures-00.txt IAB
Expires: July 2003 January 2003
The Rise of the Middle and the Future of End to End:
Reflections on the Evolution of the Internet Architecture
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Abstract
The end to end principle is the core architectural principle of the
Internet. In this document, we briefly examine the development of the
end to end principle as it has been applied to the Internet
architecture over the years. We discuss current trends in the evolution
of the Internet architecture in relation to the end to end principle,
and try to draw some conclusion about the evolution of the end to end
principle, and thus for the Internet architecture which it supports, in
light of these current trends.
Table of Contents
1.0 Introduction.................................................2
2.0 A Brief History of End to End................................2
3.0 Trends Opposing to End to End................................5
4.0 Whither End to End?..........................................7
5.0 Internet Standards as an Arena for Conflict..................9
6.0 Conclusions..................................................9
7.0 Acknowledgements............................................10
Kempf and Austein Expires July 2003 [Page 1]
Internet Draft Future of End to End November, 2002
8.0 References..................................................10
9.0 Security Considerations.....................................11
10.0 IANA Considerations........................................11
11.0 Author Information.........................................11
12.0 Full Copyright Statement...................................11
1.0 Introduction
One of the key architectural principles of the Internet is the end to
end principle [1][2]. The end to end principle was originally
articulated as a question of where best not to put functions in a
communication system. Yet, in the ensuing years, it has evolved to
address concerns of maintaining openness, increasing reliability and
robustness, and preserving the properties of user choice and ease of
new service development [3]; concerns that were not part of the
original articulation of the end to end principle.
In this document, we examine how the interpretation of the end to end
principle has evolved over the years, and where it stands currently. We
examine trends in the development of the Internet that have lead to
pressure to define services in the network, a topic that has already
received some amount of attention from the IAB [4]. We describe some
considerations about how the end to end principle might evolve in light
of these trends.
2.0 A Brief History of End to End
2.1 In the Beginnning...
The end to end principle was originally fairly broadly articulated as a
question of where best not to put functions in a communication system:
The function in question can completely and correctly be implemented
only with the knowledge and help of the application standing at the
end points of the communication system. Therefore, providing that
questioned function as a feature of the communication system itself
is not possible. (Sometimes an incomplete version of the function
provided by the communication system may be useful as a performance
enhancement.) [1].
However, the specific examples given in [1] and other references at the
time [2] primarily involve transmission of data packets: data
integrity, delivery guarantees, duplicate message suppression, per
packet encryption, and transaction management. From the viewpoint of
today's Internet architecture, we would view most of these as transport
layer functions (data integrity, delivery guarantees, duplicate message
suppression, and perhaps transaction management), others as network
layer functions with support at other layers where necessary (for
example, packet encryption). The authors of [1] cite a few other
application areas, outside of the specific area of communication system
design, where the end to end principle seems to apply, including RISC
architectures.
Kempf and Austein Expires July 2003 [Page 2]
Internet Draft Future of End to End November, 2002
Interestingly, the expression of the end to end principle cited above
is phrased as a negative: what should *not* be provided in a
communication system rather than what should be provided. Much of the
wider applicability later attributed to the end to end principle,
outside of the original application to the transport of packets,
derives from this phrasing.
The end to end principle itself only provides the designer with
guidance about what not to do. If a function is not supplied in the
communication system, then it must be supplied in the application. In
order to avoid duplicating functions in the application, the primitives
in the communication system must provide the right set of non-
overlapping functions (orthogonal primitives) upon which the
application can be based. Though the principle of orthogonal primitives
is not explicitly cited in [1], its influence is apparent throughout
the examples.
2.2 ...In the Middle...
As the Internet developed, the end to end principle gradually widened
to concerns about where best to put services in the Internet: in the
network or at end nodes. The best example is the description in RFC
1958:
This principle has important consequences if we require applications
to survive partial network failures. An end-to-end protocol design
should not rely on the maintenance of state (i.e. information about
the state of the end-to-end communication) inside the network. Such
state should be maintained only in the endpoints, in such a way that
the state can only be destroyed when the endpoint itself breaks
(known as fate-sharing). An immediate consequence of this is that
datagrams are better than classical virtual circuits. The network's
job is to transmit datagrams as efficiently and flexibly as
possible. Everything else should be done at the fringes.[5]
The original broad articulation of the end to end principle as where
not to define functions in a communication system took a while to
percolate through the engineering community, and had evolved by this
point to an explicit architectural statement (but still quite broad)
about what belongs in the network and what doesn't. Functions that
require maintaining application state belong in the applications.
"Fate-sharing" describes this quite clearly: the fate of a conversation
between two applications is only shared between the two applications;
the fate does not depend on anything in the network, except for the
network's ability to get packets from one application to the other.
The end to end principle in this formulation is specifically about what
kind of state is maintained where:
To perform its services, the network maintains some state
information: routes, QoS guarantees that it makes, session
information where that is used in header compression, compression
histories for data compression, and the like. This state must be
self-healing; adaptive procedures or protocols must exist to derive
Kempf and Austein Expires July 2003 [Page 3]
Internet Draft Future of End to End November, 2002
and maintain that state, and change it when the topology or activity
of the network changes. The volume of this state must be minimized,
and the loss of the state must not result in more than a temporary
denial of service given that connectivity exists. Manually
configured state must be kept to an absolute minimum.[5]
In this formulation of the end to end principle, state involved in
getting packets from one end of the network to the other is maintained
in the network. The state is "soft state," in sense that it can be
quickly dropped and reconstructed (or even required to be periodically
renewed) as the network topology changes due to routers and switches
going on and off line.
In summary, the general awareness both of the principle itself and of
its implications for how unavoidable state should be handled grew over
time to become a (if not the) foundation principle of the Internet
architecture.
2.3 ...And Now.
An interesting example of how end to end continues to influence the
technical debate in the Internet community is route optimization in
Mobile IPv6 [6]. This is a specific example in which the end to end
principle has been applied to something that is the quintessential
network function, namely routing.
Route optimization in Mobile IPv6 requires a moving mobile node to send
routing updates to its correspondents when the mobile node moves from
one subnet to another. These routing updates allow the correspondent to
match the mobile node's home network address to its current care-of
address. Packets originally addressed to the home address by the
transport layer can be routed by the correspondent's Mobile IP stack
directly to the mobile node's care-of address, avoiding the overhead
and latency of tunneling through the home agent. A consequence of this
design is that any IPv6 node in the Internet (not just a mobile node)
that wants to participate in route optimization must maintain a host
route table for those mobile nodes with which it is in conversation.
This design extends host-maintained routing from end to end
communication in the subnet to end to end communication across the
Internet.
The vision emerging out of the IETF working groups developing standards
for mobile networking is of a largely autonomous mobile node with
multiple wireless link options among which the mobile node picks and
chooses, a kind of innovative application of end to end that derives
from the same basic considerations of reliability and robustness
(wireless link integrity, changes in connectivity and service
availability with movement, etc.) which motivated the original
development of end to end. Route optimization is one example. The
mobile node's autonomy is supported by having the mobile node itself
take care of link selection and routing, expanding the number of multi-
homed hosts by several orders of magnitude. Whether this vision will
provide adequate performance without more network support is currently
an open question (and see Section 5.0); but as the last line in the
Kempf and Austein Expires July 2003 [Page 4]
Internet Draft Future of End to End November, 2002
original formulation of the end to end principle states, sometimes
performance enhancements within the network may be necessary. This new
application of the end to end principle suggests that the principle is
still a relevant consideration in the design of new technical
applications.
3.0 Trends Opposing to End to End
While the situation with mobile computing looks encouraging, the
specific application of end to end described in RFC 1958 has
increasingly come into question from people who want to locate
services, together with their state, in the network. In particular the
recently published IAB opinion on Open Pluggable Edge Services (OPES)
[4] was intended to assess the architectural desirability of defining
services in the network and to raise questions about how such services
might result in compromises of the end to end principle. The topic of
service definition the network is also taken up in [7] and [8].
Perhaps the best review of the forces militating against end to end is
[3]. The authors make the point that the Internet originally developed
among a community of like-minded technical professionals who trusted
each other, and was administered by academic and government
institutions who enforced a policy of no commercial use. The major
stakeholders in the Internet are quite different today. As a
consequence, new requirements have evolved over the last decade.
Examples of these requirements are discussed in the following
subsections. Other discussions about pressures on the end to end
principle in today's Internet can be found in [9] and [10].
3.1 Lack of Trust
Perhaps the single most important change from the Internet of 15 years
ago is the lack of trust between end nodes. Because the end users in
the Internet of 15 years ago were few, and were largely dedicated to
using the Internet as a tool for computer science research and for
communicating research results, trust between end users (and thus
between the end nodes that they use) was simply not an issue in
general. Today, the motivations of some individuals using the Internet
are not always entirely ethical, and, even if they are, the assumption
that end nodes will always co-operate to achieve some mutually
beneficial action, as implied by the end to end principle, is not
always accurate. One of the most common examples of network elements
interposing between end hosts are those dedicated to security:
firewalls, VPN tunnel endpoints, certificate servers, etc. These
intermediaries are designed to either protect the network from
unimpeded attack or to allow two end nodes that may have no inherent
reason to trust each other to achieve some level of trust. Trusted
intermediaries are a major example of services defined in the network.
3.2 New Service Models
New service models inspired by new applications require achieving the
proper performance level as a fundamental part of the delivered
service. These service models are a significant change from the
Kempf and Austein Expires July 2003 [Page 5]
Internet Draft Future of End to End November, 2002
original best effort service model. Email, file transfer, and even Web
access aren't perceived as failing if performance degrades, though the
user may become frustrated at the time required to complete the
transaction. However, for streaming audio and video, to say nothing of
real time bidirectional voice and video, achieving the proper
performance level is part of delivering the service, and a customer
contracting for the service has a right to expect the level of
performance for which they have contracted. Designs for achieving
additional performance in the Internet often require network elements
that maintain some kind of state involving the real time flow, for
example, caching streaming media content locally so it can be delivered
more quickly.
3.3 Rise of the Third Party
The Internet of 10 years ago was run by academic and government
institutions. These institutions did not expect to make a profit from
their investment in networking technology. In contrast, the network
operator with which most Internet users deal today is the ISP. ISPs run
their networks as a business, and expect to make a profit (or at least
not lose much) on their investment in the network. While this radical
change in business model is not an excuse for modifying an
architectural principle that has exhibited its value over time, it does
put a certain amount of pressure on the end to end principle.
In particular, because an ISP delivers a commodity service, the profit
margins on basic bandwidth provision for a best effort service bit
pipe, together with the email and Web access services that are
typically bundled with bit pipe service, are fairly low. As a result,
ISPs would like to differentiate themselves by providing some services
within their networks. This desire is being met by new hardware that
allows routers to perform line speed examination of IP flows, primarily
for purposes of billing, but the capability is also available for
defining other services. An example is enhanced content delivery
performance. Many ISPs today use caching services to increase the
performance of Web page delivery, and caching services for streaming
media are also under discussion. These services are typically deployed
so that they are only accessible within the ISPs network, and as a
result, they do not contribute to open, end to end service.
ISPs are not the only third party intermediary that has appeared within
the last 10 years. Unlike the previous involvement of corporations and
governments in running the Internet, corporate network administrators,
and governmental officials have become increasingly demanding of
opportunities to interpose between two parties in an end to end
conversation. A benign motivation for this involvement is to mitigate
the lack of trust, so the third party acts as a trust anchor or
enforcer of good behavior between the two ends. A less benign
motivation is for the third party to insert policy for their own
reasons, perhaps taxation or even censorship. The requirements of third
parties often have little or nothing to do with technical concerns, but
rather derive from particular social and legal considerations.
Kempf and Austein Expires July 2003 [Page 6]
Internet Draft Future of End to End November, 2002
3.4 The Consumer as the Primary User
The original users of the Internet were technologists who understood
how it worked and had no reservations about tinkering with the software
or hardware on their end hosts. The users of today are primarily
consumers who buy packaged hardware and software from vendors and
contract with ISPs for service. They expect their Internet service to
function smoothly like any other product they buy, without much
involvement on their part in keeping the product functional. This
development matches closely the development of other technologies, for
example the automobile, that have become mainstream.
This pressure to simplify the user experience has resulted in a
corresponding pressure to reduce the amount of installation,
configuration, maintenance, and upgrade on end nodes. Requiring user
involvement in the deployment of new software or hardware on the end
nodes, in order to deploy new services, runs directly counter to this
trend. One response has been the tendency to move deployment of new
services to servers running an existing protocol, such as HTTP, or
downloadable code, such as Java or browser plug-ins, which don't
require any user involvement to install. Another response has been
network intermediaries to provide the service. Typically, these
intermediaries don't interpose on a flow between a client and a server,
but they may act more like DNS, in that the intermediary is required in
order to get access to the service. A further development of this trend
would be to move much of the context and configuration for a user into
a node in the network, where it can be upgraded without any user
involvement. This development would remove the end host as the
definitive location for the application and spread it out between the
network and the end host.
4.0 Whither End to End?
Given the pressures on end to end discussed in the previous section, a
question arises about the future of end to end. Does end to end have a
future in the Internet architecture or not? If it does have a future,
how should it be applied? Clearly, an unproductive approach to
answering this question is to insist upon end to end as a
fundamentalist principle that allows no compromise. The pressures
described above are real and powerful, and if the current Internet
technical community chooses to ignore these pressures, the likely
result is that a market opportunity will be created for a new technical
community that does not ignore these pressures but which may not
understand the implications of their design choices. A more productive
approach is to return to first principles and re-examine what end to
end is trying to accomplish, and then update our definition and
exposition of the end to end principle given the complexities of the
Internet today.
In the next two subsections, we consider the two basic motivations for
end to end: protecting innovation and providing reliability and
robustness.
Kempf and Austein Expires July 2003 [Page 7]
Internet Draft Future of End to End November, 2002
4.1 Protecting Innovation
One motivation underlying continued application of the end to end
principle is to protect innovation. Requiring modification in the
network in order to deploy new services is still typically more
difficult than modifying end nodes. The counterargument in Section 3.4
- that many end nodes are now essentially closed boxes that are not
updatable and that most users don't want to update them anyway - does
not apply to all nodes and all users. Many end nodes are still user
configurable and a sizable percentage of users are "early adopters,"
who are willing to put up with a certain amount of technological grief
in order to try out a new idea. Requiring someone with a new idea for a
service to convince a bunch of ISPs to modify their networks is much
more difficult than simply putting up a Web page with some downloadable
software implementing the service. End to end design thus remains a
viable way of fostering the service innovation that has been so
important in making the Internet a useful tool in people's lives.
At the same time, the pressure for lots of different parties to get
involved in existing, successful applications will be irresistible [7].
Application service providers, ISPs, corporate networks, and others
will want to provide special performance, reliability, and add-on
services. Such pressure is fueling the desire for OPES as an adjunct to
existing Web services. The result of all this additional complexity in
the network is likely to move the services away from their original
simplicity, and foster interoperability and other problems. About the
best one can hope for is to provide protocol and application designers
with guidance in maintaining end user choice, so that the basic end to
end principle is maintained. To a large extent, the IAB OPES [4] work
is directed at providing such guidance.
4.2 Reliability and Robustness
The second motivation for continued application of the end to end
principle is to increase the reliability and robustness of the exchange
between the two parties in the conversation. During the early
development of the Internet, the basic reliability of the hardware and
software was fairly low, so involving additional network elements
between the two ends could radically decrease the reliability of the
overall connection. Technical reliability has improved considerably,
but reliability due to involvement of network elements is still a
concern [4].
Of more concern today, however, is the decrease in reliability and
robustness that results from deliberate, active attacks on the network
infrastructure and end nodes. While the original developers of the
Internet were concerned by large scale system failures, attacks of the
subtlety and variety that the Internet experiences today were not a
problem during the original development of the Internet. By and large,
the end to end principle was not addressed to the decrease in
reliability resulting from subtlety engineered attacks. These attacks
are part of the larger issue of the trust breakdown discussed in
Section 3.1. Thus, the issue of the trust breakdown can be considered
Kempf and Austein Expires July 2003 [Page 8]
Internet Draft Future of End to End November, 2002
another forcing function on the Internet architecture, similar to the
issue of reliability and robustness due to technical.
The immediate reaction to this trust breakdown has been to try to back
fit security into existing protocols. While this effort is necessary,
it is not enough. The issue of trust must become as firm an
architectural principle in protocol design for the future as the end to
end principle is today. Trust isn't simply a matter of adding some
cryptographic protection to a protocol after it is designed. Rather,
prior to designing the protocol, the trust relationships between the
network elements involved in the protocol must be defined, and
boundaries must be drawn between those network elements that share a
trust relationship. The trust boundaries should be used to determine
what type of signaling occurs between the network elements involved in
the protocol and which network elements signal each other. When
signaling occurs across a trust boundary, cryptographic or other
security protection of some sort may be necessary. Additional measures
may be necessary to secure the protocol when communicating network
elements do not share a trust relationship. For example, a protocol may
need to minimize state in the recipient prior to establishing the
validity of the credentials from the sender in order to avoid a memory
depletion DoS attack. The end to end principle is insufficient to cover
this case, because the recipient is an end node.
5.0 Internet Standards as an Arena for Conflict
Internet standards have increasingly become an arena for conflict [7].
ISPs have particular business concerns, and their concerns drive some
of the pressure for defining services in the network, as described in
Section 3.3. Businesses and government have other concerns, and vendors
of networking hardware and software still others. For example, as
discussed in Section 2.3, the trend in the IETF mobile networking
standards is to apply the end to end principle quite radically, to
enable an autonomous mobile host. This trend strengthens the important
properties of choice and user empowerment. Yet, the business pressure
for mobile network operators is to hold on to the customer as long as
possible, in order to derive maximum revenue from the customer. The
result is a conflict between the desire of the ISP to obtain revenue to
run their business and the desires of the users for choice.
These conflicts will inevitably be reflected in the Internet
architecture going forward. Some of these conflicts are impossible to
resolve on a technical level, nor would it even be desirable, because
they involve social and legal choices that the IETF is not empowered to
make (for a counter argument in the area of privacy, see [11]). But for
those conflicts that do involve technical choices, the important
properties of user choice and empowerment, reliability and integrity of
end to end service, supporting trust and "good network citizen
behavior"," and fostering innovation in services should be the basis
upon which resolution is made. The conflict will then play out on the
field of the resulting architecture.
6.0 Conclusions
Kempf and Austein Expires July 2003 [Page 9]
Internet Draft Future of End to End November, 2002
The end to end principle continues to guide technical development of
Internet standards, and remains as important today for the Internet
architecture as in the past. While the end to end principle originated
as a focused argument about where best not to put functions in a
communication system, particular properties developed by the Internet
as a result of the end to end principle have come to be recognized as
being as important, if not more so, than the principle itself.
Protection of innovation, end user choice and empowerment, reliability,
integrity of service, support for trust, and "good network citizen
behavior" are all properties that have developed as a consequence of
the end to end principle. Recognizing these properties in a particular
proposal for modifications to the Internet has become more important
than before as the pressures to incorporate services into the network
have increased. Any proposal to incorporate services in the network
should be weighed against these properties before proceeding.
7.0 Acknowledgements
Many of the ideas presented here originally appeared in the works of
Dave Clark, John Wroclawski, Bob Braden, Karen Sollins, Marjory
Blumenthal, and Dave Reed on forces currently influencing the evolution
of the Internet. The authors would particularly like to single out the
work of Dave Clark, who was the original articulator of the end to end
principle and who continues to inspire and guide the evolution of the
Internet architecture, and John Wroclawski, with whom conversations
during the development of this paper helped to clarify issues involving
tussle and the Internet.
8.0 References
[1] Saltzer, J.H., Reed, D.P., and Clark, D.D., "End to End
Arguments in System Design," Communications Policy in
Transition: The Internet and Beyond, B. Compaine and S.
Greenstein, eds. MIT Press, September 2001.
[2] Clark, D., "The Design Philosophy of the DARPA Internet
Protocols," Proc SIGCOMM 88, ACM CCR Vol 18, Number 4, August
1988, pp. 106-114.
[3] Blumenthal, M., Clark, D.D., "Rethinking the design of the
Internet: The end to end arguments vs. the brave new world",
ACM Transactions on Internet Technology, Vol. 1, No. 1, August
2001, pp 70-109.
[4] Floyd, S., and Daigle, L., "IAB Architectural and Policy
Considerations for Open Pluggable Edge Services", RFC 3238,
January 2002.
[5] Carpenter, B., "Architectural Principles of the Internet," RFC
1958, June, 1996.
[6] Johnson, D., Perkins, C., and Arkko, J., "Mobility Support in
IPv6," draft-ietf-mobileip-ipv6-17.txt, a work in progress.
[7] Clark, D.D., Wroclawski, J., Sollins, K., and Braden, B.,
"Tussle in Cyberspace: Defining Tommorow's Internet",
Proceedings of Sigcomm 2002.
[8] Carpenter, B., and Brim, S., "Middleboxes: Taxonomy and
Issues," RFC 3234, February, 2002.
Kempf and Austein Expires July 2003 [Page 10]
Internet Draft Future of End to End November, 2002
[9] Reed, D., "The End of the End-to-End Argument?",
http://www.reed.com/dprframeweb/dprframe.asp?section=paper&fn=e
ndofendtoend.html, April, 2000.
[10] Moors, T., "A Critical Review of End-to-end Arguments in System
Design," Proc. 2000 IEEE International Conference on
Communications, pp. 1214-1219, April, 2002.
[11] Goldberg, I., Wagner, D., and Brewer, E., "Privacy-enhancing
technologies for the Internet," Proceedings of IEEE COMPCON 97,
pp. 103-109, 1997.
9.0 Security Considerations
This document does not propose any new protocols, and therefore does
not involve any security considerations in that sense. However,
throughout this document there are discussions of the privacy and
integrity issues and the architectural requirements created by those
issues.
10.0 IANA Considerations
There are no IANA considerations regarding this document.
11.0 Author Information
Internet Architecture Board
EMail: iab@iab.org
IAB Membership at time this document was completed:
Harald Alvestrand
Ran Atkinson
Rob Austein
Fred Baker
Leslie Daigle
Steve Deering
Sally Floyd
Ted Hardie
Geoff Huston
Charlie Kaufman
James Kempf
Eric Rescorla
Mike St. Johns
This draft was created in January 2003.
12.0 Full Copyright Statement
Copyright (C) The Internet Society (2001). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
Kempf and Austein Expires July 2003 [Page 11]
Internet Draft Future of End to End November, 2002
kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English. The limited permissions granted above are perpetual and
will not be revoked by the Internet Society or its successors or
assigns. This document and the information contained herein is
provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE
INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Kempf and Austein Expires July 2003 [Page 12]
